BatchRequest
Batch insert, delete, and transform operations in a single request. **Atomicity**: - **Single shard**: Operations are atomic within shard boundaries - **Multiple shards**: Uses distributed 2-phase commit (2PC) for atomic cross-shard writes **How distributed transactions work**: 1. Metadata server allocates HLC timestamp and selects coordinator shard 2. Coordinator writes transaction record, participants write intents 3. After all intents succeed, coordinator commits transaction 4. Participants are notified asynchronously to resolve intents 5. Recovery loop ensures notifications complete even after coordinator failure **Performance**: - Single-shard batches: < 5ms latency - Cross-shard transactions: ~20ms latency - Intent resolution: < 30 seconds worst-case (via recovery loop) **Guarantees**: - All writes succeed or all fail (atomicity across all shards) - Coordinator failure is recoverable (new leader resumes notifications) - Idempotent resolution (duplicate notifications are safe) **Benefits**: - Reduces network overhead compared to individual requests - More efficient indexing (updates are batched) - Automatic distributed transactions when operations span shards The inserts are upserts - existing keys are overwritten, new keys are created.
Artificial IntelligenceSearchRetrievalVectorsHybrid
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://artifacts.apis.io/antfly/json-schema/antfly-batch-request-schema.json",
"title": "BatchRequest",
"description": "Batch insert, delete, and transform operations in a single request.\n\n**Atomicity**:\n- **Single shard**: Operations are atomic within shard boundaries\n- **Multiple shards**: Uses distributed 2-phase commit (2PC) for atomic cross-shard writes\n\n**How distributed transactions work**:\n1. Metadata server allocates HLC timestamp and selects coordinator shard\n2. Coordinator writes transaction record, participants write intents\n3. After all intents succeed, coordinator commits transaction\n4. Participants are notified asynchronously to resolve intents\n5. Recovery loop ensures notifications complete even after coordinator failure\n\n**Performance**:\n- Single-shard batches: < 5ms latency\n- Cross-shard transactions: ~20ms latency\n- Intent resolution: < 30 seconds worst-case (via recovery loop)\n\n**Guarantees**:\n- All writes succeed or all fail (atomicity across all shards)\n- Coordinator failure is recoverable (new leader resumes notifications)\n- Idempotent resolution (duplicate notifications are safe)\n\n**Benefits**:\n- Reduces network overhead compared to individual requests\n- More efficient indexing (updates are batched)\n- Automatic distributed transactions when operations span shards\n\nThe inserts are upserts - existing keys are overwritten, new keys are created.\n",
"x-generated": "2026-11-10",
"x-method": "derived",
"x-generator": "derive-json-schema.py",
"x-source": "openapi/antfly-data-operations-api-openapi.yml#/components/schemas/BatchRequest",
"type": "object",
"properties": {
"inserts": {
"type": "object",
"description": "Map of document IDs to document objects. Each key is the unique identifier for the document.\n\nBest practices:\n- Use consistent key naming schemes (e.g., \"user:123\", \"article:456\")\n- Key length affects storage and performance - keep them reasonably short\n- Keys are sorted lexicographically, so choose prefixes that support range scans\n",
"additionalProperties": {
"type": "object",
"description": "Document object with arbitrary fields matching your table schema."
}
},
"deletes": {
"type": "array",
"description": "Array of document IDs to delete. Documents are removed from all indexes.\n\nNotes:\n- Non-existent keys are silently ignored\n- Deletions take precedence over inserts for the same key in one batch\n- Keys are permanently removed from storage and indexes\n",
"items": {
"type": "string"
}
},
"transforms": {
"type": "array",
"description": "Array of transform operations for in-place document updates using MongoDB-style operators.\n\nTransform operations allow you to modify documents without read-modify-write races:\n- Operations are applied atomically on the server\n- Multiple operations per document are applied in sequence\n- Supports numeric and set-like operations ($inc, $min, $max, $addToSet, $pull)\n\nCommon use cases:\n- Increment counters (views, likes, votes)\n- Update timestamps ($set)\n- Add or remove array values ($addToSet, $pull)\n- Update nested fields without overwriting the entire document\n",
"items": {
"$ref": "#/$defs/Transform"
}
},
"sync_level": {
"$ref": "#/$defs/SyncLevel"
}
},
"$defs": {
"SyncLevel": {
"type": "string",
"enum": [
"propose",
"write",
"full_text",
"enrichments",
"full_index"
],
"default": "propose",
"description": "Synchronization level for batch operations:\n- \"propose\": Wait for Raft proposal acceptance (fastest, default)\n- \"write\": Wait for the write to be durably applied to the local key-value store\n- \"full_text\": Wait for full-text index WAL write\n- \"enrichments\": Precompute enrichments before committing the document. A synchronous\n producer failure rejects the write; post-commit worker failures retain the document\n and may return `committed_repair_required`.\n- \"full_index\": Wait for all index writes to complete (full-text + enrichments + vector indexes)\n"
},
"Transform": {
"type": "object",
"required": [
"key",
"operations"
],
"description": "In-place document transformation using MongoDB-style operators. Transforms are applied atomically\nat the storage layer, eliminating read-modify-write races.\n\n**Important:** Transform results are NOT validated against the table schema. This improves performance\nbut means it's possible to create invalid documents. Use with care and ensure your operations maintain\nschema compliance.\n",
"properties": {
"key": {
"type": "string",
"description": "Document key (must be a string, not an object like inserts)"
},
"operations": {
"type": "array",
"items": {
"$ref": "#/$defs/TransformOp"
},
"description": "List of operations to apply in sequence"
},
"upsert": {
"type": "boolean",
"default": false,
"description": "If true, create document if it doesn't exist (like MongoDB upsert)"
}
}
},
"TransformOp": {
"type": "object",
"required": [
"op",
"path"
],
"properties": {
"op": {
"$ref": "#/$defs/TransformOpType"
},
"path": {
"type": "string",
"description": "JSONPath to field (e.g., \"$.user.name\", \"$.tags\", or \"user.name\")"
},
"value": {
"description": "Value for operation (not required for $unset). Type depends on the supported operator. `$pull` removes every array element exactly equal to this JSON value; projected graph-edge values identify the relationship by `target`."
}
}
},
"TransformOpType": {
"type": "string",
"enum": [
"$set",
"$setOnInsert",
"$unset",
"$inc",
"$push",
"$pull",
"$addToSet",
"$min",
"$max"
],
"x-enum-varnames": [
"SET",
"SET_ON_INSERT",
"UNSET",
"INC",
"PUSH",
"PULL",
"ADD_TO_SET",
"MIN",
"MAX"
],
"description": "MongoDB-style update operator"
}
}
}
Every JSON Schema here is available over the APIs.io API and to AI agents over MCP.