CData benchmarks MCP gateways on cost per correct task, and publishes the harness

CData benchmarks MCP gateways on cost per correct task, and publishes the harness

CData has published a benchmark of two MCP gateways serving the same business context to probabilistic agents. Its headline: through CData Connect AI Gateway, 27 models reached correct answers with “about a tenth of the tool calls,” at $0.26 per correct answer against $5.67 through an open-source MCP gateway. The study covered 810 runs by models from 12 developers on three account-health tasks across a CRM, a data warehouse and an ITSM system. The last task was a write, adding exactly the right 21 accounts to a review queue, where “a wrong write lands in a live system.” The metric is the useful part. Cost per correct task is total spend divided by tasks done right, because “failed runs still cost money.”

All of these figures come from a vendor benchmarking its own product against an unnamed competitor, and they should be read that way. The headline 21x rests on completion rates of 83% against 36%, and total spend of $89 against $828 over 405 tasks a side. The median run took 47 tool calls on the open-source gateway and 8 on CData’s. Economy models completed 78% of tasks at $0.06 each on Connect AI and 17% at $1.48 on the other. Two things hold up apart from the marketing. First, CData published the benchmark harness so anyone can rerun it with their own data, and few vendor benchmarks do that. Second, the mechanism is plausible: the open-source run lost because “the model did the work itself,” paging and joining in its own reasoning. A gateway that runs the query costs fewer tokens than a model that reconstructs the join.

The catalog has the product in the story. The CData provider page lists 14 API pages, led by the CData Connect AI MCP Server. The gateway’s “one query across all three systems” is the CData Query API, its system context is served by the CData Metadata API, and the live sources are defined in the CData Connection API. The agentic-access profile counts 76 operations: 43 act on data and 1 is flagged human-in-the-loop.

CData has a Kin Score of 83.8, in the exemplar band, with operational transparency at 92.1 and access clarity at 86.8. Contract governance at 35.6 is its weakest facet. Its Agent Readiness of 67.3 is high. Idempotency, delegated identity, error semantics and dynamic client registration are all lit, and that suits a gateway that executes queries on an agent’s behalf. Two dimensions are unlit: dry-run mode and reversibility documentation. Those are the ones the benchmark’s third task needed. CData defines a governed result as the agent writing “the right records and nothing it shouldn’t have,” but nothing in the published contract lets an agent preview a write or undo one.

← Agorapulse's MCP ships without a publish-now button, on purpose
Central and Eastern Europe has 77 providers and its headquarters signal missed the one that leads →