established · 1.4 capability resource

Evaluation

Score 24.7 / 100 49 providers +41 via APIs 60 APIs Search apis.io →
Business capabilities this tag reaches
Information & Data Management 3 Software Product Engineering Management 1 Software Release & Deployment Management 1
Variants seen in the corpus: EvaluationsEvaluation

Providers using this tag (49)

Ranked by API Evangelist rating — Exemplar and Strong are expanded by default.

Strong 6 Solid coverage with minor gaps
CometCompanyAi Enterprise SoftwareLLM ObservabilityLLMOps1 API62.7RunloopArtificial IntelligenceAI AgentsCoding AgentsSandbox14 APIs59.6CovalCompanyAI AgentsVoice AITesting20 APIs57.3BluejayCompanyArtificial IntelligenceAI AgentsVoice AI1 API56.7VijilCompanyArtificial IntelligenceAI AgentsAgent Security2 APIs55.8Prime IntellectArtificial IntelligenceReinforcement LearningGPU ComputeDecentralized Compute3 APIs55.6
Developing 11 Usable, with meaningful gaps to close
ScorecardCompanyArtificial IntelligenceAgents1 API53.0OpikLLMObservabilityTracing1 API48.5LaminarObservabilityLLMAI AgentsTracing3 APIs46.2BraintrustArtificial IntelligenceLLMObservability1 API45.6TraceloopLLM ObservabilityOpenTelemetryAI MonitoringTracing2 APIs44.8Parea AILLMObservabilityTesting2 APIs44.3LangChainLLMLangChainAgentsArtificial Intelligence1 API43.9Understudy LabsArtificial IntelligenceLLM GatewayMachine LearningOpen Source1 API42.7Weights and BiasesMLOpsExperiment TrackingLLM ObservabilityModel Registry1 API41.7Vellum AILLM PlatformPrompt EngineeringWorkflows2 APIs40.7Arize AILLM ObservabilityML MonitoringOpen SourceOpenTelemetry1 API40.1
Thin 19 Limited public surface area
Bespoke LabsCompanyArtificial IntelligenceMachine LearningLLM1 API38.7HumanloopLLM PlatformPrompt ManagementLLMOps1 API38.6LangSmithArtificial IntelligenceLLMObservability1 API38.4PromptLayerArtificial IntelligenceLLMPrompt EngineeringPrompt Management1 API37.8KluArtificial IntelligenceLLMLLM App PlatformPrompt Engineering1 API36.5FreeplayArtificial IntelligenceLLMObservability1 API36.1OpenlayerArtificial IntelligenceTestingObservability1 API36.1Athina AIArtificial IntelligenceLLMObservability1 API35.9CovalArtificial IntelligenceAgentsVoice AISimulation1 API35.5liars.townCompanyAI AgentsMulti-AgentGames4 APIs35.4Literal AIArtificial IntelligenceLLMObservability1 API35.4LangfuseArtificial IntelligenceLLMObservabilityOpen Source1 API34.1ArchalCompanyAI AgentsAgent TestingDeveloper Tools1 API33.6Snorkel AIArtificial IntelligenceData DevelopmentProgrammatic LabelingWeak Supervision1 API32.9Log10LLMLoggingObservability1 API32.4UpTrainArtificial IntelligenceLLMLLM Evaluation1 API31.7DeepchecksArtificial IntelligenceLLMTesting1 API31.4Hamming AIArtificial IntelligenceVoice AgentsLLMTesting1 API30.5AIMonCompanyAi MlLLM ObservabilityAI Monitoring1 API28.3
Emerging 8 Early or largely undocumented
AgentOpsAI AgentsObservabilityTracing3 APIs26.0Bentolabs AICompanyAI AgentsObservabilityOpenTelemetry25.3Maxim AIArtificial IntelligenceLLMAgentsObservability1 API24.9AfterQueryCompanyArtificial IntelligenceMachine LearningTraining Data1 API23.7Vals AiCompanyArtificial IntelligenceLLM22.4AshrCompanyAI AgentsAgent Testing22.0InstanceCompanyRoboticsMachine LearningVerification1 API17.1Pi LabsCompanyArtificial IntelligenceLLM1 API15.2
Minimal 5 Almost no public developer surface
Context.ai *CompanyArtificial IntelligenceAI AgentsEnterprise AI10.7Ooak DataCompanyArtificial IntelligenceMachine LearningReinforcement Learning10.7RobocurveCompanyRoboticsPhysical AI10.2IsidorCompanyArtificial IntelligenceEnterprise AIMachine Learning3.3GentraceCompanyArtificial IntelligenceLLM2 APIs0.0

APIs with this tag (59)

Ranked by the provider's API Evangelist rating — the Kin Score is scored per provider, not per API, so every API of a provider shares its band. How the rating works →

Exemplar 7 Complete, well-documented, and agent-ready
Harness Evaluations APIAccess evaluation results93.0OpenAI Evals APIManage and run evals in the OpenAI platform.84.8Cloudflare Evaluation APIThe Evaluation API from Cloudflare — 1 operation(s) for evaluation.81.3Qlik Sense Evaluation APIThe evaluation API from Qlik Sense — 10 operation(s) for evaluation.72.5Qlik Sense Evaluations APIThe evaluations API from Qlik Sense — 2 operation(s) for evaluations.72.58x8 Evaluations APIThe Evaluations API from 8x8 — 3 operation(s) for evaluations.70.8Google Dialogflow Evaluations APIThe evaluations API from Google Dialogflow — 6 operation(s) for evaluations.68.0
Strong 7 Solid coverage with minor gaps
n8n Evaluation APIOperations about evaluation test runs64.9Amigo Evaluations APIThe Evaluations API from Amigo — 4 operation(s) for evaluations.64.8Gumloop Evaluations APIThe Evaluations API from Gumloop — 4 operation(s) for evaluations.61.6Amplitude Evaluation APIRemote flag and experiment evaluation operations57.8Zoom Quality Management APIThe Zoom Quality Management API is designed to help contact centers track and analyze customer interactions...56.3Observe.AI Evaluations APIEvaluations API can be used to pull all the Evaluations(Manual and Auto QA) done on Observe AI platform. Pl...54.9Alloy Evaluations APIRun, retrieve, and audit identity, KYC, KYB, AML, fraud, and credit evaluations.54.7
Developing 19 Usable, with meaningful gaps to close
Avito Evaluation APIThe Evaluation API from Avito — 5 operation(s) for evaluation.53.4Qlik Evaluation APIThe evaluation API from Qlik — 5 operation(s) for evaluation.53.4Rapidata Evaluation APIThe Evaluation API from Rapidata — 1 operation(s) for evaluation.53.4Buk Evaluations APIThe Evaluations API from Buk — 3 operation(s) for evaluations.51.6Convai Evaluation APIThe Evaluation API from Convai — 1 operation(s) for evaluation.51.6Authenticx Evaluations APIThe Evaluations API from Authenticx — 2 operation(s) for evaluations.50.7Laminar Evaluations APIThe lower-level LaminarClient.evals surface for wiring evaluations into an existing pipeline: create an eva...46.2Braintrust APIThe Braintrust REST API provides programmatic access to projects, experiments, datasets, prompts, functions...45.6BigML Evaluations APIEvaluate model performance against a test dataset42.7Understudy Control Plane APIREST surface behind the dashboard and CLI, served from the gateway host under /admin/v1 and /customer/v1, o...42.7Trakstar Evaluations APIManage candidate evaluations42.4W&B Weave (LLM Observability)LLM observability and evaluation platform providing tracing, output evaluation, cost estimation, prompt pla...41.7Vellum LLM Platform APIThe Vellum REST API exposes prompts, workflows, evaluations, datasets, document indexes, deployments, and e...40.7HashiCorp Nomad Evaluations APIEndpoints for querying evaluations. Evaluations are the mechanism by which Nomad makes scheduling decisions.40.6Eigenpal Evaluation APIManage datasets, examples, evaluators, experiment batches, evaluator scores, and run promotion workflows.40.4AlyxAlyx is Arize's AI engineering agent that helps developers debug traces, create evaluators, build dashboard...40.1Arize AXArize AX is the commercial AI engineering platform covering tracing, evaluation, experiments, prompt manage...40.1PhoenixPhoenix is Arize's open-source LLM observability platform offering local tracing, evaluation, experiments, ...40.1AmeriCorps Open Data SODA APIThe AmeriCorps Open Data portal provides programmatic access to AmeriCorps research, evaluation, and progra...39.7
Thin 20 Limited public surface area
Humanloop LLM Platform APIThe Humanloop REST API and SDKs covered prompts, tools, datasets, evaluations, evaluators, and logs for LLM...38.6Docling EvalEnd-to-end evaluation framework for document parsing models and services. Provides standard datasets and me...38.3Split Evaluation APIEndpoints for evaluating feature flags and retrieving treatment values for given keys and feature flag names.38.0GentraceGentrace was an AI evaluation and observability product; the company has shut down and its codebase is now ...37.4Oxen Evaluations APIThe evaluations API from Oxen — 2 operation(s) for evaluations.36.7Extend Evaluations APIThe Evaluations API from Extend — 4 operation(s) for evaluations.36.1Athina AI Evaluations APIRun evaluations against datasets and logged inferences.35.9Workable Evaluations APISubmit and read interviewer evaluations and scorecards aligned to the job's interview kit.35.9Agenta Evaluations APIRun evaluations of variants against testsets.35.1Together AI Evaluation APIThe evaluation API from Together AI — 4 operation(s) for evaluation.35.0Tonic ValidateTonic Validate is an open-source RAG evaluation framework and metrics platform for measuring retrieval-augm...34.4LiteLLM Evals APIProvides /evals endpoints for the Evaluations API, enabling measurement and benchmarking of model performan...34.0EvaluationPlatform and SDK capabilities for assessing model performance across data splits, running error and slice a...32.9Log10 Evaluation APIAPI for running automated evaluations and benchmarking logged completions across multiple LLM providers, ge...32.4Log10 Feedback APIFeedback32.4UpTrain Evaluation APIThe Evaluation API from UpTrain — 3 operation(s) for evaluation.31.7Lunary APIREST API for Lunary covering ingestion (logs/traces), prompts (template management with versions and labels...29.4AIMon APIREST API for AIMon LLM monitoring and evaluation — manage users, models, applications, evaluations and eval...28.3Opik (GenAI Observability)Opik is Comet's open-source GenAI observability product. It provides spans-based tracing, evaluations, prom...28.0Confident AI PlatformConfident AI is the hosted platform that complements DeepEval with observability, centralized reporting, re...27.3
Emerging 6 Early or largely undocumented
USAID Development Experience Clearinghouse APIThe USAID Development Experience Clearinghouse (DEC) API provides programmatic access to the largest online...25.8Maxim AI APIThe Maxim REST API provides programmatic access to prompts, workflows, agents, datasets, evaluators, test r...24.9Ragas Python LibraryThe Ragas Python library is the primary surface of the project, installed via `pip install ragas` and impor...23.1Patronus Python SDKThe Patronus Python SDK provides decorators and clients for instrumenting LLM applications, running evaluat...19.6PercivalPercival is Patronus's agent debugging product that ingests agent traces and surfaces failure modes, tool m...19.6Pi Scoring APIOpenAPI-compatible REST API for scoring, calibrating, and optimizing LLM outputs against rubrics of natural...15.2

Companies reaching this through an API (41)

These companies publish an API, specification or operation carrying “Evaluation” but do not classify their business under it. Listed unranked and kept out of the count above, because one tagged operation is not a statement about what a company does.

8x8 Agent Health Monitor Agenta AI Gateway Alloy AmeriCorps Amigo Amplitude Authenticx Avito BigML Buk Cloudflare Comet Confident AI Convai Docling Eigenpal Extend Google Dialogflow Gumloop Harness HashiCorp Nomad LiteLLM Lunary n8n Observe.AI OpenAI Oxen Patronus AI Qlik Qlik Sense Ragas Rapidata Split Together AI Tonic.ai Trakstar USAID Workable Zoom

Score breakdown

Frequency
58.6
log-scaled weighted occurrences
Breadth
1.3
spread across providers
Quality lift
42.3
mean composite of providers using it
Cohesion
8.8
strength of nearest seed neighbor

Related tags

Tracing 15 co-occurrences Prompts 14 co-occurrences Experiments 12 co-occurrences Benchmarks 10 co-occurrences Prompt Management 8 co-occurrences LLM 36 co-occurrences Traces 10 co-occurrences Datasets 25 co-occurrences

Where this tag comes from

Api tag60
Provider tag49
Openapi tag23
Openapi op tag79

Cohort brief

Auto-generated

The 49 providers in the APIs.io catalog tagged Evaluation, scored on the Kin Score. Every figure below is computed from the catalog — nothing here is written.

Providers
49
all scored
Mean Kin Score
34.4
+11.5 vs catalog 22.9
Mean Agent Readiness
21.3
+9.4 vs catalog 11.9
Spread
0–62.7
median 35.9 · σ 13.7
How the 49 split by band
Exemplar 0Strong 3Developing 14Thin 19Emerging 8Minimal 5
Facet averages, against the whole catalog
FacetThis cohortCatalogDifferenceScored
Contract Quality 35 17.6 +17.4 49
Developer Ergonomics 37.7 23.2 +14.5 49
Operational Transparency 23.5 13.7 +9.8 49
Access Clarity 35.6 26.5 +9.1 49
Discoverability 65.7 58.1 +7.6 49
Contract Governance 4.9 6.3 -1.4 49

A facet is averaged over the members that carry it, not over the whole cohort — the “Scored” column is that count. Averaging an absent facet as zero would score our own coverage gaps as the providers’ posture.

What this cohort publishes
ArtifactThis cohortCatalogDifference
MCP server (any) 31% 14% +17
MCP server (first-party) 31% 12% +19
Agent Skills 0% 0% 0
OAuth scopes 2% 11% -9
Security 100% 98% +2
Arazzo workflows 6% 2% +4
Governance rules 12% 14% -2

mcp_pct counts any mcp/ artifact including ones API Evangelist derived from the provider OpenAPI; mcp_first_party_pct counts only servers the provider publishes. Prefer the latter.

Top by Kin Score
  1. 1 Comet 62.7
  2. 2 Runloop 59
  3. 3 Prime Intellect 55
  4. 4 Scorecard 53
  5. 5 Coval 51.6
  6. 6 Bluejay 50.5
  7. 7 Opik 48.5
  8. 8 Vijil 48
  9. 9 Laminar 46.2
  10. 10 Braintrust 45.6
Top by Agent Readiness
  1. 1 Coval 59.8
  2. 2 Braintrust 49.6
  3. 3 Scorecard 48.7
  4. 4 Openlayer 44.6
  5. 5 Bluejay 38.6
  6. 6 liars.town 35.5
  7. 7 Langfuse 34.8
  8. 8 Prime Intellect 33.7
  9. 9 LangChain 33
  10. 10 Comet 31
All 49 members of this roster resolve to a scored provider in the catalog.
Coverage caveat. 33 of 49 scored members had under half their catalog-attributable points satisfied at all (catalog_earned / catalog_max < 0.5). They are INCLUDED in the mean above. A large share here means the cohort mean is partly a measurement of how far our enrichment reached, not only of how these providers publish.Generated from the catalog build of 5 October 2026, across 29093 providers, using the same computation served by the APIs.io cohort API. Briefs are published for rosters of 5 or more scored providers; below that a distribution is not meaningful.

Work with this as data

Every tag here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for tags

7 MCP tools reach this
  • find_tagsBrowse and filter every tag in the catalog.
  • get_cohortThis tag as a scored cohort — every provider carrying it, with scores.
  • cohort_statsPRO — the distribution across this tag: mean, median, band split, adoption rates.
  • cohort_rankingsPRO — the leaderboard, on composite AND agent-readiness axes.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This tag 
curl "https://apis.io/api/v1/tags/evaluation"
All tags 
curl "https://apis.io/api/v1/tags?limit=25"
As a scored cohort 
curl "https://apis.io/api/v1/cohorts/tag/evaluation"
The distribution (Pro) 
curl "https://apis.io/api/v1/cohorts/tag/evaluation/stats" \
  -H "X-API-Key: $APIS_IO_KEY"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.