Agenta Evaluations API

Run evaluations of variants against testsets.

GitHub OpenAPI
Business capability
Artificial Intelligence Management BC-610.60

Operations 69

POST /evaluations/runs/ Create Runs #
DELETE /evaluations/runs/ Delete Runs #
PATCH /evaluations/runs/ Edit Runs #
POST /evaluations/runs/query Query Runs #
POST /evaluations/runs/close Close Runs #
POST /evaluations/runs/open Open Runs #
GET /evaluations/runs/{run_id} Fetch Run #
PATCH /evaluations/runs/{run_id} Edit Run #
DELETE /evaluations/runs/{run_id} Delete Run #
POST /evaluations/runs/{run_id}/close Close Run #
POST /evaluations/runs/{run_id}/open Open Run #
GET /evaluations/runs/{run_id}/queues/default Fetch Default Queue #
POST /evaluations/scenarios/ Create Scenarios #
DELETE /evaluations/scenarios/ Delete Scenarios #
PATCH /evaluations/scenarios/ Edit Scenarios #
POST /evaluations/scenarios/query Query Scenarios #
GET /evaluations/scenarios/{scenario_id} Fetch Scenario #
PATCH /evaluations/scenarios/{scenario_id} Edit Scenario #
DELETE /evaluations/scenarios/{scenario_id} Delete Scenario #
PUT /evaluations/results/ Set Results #
POST /evaluations/results/ Set Results #
DELETE /evaluations/results/ Delete Results #
PATCH /evaluations/results/ Set Results #
POST /evaluations/results/query Query Results #
GET /evaluations/results/{result_id} Fetch Result #
DELETE /evaluations/results/{result_id} Delete Result #
POST /evaluations/metrics/refresh Refresh Metrics #
PUT /evaluations/metrics/ Set Metrics #
POST /evaluations/metrics/ Set Metrics #
DELETE /evaluations/metrics/ Delete Metrics #
PATCH /evaluations/metrics/ Set Metrics #
POST /evaluations/metrics/query Query Metrics #
GET /evaluations/metrics/{metrics_id} Fetch Metric #
DELETE /evaluations/metrics/{metrics_id} Delete Metric #
POST /evaluations/queues/ Create Queues #
DELETE /evaluations/queues/ Delete Queues #
PATCH /evaluations/queues/ Edit Queues #
POST /evaluations/queues/query Query Queues #
GET /evaluations/queues/{queue_id} Fetch Queue #
PATCH /evaluations/queues/{queue_id} Edit Queue #
DELETE /evaluations/queues/{queue_id} Delete Queue #
POST /evaluations/queues/{queue_id}/scenarios/query Query Queue Scenarios #
POST /simple/evaluations/ Create Evaluation #
POST /simple/evaluations/query Query Evaluations #
GET /simple/evaluations/{evaluation_id} Fetch Evaluation #
PATCH /simple/evaluations/{evaluation_id} Edit Evaluation #
DELETE /simple/evaluations/{evaluation_id} Delete Evaluation #
POST /simple/evaluations/{evaluation_id}/start Start Evaluation #
POST /simple/evaluations/{evaluation_id}/stop Stop Evaluation #
POST /simple/evaluations/{evaluation_id}/close Close Evaluation #
POST /simple/evaluations/{evaluation_id}/open Open Evaluation #
POST /simple/evaluations/{evaluation_id}/populate Populate Evaluation Slice #
POST /simple/evaluations/{evaluation_id}/process Process Evaluation Slice #
POST /simple/evaluations/{evaluation_id}/probe Probe Evaluation Slice #
POST /simple/evaluations/{evaluation_id}/prune Prune Evaluation Slice #
POST /simple/evaluations/{evaluation_id}/refresh Refresh Evaluation Slice #
POST /simple/evaluations/{evaluation_id}/scenarios/add Add Evaluation Scenarios #
POST /simple/evaluations/{evaluation_id}/scenarios/remove Remove Evaluation Scenarios #
POST /simple/evaluations/{evaluation_id}/steps/add Add Evaluation Steps #
POST /simple/evaluations/{evaluation_id}/steps/remove Remove Evaluation Steps #
POST /simple/evaluations/{evaluation_id}/repeats/set Set Evaluation Repeats #
POST /simple/queues/ Create Simple Queue #
DELETE /simple/queues/ Delete Simple Queues #
POST /simple/queues/query Query Simple Queues #
GET /simple/queues/{queue_id} Fetch Simple Queue #
DELETE /simple/queues/{queue_id} Delete Simple Queue #
POST /simple/queues/{queue_id}/scenarios/query Query Simple Queue Scenarios #
POST /simple/queues/{queue_id}/traces/ Add Simple Queue Traces #
POST /simple/queues/{queue_id}/testcases/ Add Simple Queue Testcases #

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API
curl "https://apis.io/api/v1/apis/agenta:agenta-evaluations-api"
All apis
curl "https://apis.io/api/v1/apis?limit=25"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

agenta-evaluations-api-openapi.yml Raw ↑
openapi: 3.2.0
info:
  title: Agenta Evaluations API
  description: Agenta API
  contact:
    name: Agenta
    url: https://agenta.ai/
    email: team@agenta.ai
  version: 0.1.0
servers:
- url: /api
- url: https://eu.cloud.agenta.ai/api
security:
- APIKeyHeader: []
tags:


# --- truncated at 32 KB (138 KB total) ---
# Full source: https://artifacts.apis.io/agenta/openapi/agenta-evaluations-api-openapi.yml