Traceloop

Traceloop Evaluators API

The evaluators API from Traceloop — 36 operation(s) for evaluators.

GitHub OpenAPI

Operations 36

POST /v2/evaluators Create a custom LLM-as-a-judge evaluator #
POST /v2/evaluators/agent-efficiency/execute Execute agent-efficiency evaluator #
POST /v2/evaluators/agent-flow-quality/execute Execute agent-flow-quality evaluator #
POST /v2/evaluators/agent-goal-accuracy/execute Execute agent-goal-accuracy evaluator #
POST /v2/evaluators/agent-goal-completeness/execute Execute agent-goal-completeness evaluator #
POST /v2/evaluators/agent-tool-error-detector/execute Execute agent-tool-error-detector evaluator #
POST /v2/evaluators/agent-tool-trajectory/execute Execute agent-tool-trajectory evaluator #
POST /v2/evaluators/answer-completeness/execute Execute answer-completeness evaluator #
POST /v2/evaluators/answer-correctness/execute Execute answer-correctness evaluator #
POST /v2/evaluators/answer-relevancy/execute Execute answer-relevancy evaluator #
POST /v2/evaluators/char-count/execute Execute char-count evaluator #
POST /v2/evaluators/char-count-ratio/execute Execute char-count-ratio evaluator #
POST /v2/evaluators/context-relevance/execute Execute context-relevance evaluator #
POST /v2/evaluators/conversation-quality/execute Execute conversation-quality evaluator #
POST /v2/evaluators/faithfulness/execute Execute faithfulness evaluator #
POST /v2/evaluators/html-comparison/execute Execute html-comparison evaluator #
POST /v2/evaluators/instruction-adherence/execute Execute instruction-adherence evaluator #
POST /v2/evaluators/intent-change/execute Execute intent-change evaluator #
POST /v2/evaluators/json-validator/execute Execute json-validator evaluator #
POST /v2/evaluators/perplexity/execute Execute perplexity evaluator #
POST /v2/evaluators/pii-detector/execute Execute pii-detector evaluator #
POST /v2/evaluators/placeholder-regex/execute Execute placeholder-regex evaluator #
POST /v2/evaluators/profanity-detector/execute Execute profanity-detector evaluator #
POST /v2/evaluators/prompt-injection/execute Execute prompt-injection evaluator #
POST /v2/evaluators/prompt-perplexity/execute Execute prompt-perplexity evaluator #
POST /v2/evaluators/regex-validator/execute Execute regex-validator evaluator #
POST /v2/evaluators/secrets-detector/execute Execute secrets-detector evaluator #
POST /v2/evaluators/semantic-similarity/execute Execute semantic-similarity evaluator #
POST /v2/evaluators/sexism-detector/execute Execute sexism-detector evaluator #
POST /v2/evaluators/sql-validator/execute Execute sql-validator evaluator #
POST /v2/evaluators/tone-detection/execute Execute tone-detection evaluator #
POST /v2/evaluators/topic-adherence/execute Execute topic-adherence evaluator #
POST /v2/evaluators/toxicity-detector/execute Execute toxicity-detector evaluator #
POST /v2/evaluators/uncertainty-detector/execute Execute uncertainty-detector evaluator #
POST /v2/evaluators/word-count/execute Execute word-count evaluator #
POST /v2/evaluators/word-count-ratio/execute Execute word-count-ratio evaluator #

Documentation

📖
Documentation
https://www.traceloop.com/docs/api-reference/introduction

Specifications

⚙
OpenAPI
https://raw.githubusercontent.com/api-evangelist/traceloop/refs/heads/main/openapi/traceloop-evaluators-api-openapi.yml

Schemas & Data

📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/traceloop/refs/heads/main/json-schema/traceloop-schema.json

Other Resources

🔗
Examples
https://raw.githubusercontent.com/api-evangelist/traceloop/refs/heads/main/examples/traceloop-examples.yml
🔗
Vocabulary
https://raw.githubusercontent.com/api-evangelist/traceloop/refs/heads/main/vocabulary/traceloop-vocabulary.yml
🔗
JSONLDContext
https://raw.githubusercontent.com/api-evangelist/traceloop/refs/heads/main/json-ld/traceloop-context.jsonld

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/traceloop-evaluators-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

traceloop-evaluators-api-openapi.yml Raw ↑ 
openapi: 3.2.0
info:
  title: Traceloop Evaluators API
  version: 1.0.0
  contact: {}
servers:
- url: https://api.traceloop.com
tags:


# --- truncated at 32 KB (80 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/traceloop/refs/heads/main/openapi/traceloop-evaluators-api-openapi.yml