OpenAI

OpenAI Evals API

The Evals API allows you to programmatically configure and run evaluations to test model outputs against your expectations. Evaluations ensure model responses meet style and content criteria you specify, and are essential for building reliable LLM applications, especially when upgrading or trying new models.

Documentation GitHub OpenAPI

Operations 12

GET /evals List evaluations for a project #

POST /evals Create the structure of an evaluation that can be used to test a model's… #

GET /evals/{eval_id} Get an evaluation by ID #

POST /evals/{eval_id} Update certain properties of an evaluation #

DELETE /evals/{eval_id} Delete an evaluation #

GET /evals/{eval_id}/runs Get a list of runs for an evaluation #

POST /evals/{eval_id}/runs Kicks off a new run for a given evaluation, specifying the data source, and… #

GET /evals/{eval_id}/runs/{run_id} Get an evaluation run by ID #

POST /evals/{eval_id}/runs/{run_id} Cancel an ongoing evaluation run #

DELETE /evals/{eval_id}/runs/{run_id} Delete an eval run #

GET /evals/{eval_id}/runs/{run_id}/output_items Get a list of output items for an evaluation run #

GET /evals/{eval_id}/runs/{run_id}/output_items/{output_item_id} Get an evaluation run output item by ID #

Documentation

https://platform.openai.com/docs/api-reference/evals

https://platform.openai.com/docs/guides/evals

https://platform.openai.com/docs/assistants/overview

https://platform.openai.com/docs/api-reference/assistants

https://platform.openai.com/docs/guides/text-to-speech

https://platform.openai.com/docs/api-reference/audio

https://platform.openai.com/docs/guides/speech-to-text

https://developers.openai.com/api/docs/guides/audio/

https://developers.openai.com/api/docs/guides/voice-agents/

https://platform.openai.com/docs/api-reference/chat

https://platform.openai.com/docs/guides/embeddings

https://platform.openai.com/docs/api-reference/embeddings

https://platform.openai.com/docs/api-reference/files

https://platform.openai.com/docs/guides/fine-tuning

https://platform.openai.com/docs/api-reference/fine-tuning

https://platform.openai.com/docs/guides/images

https://platform.openai.com/docs/api-reference/images

https://platform.openai.com/docs/guides/image-generation

https://platform.openai.com/docs/guides/images-vision

https://platform.openai.com/docs/models

https://platform.openai.com/docs/api-reference/models

https://platform.openai.com/docs/assistants/how-it-works/managing-threads-and-messages

https://platform.openai.com/docs/api-reference/threads

https://platform.openai.com/docs/api-reference/completions

Specifications

https://raw.githubusercontent.com/api-evangelist/openai/refs/heads/main/openapi/openai-evals-api-openapi.yml

Schemas & Data

https://raw.githubusercontent.com/api-evangelist/openai/refs/heads/main/json-schema/openai-chat-completion-schema.json

https://raw.githubusercontent.com/api-evangelist/openai/refs/heads/main/json-schema/openai-embedding-schema.json

Other Resources

https://raw.githubusercontent.com/api-evangelist/openai/refs/heads/main/json-ld/openai-context.jsonld

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this

find_apisBrowse and filter every API in the catalog.
get_api_artifactsOne API's artifacts, grouped by type.
get_openapiThe primary OpenAPI for this API.
find_similar_apisAPIs that look like this one.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This API

curl "https://apis.io/api/v1/apis/openai-evals-api"

All apis

curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

openapi: 3.2.0
info:
  title: OpenAI Evals API
  description: The OpenAI REST API. Please see https://platform.openai.com/docs/api-reference for more details.
  version: 2.3.0
  termsOfService: https://openai.com/policies/terms-of-use
  contact:
    name: OpenAI Support
    url: https://help.openai.com/
  license:
    name: MIT
    url: https://github.com/openai/openai-openapi/blob/master/LICENSE
servers:
- url: https://api.openai.com/v1
security:
- ApiKeyAuth: []
tags:


# --- truncated at 32 KB (221 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/openai/refs/heads/main/openapi/openai-evals-api-openapi.yml