Opik

Opik Manual Evaluation API

Manual evaluation resources for traces, threads, and spans

GitHub OpenAPI

Operations 3

POST /v1/private/manual-evaluation/spans Manually evaluate spans #
POST /v1/private/manual-evaluation/threads Manually evaluate threads #
POST /v1/private/manual-evaluation/traces Manually evaluate traces #

Documentation

📖
Documentation
https://www.comet.com/docs/opik/reference/rest-api/overview

Specifications

⚙
OpenAPI
https://raw.githubusercontent.com/api-evangelist/opik/refs/heads/main/openapi/opik-manual-evaluation-api-openapi.yml

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/opik-manual-evaluation-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

opik-manual-evaluation-api-openapi.yml Raw ↑ 
openapi: 3.2.0
info:
  title: Opik REST Manual Evaluation API
  description: The Opik REST API is currently in beta and subject to change.
  contact:
    name: Github Repository
    url: https://github.com/comet-ml/opik
  license:
    name: Apache 2.0
    url: http://www.apache.org/licenses/LICENSE-2.0.html
  version: 1.0.0
servers:
- url: http://localhost:5173/api
  description: Local server
- url: https://www.comet.com/opik/api
  description: Opik Cloud
tags:
- name: Manual Evaluation
  description: Manual evaluation resources for traces, threads, and spans
paths:
  /v1/private/manual-evaluation/spans:
    post:
      tags:
      - Manual Evaluation
      summary: Manually evaluate spans
      description: Manually trigger evaluation rules on selected spans. Bypasses sampling and enqueues all specified spans for evaluation.
      operationId: evaluateSpans
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ManualEvaluationRequest'
      responses:
        '202':
          description: Accepted - Evaluation request queued successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ManualEvaluationResponse'
        '400':
          description: Bad Request - Invalid request or missing automation rules
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '404':
          description: Not Found - Project not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
  /v1/private/manual-evaluation/threads:
    post:
      tags:
      - Manual Evaluation
      summary: Manually evaluate threads
      description: Manually trigger evaluation rules on selected threads. Bypasses sampling and enqueues all specified threads for evaluation.
      operationId: evaluateThreads
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ManualEvaluationRequest'
      responses:
        '202':
          description: Accepted - Evaluation request queued successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ManualEvaluationResponse'
        '400':
          description: Bad Request - Invalid request or missing automation rules
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '404':
          description: Not Found - Project not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
  /v1/private/manual-evaluation/traces:
    post:
      tags:
      - Manual Evaluation
      summary: Manually evaluate traces
      description: Manually trigger evaluation rules on selected traces. Bypasses sampling and enqueues all specified traces for evaluation.
      operationId: evaluateTraces
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ManualEvaluationRequest'
      responses:
        '202':
          description: Accepted - Evaluation request queued successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ManualEvaluationResponse'
        '400':
          description: Bad Request - Invalid request or missing automation rules
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '404':
          description: Not Found - Project not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
components:
  schemas:
    ManualEvaluationResponse:
      type: object
      properties:
        entities_queued:
          type: integer
          description: Number of entities queued for evaluation
          format: int32
          example: 5
        rules_applied:
          type: integer
          description: Number of rules that will be applied
          format: int32
          example: 2
    ManualEvaluationRequest:
      required:
      - entity_ids
      - entity_type
      - project_id
      - rule_ids
      type: object
      properties:
        project_id:
          type: string
          description: Project ID
          format: uuid
          example: 550e8400-e29b-41d4-a716-446655440000
        entity_ids:
          minItems: 1
          type: array
          description: List of entity IDs (trace IDs or thread IDs) to evaluate
          example:
          - 550e8400-e29b-41d4-a716-446655440000
          - 550e8400-e29b-41d4-a716-446655440001
          items:
            type: string
            description: List of entity IDs (trace IDs or thread IDs) to evaluate
            format: uuid
        rule_ids:
          minItems: 1
          type: array
          description: List of automation rule IDs to apply
          example:
          - 660e8400-e29b-41d4-a716-446655440000
          items:
            type: string
            description: List of automation rule IDs to apply
            format: uuid
        entity_type:
          type: string
          description: Type of entity to evaluate (trace or thread)
          example: trace
          enum:
          - trace
          - thread
          - span
    ErrorMessage:
      type: object
      properties:
        code:
          type: integer
          format: int32
        message:
          type: string
        details:
          type: string