Comet Manual Evaluation API

openapi: 3.1.0
info:
  title: Opik REST Manual Evaluation API
  description: "The Opik REST API is currently in beta and subject to change. If you have any questions or feedback about the APIs, please reach out on GitHub: https://github.com/comet-ml/opik.\n\nAll of the methods listed in this documentation are used by either the SDK or the UI to interact with the Opik server. As a result,\nthe methods have been optimized for these use-cases in mind. If you are looking for a method that is not listed above, please create\nand issue on GitHub or raise a PR!\n\nOpik includes two main deployment options that results in slightly different API usage:\n\n- **Self-hosted Opik instance:** You will simply need to specify the URL as `http://localhost:5173/api/<endpoint_path>` or similar. This is the default option for the docs.\n- **Opik Cloud:** You will need to specify the Opik API Key and Opik Workspace in the header. The format of the header should be:\n\n  ```\n  {\n    \"Comet-Workspace\": \"your-workspace-name\",\n    \"authorization\": \"your-api-key\"\n  }\n  ```\n\n  The full payload would therefore look like:\n  \n  ```\n  curl -X GET 'https://www.comet.com/opik/api/v1/private/projects' \\\n  -H 'Accept: application/json' \\\n  -H 'Comet-Workspace: <your-workspace-name>' \\\n  -H 'authorization: <your-api-key>'\n  ```\n\n  Do take note here that the authorization header value does not include the `Bearer ` prefix. To switch to using the Opik Cloud in the documentation, you can\n  click on the edit button displayed when hovering over the `Base URL` displayed on the right hand side of the docs.\n"
  contact:
    name: Github Repository
    url: https://github.com/comet-ml/opik
  license:
    name: Apache 2.0
    url: http://www.apache.org/licenses/LICENSE-2.0.html
  version: 1.0.0
servers:
- url: http://localhost:5173/api
  description: Local server
- url: https://www.comet.com/opik/api
  description: Opik Cloud
tags:
- name: Manual Evaluation
  description: Manual evaluation resources for traces, threads, and spans
paths:
  /v1/private/manual-evaluation/spans:
    post:
      tags:
      - Manual Evaluation
      summary: Manually evaluate spans
      description: Manually trigger evaluation rules on selected spans. Bypasses sampling and enqueues all specified spans for evaluation.
      operationId: evaluateSpans
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ManualEvaluationRequest'
      responses:
        '202':
          description: Accepted - Evaluation request queued successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ManualEvaluationResponse'
        '400':
          description: Bad Request - Invalid request or missing automation rules
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '404':
          description: Not Found - Project not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
  /v1/private/manual-evaluation/threads:
    post:
      tags:
      - Manual Evaluation
      summary: Manually evaluate threads
      description: Manually trigger evaluation rules on selected threads. Bypasses sampling and enqueues all specified threads for evaluation.
      operationId: evaluateThreads
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ManualEvaluationRequest'
      responses:
        '202':
          description: Accepted - Evaluation request queued successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ManualEvaluationResponse'
        '400':
          description: Bad Request - Invalid request or missing automation rules
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '404':
          description: Not Found - Project not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
  /v1/private/manual-evaluation/traces:
    post:
      tags:
      - Manual Evaluation
      summary: Manually evaluate traces
      description: Manually trigger evaluation rules on selected traces. Bypasses sampling and enqueues all specified traces for evaluation.
      operationId: evaluateTraces
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ManualEvaluationRequest'
      responses:
        '202':
          description: Accepted - Evaluation request queued successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ManualEvaluationResponse'
        '400':
          description: Bad Request - Invalid request or missing automation rules
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '404':
          description: Not Found - Project not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorMessage'
components:
  schemas:
    ManualEvaluationRequest:
      required:
      - entity_ids
      - entity_type
      - project_id
      - rule_ids
      type: object
      properties:
        project_id:
          type: string
          description: Project ID
          format: uuid
          example: 550e8400-e29b-41d4-a716-446655440000
        entity_ids:
          minItems: 1
          type: array
          description: List of entity IDs (trace IDs or thread IDs) to evaluate
          example:
          - 550e8400-e29b-41d4-a716-446655440000
          - 550e8400-e29b-41d4-a716-446655440001
          items:
            type: string
            description: List of entity IDs (trace IDs or thread IDs) to evaluate
            format: uuid
        rule_ids:
          minItems: 1
          type: array
          description: List of automation rule IDs to apply
          example:
          - 660e8400-e29b-41d4-a716-446655440000
          items:
            type: string
            description: List of automation rule IDs to apply
            format: uuid
        entity_type:
          type: string
          description: Type of entity to evaluate (trace or thread)
          example: trace
          enum:
          - trace
          - thread
          - span
    ManualEvaluationResponse:
      type: object
      properties:
        entities_queued:
          type: integer
          description: Number of entities queued for evaluation
          format: int32
          example: 5
        rules_applied:
          type: integer
          description: Number of rules that will be applied
          format: int32
          example: 2
    ErrorMessage:
      type: object
      properties:
        code:
          type: integer
          format: int32
        message:
          type: string
        details:
          type: string