OpenAPI Specification
openapi: 3.1.0
info:
title: Opik REST Experiments API
description: "The Opik REST API is currently in beta and subject to change. If you have any questions or feedback about the APIs, please reach out on GitHub: https://github.com/comet-ml/opik.\n\nAll of the methods listed in this documentation are used by either the SDK or the UI to interact with the Opik server. As a result,\nthe methods have been optimized for these use-cases in mind. If you are looking for a method that is not listed above, please create\nand issue on GitHub or raise a PR!\n\nOpik includes two main deployment options that results in slightly different API usage:\n\n- **Self-hosted Opik instance:** You will simply need to specify the URL as `http://localhost:5173/api/<endpoint_path>` or similar. This is the default option for the docs.\n- **Opik Cloud:** You will need to specify the Opik API Key and Opik Workspace in the header. The format of the header should be:\n\n ```\n {\n \"Comet-Workspace\": \"your-workspace-name\",\n \"authorization\": \"your-api-key\"\n }\n ```\n\n The full payload would therefore look like:\n \n ```\n curl -X GET 'https://www.comet.com/opik/api/v1/private/projects' \\\n -H 'Accept: application/json' \\\n -H 'Comet-Workspace: <your-workspace-name>' \\\n -H 'authorization: <your-api-key>'\n ```\n\n Do take note here that the authorization header value does not include the `Bearer ` prefix. To switch to using the Opik Cloud in the documentation, you can\n click on the edit button displayed when hovering over the `Base URL` displayed on the right hand side of the docs.\n"
contact:
name: Github Repository
url: https://github.com/comet-ml/opik
license:
name: Apache 2.0
url: http://www.apache.org/licenses/LICENSE-2.0.html
version: 1.0.0
servers:
- url: http://localhost:5173/api
description: Local server
- url: https://www.comet.com/opik/api
description: Opik Cloud
tags:
- name: Experiments
description: Experiment resources
paths:
/v1/private/experiments/batch:
patch:
tags:
- Experiments
summary: Batch update experiments
description: Update multiple experiments
operationId: batchUpdateExperiments
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentBatchUpdate'
responses:
'204':
description: No Content
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments:
get:
tags:
- Experiments
summary: Find experiments
description: Find experiments
operationId: findExperiments
parameters:
- name: page
in: query
schema:
minimum: 1
type: integer
format: int32
default: 1
- name: size
in: query
schema:
minimum: 1
type: integer
format: int32
default: 10
- name: datasetId
in: query
schema:
type: string
format: uuid
- name: optimization_id
in: query
schema:
type: string
format: uuid
- name: types
in: query
schema:
type: string
- name: name
in: query
schema:
type: string
description: Filter experiments by name (partial match, case insensitive)
- name: dataset_deleted
in: query
schema:
type: boolean
- name: prompt_id
in: query
schema:
type: string
format: uuid
- name: project_id
in: query
schema:
type: string
format: uuid
- name: project_deleted
in: query
schema:
type: boolean
- name: sorting
in: query
schema:
type: string
- name: filters
in: query
schema:
type: string
- name: experiment_ids
in: query
schema:
type: string
description: Filter experiments by a list of experiment IDs
- name: force_sorting
in: query
schema:
type: boolean
description: Force sorting even when exceeding the endpoint result set limit. May result in slower queries
default: false
responses:
'200':
description: Experiments resource
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentPage_Public'
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage_Public'
post:
tags:
- Experiments
summary: Create experiment
description: Create experiment
operationId: createExperiment
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/Experiment_Write'
responses:
'201':
description: Created
headers:
Location:
required: true
style: simple
schema:
type: string
example: ${basePath}/v1/private/experiments/{id}
/v1/private/experiments/items:
post:
tags:
- Experiments
summary: Create experiment items
description: Create experiment items
operationId: createExperimentItems
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentItemsBatch'
responses:
'204':
description: No content
/v1/private/experiments/items/delete:
post:
tags:
- Experiments
summary: Delete experiment items
description: Delete experiment items
operationId: deleteExperimentItems
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentItemsDelete'
responses:
'204':
description: No content
/v1/private/experiments/delete:
post:
tags:
- Experiments
summary: Delete experiments by id
description: Delete experiments by id
operationId: deleteExperimentsById
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteIdsHolder'
responses:
'204':
description: No content
/v1/private/experiments/execute:
post:
tags:
- Experiments
summary: Create and execute experiment
description: Creates experiments for each prompt variant and asynchronously processes all dataset items
operationId: executeExperiment
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentExecutionRequest'
required: true
responses:
'202':
description: Experiments created and processing started
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentExecutionResponse'
/v1/private/experiments/items/bulk:
put:
tags:
- Experiments
summary: Record experiment items in bulk
description: Record experiment items in bulk with traces, spans, and feedback scores. Maximum request size is 4MB.
operationId: experimentItemsBulk
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentItemBulkUpload_ExperimentItemBulkWriteView'
responses:
'204':
description: No content
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
'409':
description: Conflict
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
'422':
description: Unprocessable Content
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments/feedback-scores/names:
get:
tags:
- Experiments
summary: Find Feedback Score names
description: Find Feedback Score names
operationId: findFeedbackScoreNames
parameters:
- name: experiment_ids
in: query
schema:
type: string
- name: project_id
in: query
schema:
type: string
format: uuid
responses:
'200':
description: Feedback Scores resource
content:
application/json:
schema:
$ref: '#/components/schemas/FeedbackScoreNames_Public'
/v1/private/experiments/groups:
get:
tags:
- Experiments
summary: Find experiment groups
description: Find experiments grouped by specified fields
operationId: findExperimentGroups
parameters:
- name: groups
in: query
schema:
type: string
- name: types
in: query
schema:
type: string
- name: name
in: query
schema:
type: string
description: Filter experiments by name (partial match, case insensitive)
- name: project_id
in: query
schema:
type: string
format: uuid
- name: project_deleted
in: query
schema:
type: boolean
- name: filters
in: query
schema:
type: string
responses:
'200':
description: Experiment groups
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentGroupResponse'
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments/groups/aggregations:
get:
tags:
- Experiments
summary: Find experiment groups with aggregations
description: Find experiments grouped by specified fields with aggregation metrics
operationId: findExperimentGroupsAggregations
parameters:
- name: groups
in: query
schema:
type: string
- name: types
in: query
schema:
type: string
- name: name
in: query
schema:
type: string
description: Filter experiments by name (partial match, case insensitive)
- name: project_id
in: query
schema:
type: string
format: uuid
- name: project_deleted
in: query
schema:
type: boolean
description: Filter experiments by deleted projects
- name: filters
in: query
schema:
type: string
responses:
'200':
description: Experiment groups with aggregations
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentGroupAggregationsResponse'
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments/finish:
post:
tags:
- Experiments
summary: Finish experiments
description: Finish experiments and trigger alert events
operationId: finishExperiments
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteIdsHolder'
responses:
'204':
description: No content
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments/{id}:
get:
tags:
- Experiments
summary: Get experiment by id
description: Get experiment by id
operationId: getExperimentById
parameters:
- name: id
in: path
required: true
schema:
type: string
format: uuid
responses:
'200':
description: Experiment resource
content:
application/json:
schema:
$ref: '#/components/schemas/Experiment_Public'
'404':
description: Not found
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage_Public'
patch:
tags:
- Experiments
summary: Update experiment by id
description: Update experiment by id
operationId: updateExperiment
parameters:
- name: id
in: path
required: true
schema:
type: string
format: uuid
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentUpdate'
responses:
'204':
description: No Content
'404':
description: Not found
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments/items/{id}:
get:
tags:
- Experiments
summary: Get experiment item by id
description: Get experiment item by id
operationId: getExperimentItemById
parameters:
- name: id
in: path
required: true
schema:
type: string
format: uuid
responses:
'200':
description: Experiment item resource
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentItem_Public'
'404':
description: Not found
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorMessage_Public'
/v1/private/experiments/items/stream:
post:
tags:
- Experiments
summary: Stream experiment items
description: Stream experiment items
operationId: streamExperimentItems
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentItemStreamRequest'
responses:
'200':
description: Experiment items stream or error during process
content:
application/octet-stream:
schema:
maxItems: 2000
type: array
items:
type: string
anyOf:
- $ref: '#/components/schemas/ExperimentItem'
- $ref: '#/components/schemas/ErrorMessage'
/v1/private/experiments/stream:
post:
tags:
- Experiments
summary: Stream experiments
description: Stream experiments
operationId: streamExperiments
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ExperimentStreamRequest_Public'
responses:
'200':
description: Experiments stream or error during process
content:
application/octet-stream:
schema:
maxItems: 2000
type: array
items:
type: string
anyOf:
- $ref: '#/components/schemas/Experiment_Public'
- $ref: '#/components/schemas/ErrorMessage_Public'
components:
schemas:
ErrorMessage_Public:
type: object
properties:
code:
type: integer
format: int32
message:
type: string
details:
type: string
ExperimentScore_Public:
required:
- name
- value
type: object
properties:
name:
minLength: 1
type: string
value:
type: number
GroupDetails:
type: object
properties:
groups_details:
type: array
items:
$ref: '#/components/schemas/GroupDetail'
GroupContentWithAggregations:
type: object
properties:
label:
type: string
aggregations:
$ref: '#/components/schemas/AggregationData'
ExperimentGroupAggregationsResponse:
type: object
properties:
content:
type: object
additionalProperties:
$ref: '#/components/schemas/GroupContentWithAggregations'
AggregationData:
type: object
properties:
experiment_count:
type: integer
format: int64
trace_count:
type: integer
format: int64
total_estimated_cost:
type: number
total_estimated_cost_avg:
type: number
duration:
$ref: '#/components/schemas/PercentageValues'
feedback_scores:
type: array
items:
$ref: '#/components/schemas/FeedbackScoreAverage'
experiment_scores:
type: array
items:
$ref: '#/components/schemas/FeedbackScoreAverage'
pass_rate_avg:
type: number
passed_count_sum:
type: integer
format: int64
total_count_sum:
type: integer
format: int64
Comment_Public:
required:
- text
type: object
properties:
id:
type: string
format: uuid
readOnly: true
text:
minLength: 1
type: string
created_at:
type: string
format: date-time
readOnly: true
last_updated_at:
type: string
format: date-time
readOnly: true
created_by:
type: string
readOnly: true
last_updated_by:
type: string
readOnly: true
readOnly: true
DeleteIdsHolder:
required:
- ids
type: object
properties:
ids:
maxItems: 1000
minItems: 1
uniqueItems: true
type: array
items:
type: string
format: uuid
JsonNode:
type: object
ValueEntry:
type: object
properties:
value:
type: number
reason:
type: string
category_name:
type: string
source:
type: string
enum:
- ui
- sdk
- online_scoring
last_updated_at:
type: string
format: date-time
span_type:
type: string
span_id:
type: string
readOnly: true
ValueEntry_ExperimentItemBulkWriteView:
type: object
properties:
value:
type: number
reason:
type: string
category_name:
type: string
source:
type: string
enum:
- ui
- sdk
- online_scoring
last_updated_at:
type: string
format: date-time
span_type:
type: string
span_id:
type: string
readOnly: true
AssertionScoreAverage_Public:
required:
- name
- value
type: object
properties:
name:
minLength: 1
type: string
value:
type: number
description: Per-assertion average pass rates for test suite experiments. Null for regular experiments.
readOnly: true
ErrorMessage:
type: object
properties:
code:
type: integer
format: int32
message:
type: string
details:
type: string
Message:
required:
- content
- role
type: object
properties:
role:
minLength: 1
type: string
content:
$ref: '#/components/schemas/JsonNode'
ExperimentItemsDelete:
required:
- ids
type: object
properties:
ids:
maxItems: 1000
minItems: 1
uniqueItems: true
type: array
items:
type: string
format: uuid
JsonListString_ExperimentItemBulkWriteView:
type: object
anyOf:
- type: object
- type: array
items:
type: object
- type: string
DatasetVersionSummary_Public:
type: object
properties:
id:
type: string
description: Unique identifier of the version
format: uuid
version_hash:
type: string
description: Hash of the version content
version_name:
type: string
description: Sequential version name formatted as 'v1', 'v2', etc.
change_description:
type: string
description: Description of changes in this version
tags:
uniqueItems: true
type: array
description: Tags associated with this version
items:
type: string
description: Tags associated with this version
description: Summary of the latest dataset version
readOnly: true
ExperimentItemBulkUpload_ExperimentItemBulkWriteView:
required:
- dataset_name
- experiment_name
- items
type: object
properties:
experiment_name:
minLength: 1
type: string
dataset_name:
minLength: 1
type: string
experiment_id:
type: string
description: Optional experiment ID. If provided, items will be added to the existing experiment and experimentName will be ignored. If not provided or experiment with that ID doesn't exist, a new experiment will be created with the given experimentName
format: uuid
project_name:
pattern: (?s)^\s*(\S.*\S|\S)\s*$
type: string
description: Project for traces auto-created from items that provide evaluate_task_result (i.e. without an explicit trace). If null, the default project is used; relying on this fallback is deprecated, please provide project_name explicitly.
items:
maxItems: 1000
minItems: 1
type: array
items:
$ref: '#/components/schemas/ExperimentItemBulkRecord_ExperimentItemBulkWriteView'
ErrorInfo_ExperimentItemBulkWriteView:
required:
- exception_type
- traceback
type: object
properties:
exception_type:
minLength: 1
type: string
message:
type: string
traceback:
minLength: 1
type: string
JsonListString:
type: object
anyOf:
- type: object
- type: array
items:
type: object
- type: string
Experiment_Write:
required:
- dataset_name
type: object
properties:
id:
type: string
format: uuid
dataset_name:
minLength: 1
type: string
nullable: true
project_id:
type: string
description: Project ID. Takes precedence over project_name when both are provided.
format: uuid
project_name:
pattern: (?s)^\s*(\S.*\S|\S)\s*$
type: string
description: Project name. Creates project if it doesn't exist. Ignored when project_id is provided.
name:
type: string
metadata:
$ref: '#/components/schemas/JsonListString_Write'
tags:
maxItems: 50
minItems: 0
uniqueItems: true
type: array
items:
maxLength: 100
minLength: 0
type: string
type:
type: string
enum:
- regular
- trial
- mini-batch
- mutation
evaluation_method:
type: string
enum:
- dataset
- evaluation_suite
optimization_id:
type: string
format: uuid
status:
type: string
enum:
- running
- completed
- cancelled
experiment_scores:
type: array
items:
$ref: '#/components/schemas/ExperimentScore_Write'
prompt_version:
$ref: '#/components/schemas/PromptVersionLink_Write'
prompt_versions:
type: array
items:
$ref: '#/components/schemas/PromptVersionLink_Write'
dataset_version_id:
type: string
description: ID of the dataset version this experiment is linked to. If not provided at creation, experiment will be automatically linked to the latest version.
format: uuid
JsonListString_Public:
type: object
anyOf:
- type: object
- type: array
items:
type: object
- type: string
ExperimentScore_Write:
required:
- name
- value
type: object
properties:
name:
minLength: 1
type: string
value:
type: number
ExperimentStreamRequest_Public:
required:
- name
type: object
properties:
name:
minLength: 1
type: string
limit:
maximum: 2000
minimum: 1
type: integer
format: int32
last_retrieved_id:
type: string
format: uuid
project_name:
pattern: (?s)^\s*(\S.*\S|\S)\s*$
type: string
GroupDetail:
type: object
properties:
group_sorting:
type: array
items:
type: string
ExecutionPolicy:
type: object
properties:
runs_per_item:
maximum: 100
minimum: 1
type: integer
format: int32
pass_threshold:
maximum: 100
minimum: 1
type: integer
format: int32
readOnly: true
PromptVersionLink:
required:
- id
type: object
properties:
id:
type: string
format: uuid
commit:
type: string
readOnly: true
version_number:
type: string
description: sequential version number in the format v<N>; null for masks
readOnly: true
prompt_id:
type: string
format: uuid
readOnly: true
prompt_name:
type: string
readOnly: true
PromptVersionLink_Public:
type: object
properties:
prompt_version_id:
type: string
format: uuid
commit:
type: string
prompt_id:
type: string
format: uuid
prompt_name:
type: string
ExperimentItemsBatch:
required:
- experiment_items
type: object
properties:
experiment_items:
maxItems: 1000
minItems: 1
uniqueItems: true
type: array
items:
$ref: '#/components/schemas/ExperimentItem'
Experiment_Public:
required:
- dataset_name
type: object
properties:
id:
type: string
format: uuid
dataset_name:
minLength: 1
type: string
nullable: true
dataset_id:
type: string
format: uuid
readOnly: true
project_id:
type: string
description: Project ID. Takes precedence over project_name when both are provided.
format: uuid
project_name:
pattern: (?s)^\s*(\S.*\S|\S)\s*$
type: string
description: Project name. Creates project if it doesn't exist. Ignored when project_id is provided.
name:
type: string
metadata:
$ref: '#/components/schemas/JsonListString_Public'
tags:
maxItems: 50
minItems: 0
uniqueItems: true
type: array
items:
maxLength: 100
minLength: 0
type: string
type:
type: string
enum:
- regular
- trial
- mini-batch
- mutation
evaluation_method:
type: string
enum:
- dataset
- evaluation_suite
optimization_id:
type: string
format: uuid
feedback_scores:
type: array
readOnly: true
items:
$ref: '#/components/schemas/FeedbackScoreAverage_Public'
comments:
type: array
readOnly: true
items:
$ref: '#/components/schemas/Comment_Public'
trace_count:
type: integer
format: int64
readOnly: true
dataset_item_count:
type: integer
format: int64
readOnly: true
created_at:
type: string
format: date-time
readOnly: true
duration:
$ref: '#/components/schemas/PercentageValues_Public'
total_estimated_cost:
type: number
readOnly: true
total_estimated_cost_avg:
type: number
readOnly: true
usage:
type: object
additionalProperties:
type: number
format: double
readOnly: true
readOnly: true
last_updated_at:
type: string
format: date-time
readOnly: true
created_by:
type: string
readOnly: true
last_updated_by:
type: string
readOnly: true
status:
type: string
enum:
- running
- completed
- cancelled
experiment_scores:
type: array
items:
$ref: '#/components/schemas/ExperimentScore_Public'
prompt_version:
$ref: '#/components/schemas/PromptVersionLink_Public'
prompt_versions:
type: array
items:
$ref: '#/components/schemas/PromptVersionLink_Public'
dataset_version_id:
type: string
description: ID of the dataset version this experiment is linked to. If not provided at creation, experiment will be automatically linked to the latest version.
format: uuid
dataset_version_summary:
$ref: '#/components/schemas/DatasetVersionSummary_Public'
pass_rate:
type: number
description: Pass rate for test suite experiments (0.0-1.0). Null for regular experiments.
readOnly: true
passed_count:
type: integer
description: Number of items that passed for test suite experiments. Null for regular experiments.
format: int64
readOnly: true
total_count:
type: integer
description: Total number of items for test suite experiments. Null for regular experiments.
format: int64
readOnly: true
assertion_scores:
type: array
description: Per-assertion average pass rates for test suite experiments. Null for regular experiments.
readOnly: true
items:
$ref: '#/components/schemas/AssertionScoreAverage_Public'
JsonListString_Write:
type: object
anyOf:
- type: object
- type: array
items:
type: object
- type: string
ExperimentScore:
required:
- name
- value
type: object
properties:
name:
minLength: 1
type: string
value:
type: number
ExperimentItemStreamRequest:
required:
- experiment_name
type: object
properties:
experiment_name:
minLength: 1
type: string
limit:
maximum: 2000
minimum: 1
type: integer
format: int32
last_retrieved_i
# --- truncated at 32 KB (48 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/opik/refs/heads/main/openapi/opik-experiments-api-openapi.yml