ArthurAI RAG Experiments API
Endpoints for managing RAG experiments
Endpoints for managing RAG experiments
Every API here is available over the APIs.io API and to AI agents over MCP.
One button, every client — Claude, Cursor, VS Code and the rest.
https://apis.io/mcp
find_apisBrowse and filter every API in the catalog.get_api_artifactsOne API's artifacts, grouped by type.get_openapiThe primary OpenAPI for this API.find_similar_apisAPIs that look like this one.apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.resolveTurn a domain, URL or GitHub org into the provider it belongs to.find_cohortsEvery scored population of providers in the catalog.curl "https://apis.io/api/v1/apis/arthurai-rag-experiments-api"
curl "https://apis.io/api/v1/apis?limit=25"
Discovery needs no key. Ratings and market analysis are Pro.
Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.
A second provider on the same verified email joins the account you already have.
openapi: 3.2.0
info:
title: Arthur GenAI Engine RAG Experiments API
version: 2.1.688
description: Endpoints for managing RAG experiments
tags:
- name: RAG Experiments
description: Endpoints for managing RAG experiments
paths:
/api/v1/tasks/{task_id}/rag_experiments:
get:
tags:
- RAG Experiments
summary: List RAG experiments
description: List all RAG experiments for a task with optional filtering and pagination
operationId: list_rag_experiments_api_v1_tasks__task_id__rag_experiments_get
security:
- API Key: []
parameters:
- name: task_id
in: path
required: true
schema:
type: string
format: uuid
title: Task Id
- name: search
in: query
required: false
schema:
anyOf:
- type: string
- type: 'null'
description: Search text to filter experiments by name or description
title: Search
description: Search text to filter experiments by name or description
- name: dataset_id
in: query
required: false
schema:
anyOf:
- type: string
format: uuid
- type: 'null'
description: Filter experiments by dataset ID
title: Dataset Id
description: Filter experiments by dataset ID
- name: sort
in: query
required: false
schema:
$ref: '#/components/schemas/PaginationSortMethod'
description: Sort the results (asc/desc)
default: desc
description: Sort the results (asc/desc)
- name: page_size
in: query
required: false
schema:
type: integer
description: Page size. Default is 10. Must be greater than 0 and less than 5000.
default: 10
title: Page Size
description: Page size. Default is 10. Must be greater than 0 and less than 5000.
- name: page
in: query
required: false
schema:
type: integer
description: Page number
default: 0
title: Page
description: Page number
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/RagExperimentListResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
post:
tags:
- RAG Experiments
summary: Create and run a RAG experiment
description: Create a new RAG experiment and initiate execution
operationId: create_rag_experiment_api_v1_tasks__task_id__rag_experiments_post
security:
- API Key: []
parameters:
- name: task_id
in: path
required: true
schema:
type: string
format: uuid
title: Task Id
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/CreateRagExperimentRequest'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/RagExperimentSummary'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/api/v1/rag_experiments/{experiment_id}:
get:
tags:
- RAG Experiments
summary: Get RAG experiment details
description: Get detailed information about a specific RAG experiment including summary results
operationId: get_rag_experiment_api_v1_rag_experiments__experiment_id__get
security:
- API Key: []
parameters:
- name: experiment_id
in: path
required: true
schema:
type: string
title: Experiment ID
description: The ID of the experiment to retrieve
description: The ID of the experiment to retrieve
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/RagExperimentDetail'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
delete:
tags:
- RAG Experiments
summary: Delete RAG experiment
description: Delete a RAG experiment and all its associated data
operationId: delete_rag_experiment_api_v1_rag_experiments__experiment_id__delete
security:
- API Key: []
parameters:
- name: experiment_id
in: path
required: true
schema:
type: string
title: Experiment ID
description: The ID of the experiment to delete
description: The ID of the experiment to delete
responses:
'204':
description: Experiment deleted successfully.
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/api/v1/rag_experiments/{experiment_id}/test_cases:
get:
tags:
- RAG Experiments
summary: Get experiment test cases
description: Get paginated list of test case results for a RAG experiment
operationId: get_rag_experiment_test_cases_api_v1_rag_experiments__experiment_id__test_cases_get
security:
- API Key: []
parameters:
- name: experiment_id
in: path
required: true
schema:
type: string
title: Experiment ID
description: The ID of the experiment
description: The ID of the experiment
- name: sort
in: query
required: false
schema:
$ref: '#/components/schemas/PaginationSortMethod'
description: Sort the results (asc/desc)
default: desc
description: Sort the results (asc/desc)
- name: page_size
in: query
required: false
schema:
type: integer
description: Page size. Default is 10. Must be greater than 0 and less than 5000.
default: 10
title: Page Size
description: Page size. Default is 10. Must be greater than 0 and less than 5000.
- name: page
in: query
required: false
schema:
type: integer
description: Page number
default: 0
title: Page
description: Page number
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/RagTestCaseListResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/api/v1/rag_experiments/{experiment_id}/rag_configs/{rag_config_key}/results:
get:
tags:
- RAG Experiments
summary: Get RAG config results
description: Get paginated list of results for a specific RAG configuration within an experiment (supports both saved and unsaved configs)
operationId: get_rag_config_results_api_v1_rag_experiments__experiment_id__rag_configs__rag_config_key__results_get
security:
- API Key: []
parameters:
- name: experiment_id
in: path
required: true
schema:
type: string
title: Experiment ID
description: The ID of the experiment
description: The ID of the experiment
- name: rag_config_key
in: path
required: true
schema:
type: string
title: RAG Config Key
description: 'The RAG config key (format: ''saved:setting_config_id:version'' or ''unsaved:uuid''). URL-encode colons as %3A'
description: 'The RAG config key (format: ''saved:setting_config_id:version'' or ''unsaved:uuid''). URL-encode colons as %3A'
- name: sort
in: query
required: false
schema:
$ref: '#/components/schemas/PaginationSortMethod'
description: Sort the results (asc/desc)
default: desc
description: Sort the results (asc/desc)
- name: page_size
in: query
required: false
schema:
type: integer
description: Page size. Default is 10. Must be greater than 0 and less than 5000.
default: 10
title: Page Size
description: Page size. Default is 10. Must be greater than 0 and less than 5000.
- name: page
in: query
required: false
schema:
type: integer
description: Page number
default: 0
title: Page
description: Page number
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/RagConfigResultListResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/api/v1/rag_experiments/{experiment_id}/notebook:
patch:
tags:
- RAG Experiments
summary: Attach notebook to RAG experiment
description: Attach a RAG notebook to an existing experiment
operationId: attach_notebook_to_rag_experiment_api_v1_rag_experiments__experiment_id__notebook_patch
security:
- API Key: []
parameters:
- name: experiment_id
in: path
required: true
schema:
type: string
description: ID of the experiment
title: Experiment Id
description: ID of the experiment
- name: notebook_id
in: query
required: true
schema:
type: string
description: ID of the notebook to attach
title: Notebook Id
description: ID of the notebook to attach
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/RagExperimentSummary'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
components:
schemas:
_MultiTargetVectorJoinEnum:
type: integer
enum:
- 1
- 2
- 3
- 4
- 5
title: _MultiTargetVectorJoinEnum
description: Define how multi target vector searches should be combined.
RagTestCaseListResponse:
properties:
page:
type: integer
title: Page
description: Current page number (0-indexed)
page_size:
type: integer
title: Page Size
description: Number of items per page
total_pages:
type: integer
title: Total Pages
description: Total number of pages
total_count:
type: integer
title: Total Count
description: Total number of records
data:
items:
$ref: '#/components/schemas/RagTestCase'
type: array
title: Data
description: List of test cases
type: object
required:
- page
- page_size
- total_pages
- total_count
- data
title: RagTestCaseListResponse
description: Paginated list of RAG test cases
UnsavedRagConfigResponse:
properties:
type:
type: string
const: unsaved
title: Type
default: unsaved
unsaved_id:
anyOf:
- type: string
format: uuid
- type: 'null'
title: Unsaved Id
description: Unique identifier for this unsaved RAG configuration (generated by backend if not provided)
rag_provider_id:
type: string
format: uuid
title: Rag Provider Id
description: ID of the RAG provider to use for this search
settings:
anyOf:
- $ref: '#/components/schemas/WeaviateHybridSearchSettingsConfigurationResponse'
- $ref: '#/components/schemas/WeaviateVectorSimilarityTextSearchSettingsConfigurationResponse'
- $ref: '#/components/schemas/WeaviateKeywordSearchSettingsConfigurationResponse'
title: Settings
description: RAG search settings configuration
query_column:
$ref: '#/components/schemas/DatasetColumnVariableSource'
description: Dataset column to use as the RAG search query
type: object
required:
- rag_provider_id
- settings
- query_column
title: UnsavedRagConfigResponse
description: Configuration for an unsaved RAG search setting (response version)
RagExperimentDetail:
properties:
id:
type: string
title: Id
description: Unique identifier for the experiment
name:
type: string
title: Name
description: Name of the experiment
description:
anyOf:
- type: string
- type: 'null'
title: Description
description: Description of the experiment
created_at:
type: string
title: Created At
description: ISO timestamp when experiment was created
finished_at:
anyOf:
- type: string
- type: 'null'
title: Finished At
description: ISO timestamp when experiment finished
status:
$ref: '#/components/schemas/ExperimentStatus'
description: Current status of the experiment
total_rows:
type: integer
title: Total Rows
description: Total number of test rows in the experiment
completed_rows:
type: integer
title: Completed Rows
description: Number of test rows completed successfully
failed_rows:
type: integer
title: Failed Rows
description: Number of test rows that failed
total_cost:
anyOf:
- type: string
- type: 'null'
title: Total Cost
description: Total cost of running the experiment
dataset_ref:
$ref: '#/components/schemas/DatasetRef'
description: Reference to the dataset used
dataset_row_filter:
anyOf:
- items:
$ref: '#/components/schemas/NewDatasetVersionRowColumnItemRequest'
type: array
- type: 'null'
title: Dataset Row Filter
description: Optional list of column name and value filters applied to dataset rows. Only rows matching ALL specified column name-value pairs (AND condition) were included in the experiment.
notebook_id:
anyOf:
- type: string
- type: 'null'
title: Notebook Id
description: Optional notebook ID this experiment is linked to
eval_list:
items:
$ref: '#/components/schemas/EvalRef-Output'
type: array
title: Eval List
description: List of evaluations being run
rag_configs:
items:
oneOf:
- $ref: '#/components/schemas/SavedRagConfig-Output'
- $ref: '#/components/schemas/UnsavedRagConfigResponse'
discriminator:
propertyName: type
mapping:
saved: '#/components/schemas/SavedRagConfig-Output'
unsaved: '#/components/schemas/UnsavedRagConfigResponse'
type: array
title: Rag Configs
description: List of RAG configurations being tested
summary_results:
$ref: '#/components/schemas/RagSummaryResults'
description: Summary of results across all test cases
type: object
required:
- id
- name
- created_at
- status
- total_rows
- completed_rows
- failed_rows
- dataset_ref
- eval_list
- rag_configs
- summary_results
title: RagExperimentDetail
description: Detailed information about a RAG experiment
RagExperimentSummary:
properties:
id:
type: string
title: Id
description: Unique identifier for the experiment
name:
type: string
title: Name
description: Name of the experiment
description:
anyOf:
- type: string
- type: 'null'
title: Description
description: Description of the experiment
created_at:
type: string
title: Created At
description: ISO timestamp when experiment was created
finished_at:
anyOf:
- type: string
- type: 'null'
title: Finished At
description: ISO timestamp when experiment finished
status:
$ref: '#/components/schemas/ExperimentStatus'
description: Current status of the experiment
dataset_id:
type: string
format: uuid
title: Dataset Id
description: ID of the dataset used
dataset_name:
type: string
title: Dataset Name
description: Name of the dataset used
dataset_version:
type: integer
title: Dataset Version
description: Version of the dataset used
total_rows:
type: integer
title: Total Rows
description: Total number of test rows in the experiment
completed_rows:
type: integer
title: Completed Rows
description: Number of test rows completed successfully
failed_rows:
type: integer
title: Failed Rows
description: Number of test rows that failed
total_cost:
anyOf:
- type: string
- type: 'null'
title: Total Cost
description: Total cost of running the experiment
rag_configs:
items:
oneOf:
- $ref: '#/components/schemas/SavedRagConfig-Output'
- $ref: '#/components/schemas/UnsavedRagConfigResponse'
discriminator:
propertyName: type
mapping:
saved: '#/components/schemas/SavedRagConfig-Output'
unsaved: '#/components/schemas/UnsavedRagConfigResponse'
type: array
title: Rag Configs
description: List of RAG configurations being tested
type: object
required:
- id
- name
- created_at
- status
- dataset_id
- dataset_name
- dataset_version
- total_rows
- completed_rows
- failed_rows
- rag_configs
title: RagExperimentSummary
description: Summary of a RAG experiment
CreateRagExperimentRequest:
properties:
name:
type: string
title: Name
description: Name for the experiment
description:
anyOf:
- type: string
- type: 'null'
title: Description
description: Description of the experiment
dataset_ref:
$ref: '#/components/schemas/DatasetRefInput'
description: Reference to the dataset to use
eval_list:
items:
$ref: '#/components/schemas/EvalRef-Input'
type: array
title: Eval List
description: List of evaluations to run
dataset_row_filter:
anyOf:
- items:
$ref: '#/components/schemas/NewDatasetVersionRowColumnItemRequest'
type: array
- type: 'null'
title: Dataset Row Filter
description: Optional list of column name and value filters. Only rows matching ALL specified column name-value pairs (AND condition) will be included in the experiment. If not specified, all rows from the dataset will be used.
rag_configs:
items:
oneOf:
- $ref: '#/components/schemas/SavedRagConfig-Input'
- $ref: '#/components/schemas/UnsavedRagConfig'
discriminator:
propertyName: type
mapping:
saved: '#/components/schemas/SavedRagConfig-Input'
unsaved: '#/components/schemas/UnsavedRagConfig'
type: array
title: Rag Configs
description: List of RAG configurations to test. Each config specifies which dataset column to use as the query.
type: object
required:
- name
- dataset_ref
- eval_list
- rag_configs
title: CreateRagExperimentRequest
description: Request to create a new RAG experiment
RagProviderQueryResponse:
properties:
response:
$ref: '#/components/schemas/WeaviateQueryResults'
type: object
required:
- response
title: RagProviderQueryResponse
HybridFusion:
type: string
enum:
- FUSION_TYPE_RANKED
- FUSION_TYPE_RELATIVE_SCORE
title: HybridFusion
description: Define how the query's hybrid fusion operation should be performed.
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
input:
title: Input
ctx:
type: object
title: Context
type: object
required:
- loc
- msg
- type
title: ValidationError
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
WeaviateKeywordSearchSettingsConfigurationResponse:
properties:
collection_name:
type: string
title: Collection Name
description: Name of the vector collection used for the search.
limit:
anyOf:
- type: integer
- type: 'null'
title: Limit
description: Maximum number of objects to return.
include_vector:
anyOf:
- type: boolean
- type: string
- items:
type: string
type: array
- type: 'null'
title: Include Vector
description: Boolean value whether to include vector embeddings in the response or can be used to specify the names of the vectors to include in the response if your collection uses named vectors. Will be included as a dictionary in the vector property in the response.
default: false
offset:
anyOf:
- type: integer
- type: 'null'
title: Offset
description: Skips first N results in similarity response. Useful for pagination.
auto_limit:
anyOf:
- type: integer
- type: 'null'
title: Auto Limit
description: Automatically limit search results to groups of objects with similar distances, stopping after auto_limit number of significant jumps.
return_metadata:
anyOf:
- items:
type: string
enum:
- creation_time
- last_update_time
- distance
- certainty
- score
- explain_score
- is_consistent
- query_profile
type: array
- $ref: '#/components/schemas/MetadataQuery'
- type: 'null'
title: Return Metadata
description: Specify metadata fields to return.
return_properties:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Return Properties
description: Specify which properties to return for each object.
rag_provider:
type: string
const: weaviate
title: Rag Provider
default: weaviate
search_kind:
type: string
const: keyword_search
title: Search Kind
default: keyword_search
minimum_match_or_operator:
anyOf:
- type: integer
- type: 'null'
title: Minimum Match Or Operator
description: Minimum number of keywords that define a match. Objects returned will have to have at least this many matches.
and_operator:
anyOf:
- type: boolean
- type: 'null'
title: And Operator
description: Search returns objects that contain all tokens in the search string. Cannot be used with minimum_match_or_operator
type: object
required:
- collection_name
title: WeaviateKeywordSearchSettingsConfigurationResponse
RagEvalResultSummaries:
properties:
rag_config_key:
anyOf:
- type: string
- type: 'null'
title: Rag Config Key
description: 'RAG config key: ''saved:setting_config_id:version'' for saved, ''unsaved:uuid'' for unsaved'
rag_config_type:
anyOf:
- type: string
- type: 'null'
title: Rag Config Type
description: 'Type: ''saved'' or ''unsaved'''
setting_configuration_id:
anyOf:
- type: string
format: uuid
- type: 'null'
title: Setting Configuration Id
description: ID of the RAG search setting configuration (for saved configs only)
setting_configuration_version:
anyOf:
- type: integer
- type: 'null'
title: Setting Configuration Version
description: Version of the RAG search setting configuration (for saved configs only)
eval_results:
items:
$ref: '#/components/schemas/EvalResultSummary'
type: array
title: Eval Results
description: Results for each evaluation run on this RAG configuration
type: object
required:
- eval_results
title: RagEvalResultSummaries
description: Summary of evaluation results for a RAG configuration
ExperimentStatus:
type: string
enum:
- queued
- running
- failed
- completed
title: ExperimentStatus
description: Status of an experiment
UnsavedRagConfig:
properties:
type:
type: string
const: unsaved
title: Type
default: unsaved
unsaved_id:
anyOf:
- type: string
format: uuid
- type: 'null'
title: Unsaved Id
description: Unique identifier for this unsaved RAG configuration (generated by backend if not provided)
rag_provider_id:
type: string
format: uuid
title: Rag Provider Id
description: ID of the RAG provider to use for this search
settings:
anyOf:
- $ref: '#/components/schemas/WeaviateHybridSearchSettingsConfigurationRequest'
- $ref: '#/components/schemas/WeaviateKeywordSearchSettingsConfigurationRequest'
- $ref: '#/components/schemas/WeaviateVectorSimilarityTextSearchSettingsConfigurationRequest'
title: Settings
description: RAG search settings configuration
query_column:
$ref: '#/components/schemas/DatasetColumnVariableSource'
description: Dataset column to use as the RAG search query
type: object
required:
- rag_provider_id
- settings
- query_column
title: UnsavedRagConfig
description: Configuration for an unsaved RAG search setting
EvalResultSummary:
properties:
eval_name:
type: string
title: Eval Name
description: Name of the evaluation
eval_version:
type: string
title: Eval Version
description: Version of the evaluation
pass_count:
type: integer
title: Pass Count
description: Number of test cases that passed
total_count:
type: integer
title: Total Count
description: Total number of test cases evaluated
type: object
required:
- eval_name
- eval_version
- pass_count
- total_count
title: EvalResultSummary
description: Results for a single eval
WeaviateQueryResults:
properties:
rag_provider:
type: string
const: weaviate
title: Rag Provider
default: weaviate
objects:
items:
$ref: '#/components/schemas/WeaviateQueryResult'
type: array
title: Objects
description: List of search result objects
type: object
required:
- objects
title: WeaviateQueryResults
description: Response from Weaviate similarity text search
NewDatasetVersionRowColumnItemRequest:
properties:
column_name:
type: string
title: Column Name
description: Name of column.
column_value:
type: string
title: Column Value
description: Value of column for the row.
type: object
required:
- column_name
- column_value
title: NewDatasetVersionRowColumnItemRequest
description: Represents a single column-value pair in a dataset row.
EvalVariableMapping-Input:
properties:
variable_name:
type: string
title: Variable Name
description: Name of the eval variable
source:
oneOf:
- $ref: '#/components/schemas/DatasetColumnVariableSource'
- $ref: '#/components/schemas/ExperimentOutputVariableSource'
title: Source
description: Source of the variable value
discriminator:
propertyName: type
mapping:
dataset_column: '#/components/schemas/DatasetColumnVariableSource'
experiment_output: '#/components/schemas/ExperimentOutputVariableSource'
type: object
required:
- variable_name
- source
title: EvalVariableMapping
description: Mapping of an eval variable to its source (dataset column or experiment output)
RagConfigResultListResponse:
properties:
page:
type: integer
title: Page
description: Current page number (0-indexed)
page_size:
type: integer
title: Page Size
description: Number of items per page
total_pages:
type: integer
title: Total Pages
description: Total number of pages
total_count:
type: integer
title: Total Count
description: Total number of records
data:
items:
$ref: '#/components/schemas/RagConfigResult'
type: array
title: Data
description: List of results for the RAG configuration
type: object
required:
- page
- page_size
- total_pages
- total_count
- data
title: RagConfigResultListResponse
description: Paginated list of results for a specific RAG configuration
_MultiTargetVectorJoin:
properties:
combination:
$ref: '#/components/schemas/_MultiTargetVectorJoinEnum'
target_vectors:
items:
type: string
type: array
title: Target Vectors
weights:
anyOf:
- additionalProperties:
anyOf:
- type: number
- items:
type: number
type: array
type: object
- type: 'null'
title: Weights
type: object
required:
- c
# --- truncated at 32 KB (68 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/arthurai/refs/heads/main/openapi/arthurai-rag-experiments-api-openapi.yml