The runs API from LangChain — 7 operation(s) for runs.
openapi: 3.1.0
info:
title: LangSmith access_policies runs API
description: 'The LangSmith API is used to programmatically create and manage LangSmith resources.
## Host
https://api.smith.langchain.com
## Authentication
To authenticate with the LangSmith API, set the `X-Api-Key` header
to a valid [LangSmith API key](https://docs.langchain.com/langsmith/create-account-api-key#create-an-api-key).
'
version: 0.1.0
servers:
- url: /
tags:
- name: runs
paths:
/runs:
post:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: Queues a single run for ingestion. The request body must be a JSON-encoded run object that follows the Run schema.
tags:
- runs
summary: Create a Run
parameters: []
responses:
'202':
description: Run created
content:
application/json:
schema:
type: object
additionalProperties:
allOf:
- type: string
- type: object
properties:
message:
type: string
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'403':
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'409':
description: Conflict
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'422':
description: Unprocessable Entity
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'429':
description: Too Many Requests
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/runs.Run'
/runs/batch:
post:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: 'Ingests a batch of runs in a single JSON payload. The payload must have `post` and/or `patch` arrays containing run objects.
Prefer this endpoint over single‑run ingestion when submitting hundreds of runs, but `/runs/multipart` offers better handling for very large fields and attachments.'
tags:
- runs
summary: Ingest Runs (Batch JSON)
parameters: []
responses:
'202':
description: Runs batch ingested
content:
application/json:
schema:
type: object
additionalProperties:
allOf:
- type: string
- type: object
properties:
message:
type: string
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'403':
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'409':
description: Conflict
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'422':
description: Unprocessable Entity
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'429':
description: Too Many Requests
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
patch:
type: array
items:
$ref: '#/components/schemas/runs.Run'
post:
type: array
items:
$ref: '#/components/schemas/runs.Run'
/runs/multipart:
post:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: 'Ingests multiple runs, feedback objects, and binary attachments in a single `multipart/form-data` request.
**Part‑name pattern**: `<event>.<run_id>[.<field>]` where `event` ∈ {`post`, `patch`, `feedback`, `attachment`}.
* `post|patch.<run_id>` – JSON run payload.
* `post|patch.<run_id>.<field>` – out‑of‑band run data (`inputs`, `outputs`, `events`, `error`, `extra`, `serialized`).
* `feedback.<run_id>` – JSON feedback payload (must include `trace_id`).
* `attachment.<run_id>.<filename>` – arbitrary binary attachment stored in S3.
**Headers**: every part must set `Content-Type` **and** either a `Content-Length` header or `length` parameter. Per‑part `Content-Encoding` is **not** allowed; the top‑level request may be `Content-Encoding: gzip` or `Content-Encoding: zstd`.
**Best performance** for high‑volume ingestion.'
tags:
- runs
summary: Ingest Runs (Multipart)
parameters: []
responses:
'202':
description: Accepted
content:
application/json:
schema:
type: object
additionalProperties:
type: string
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'403':
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'409':
description: Conflict
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'422':
description: Unprocessable Entity
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'429':
description: Too Many Requests
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
properties:
post.{run_id}:
type: string
format: binary
description: Run to create (JSON)
patch.{run_id}:
type: string
format: binary
description: Run to update (JSON)
post.{run_id}.inputs:
type: string
format: binary
description: Large inputs object (JSON) stored out‑of‑band
patch.{run_id}.outputs:
type: string
format: binary
description: Large outputs object (JSON) stored out‑of‑band
feedback.{run_id}:
type: string
format: binary
description: Feedback object (JSON) – must include trace_id
attachment.{run_id}.{filename}:
type: string
format: binary
description: Binary attachment linked to run {run_id}
/runs/{run_id}:
patch:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: Updates a run identified by its ID. The body should contain only the fields to be changed; unknown fields are ignored.
tags:
- runs
summary: Update a Run
parameters:
- description: Run ID
name: run_id
in: path
required: true
schema:
type: string
format: uuid
responses:
'202':
description: Run updated
content:
application/json:
schema:
type: object
additionalProperties:
allOf:
- type: string
- type: object
properties:
message:
type: string
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'403':
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'404':
description: Not Found
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'409':
description: Conflict
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'422':
description: Unprocessable Entity
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
'429':
description: Too Many Requests
content:
application/json:
schema:
$ref: '#/components/schemas/runs.ErrorResponse'
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/runs.Run'
/v2/runs/query:
post:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: '**Alpha:** The request and response contract may change;
Returns a paginated list of runs for the given projects within min/max start_time. Supports filters, cursor pagination, and `selects` to select fields to return.'
tags:
- runs
summary: Query runs
parameters:
- description: application/json
name: Accept
in: header
schema:
type: string
- description: application/json (required for JSON body)
name: Content-Type
in: header
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/query.QueryRunsResponseBody'
'400':
description: Bad Request
content:
application/json:
schema:
type: object
additionalProperties:
type: string
'401':
description: Unauthorized
content:
application/json:
schema:
type: object
additionalProperties:
type: string
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/query.QueryRunsRequestBody'
/v2/runs/{run_id}:
get:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: '**Alpha:** The request and response contract may change;
Returns one run by ID for the given session and start_time. Use the `selects` query parameter (repeatable) to select fields to return.'
tags:
- runs
summary: Get a single run
parameters:
- description: application/json
name: Accept
in: header
schema:
type: string
- description: Run UUID
name: run_id
in: path
required: true
schema:
type: string
format: uuid
- description: '`project_id` is the UUID of the tracing project that owns the run.'
name: project_id
in: query
required: true
schema:
type: string
format: uuid
title: Project Id
- description: '`selects` lists which properties to include on the returned run (repeatable query parameter). Accepts any value of the `RunSelectField` enum. If omitted, only `id` is returned.'
name: selects
in: query
style: form
explode: true
schema:
type: array
items:
enum:
- ID
- NAME
- RUN_TYPE
- STATUS
- START_TIME
- END_TIME
- LATENCY_SECONDS
- FIRST_TOKEN_TIME
- ERROR
- ERROR_PREVIEW
- EXTRA
- METADATA
- EVENTS
- INPUTS
- INPUTS_PREVIEW
- OUTPUTS
- OUTPUTS_PREVIEW
- MANIFEST
- PARENT_RUN_IDS
- PROJECT_ID
- TRACE_ID
- THREAD_ID
- DOTTED_ORDER
- IS_ROOT
- REFERENCE_EXAMPLE_ID
- REFERENCE_DATASET_ID
- TOTAL_TOKENS
- PROMPT_TOKENS
- COMPLETION_TOKENS
- TOTAL_COST
- PROMPT_COST
- COMPLETION_COST
- PROMPT_TOKEN_DETAILS
- COMPLETION_TOKEN_DETAILS
- PROMPT_COST_DETAILS
- COMPLETION_COST_DETAILS
- PRICE_MODEL_ID
- TAGS
- APP_PATH
- ATTACHMENTS
- THREAD_EVALUATION_TIME
- IS_IN_DATASET
- SHARE_URL
- FEEDBACK_STATS
type: string
title: Selects
- description: '`start_time` is the run''s `start_time` (RFC3339 date-time), used together with `project_id` to locate the run.'
name: start_time
in: query
required: true
schema:
type: string
format: date-time
title: Start Time
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/query.RunResponse'
'400':
description: Bad Request
content:
application/json:
schema:
type: object
additionalProperties:
type: string
'401':
description: Unauthorized
content:
application/json:
schema:
type: object
additionalProperties:
type: string
'404':
description: Not Found
content:
application/json:
schema:
type: object
additionalProperties:
type: string
/v2/traces/{trace_id}/runs:
get:
security:
- API Key: []
- Tenant ID: []
- Bearer Auth: []
description: '**Alpha:** The request and response contract may change;
Returns runs for a trace ID within min/max start time. Optional `filter`; repeatable `selects` to select fields to return.'
tags:
- runs
summary: List runs in a trace
parameters:
- description: application/json
name: Accept
in: header
schema:
type: string
- description: Trace UUID
name: trace_id
in: path
required: true
schema:
type: string
format: uuid
- description: '`filter` narrows which runs within this trace are returned, using a LangSmith filter expression evaluated against each run. For example: `eq(run_type, "llm")` for LLM runs only, or `eq(status, "error")` for failed runs.
See https://docs.langchain.com/langsmith/trace-query-syntax#filter-query-language for syntax.'
name: filter
in: query
schema:
type: string
title: Filter
- description: '`max_start_time` is the inclusive upper bound for run `start_time` (RFC3339 date-time).'
name: max_start_time
in: query
required: true
schema:
type: string
format: date-time
title: Max Start Time
- description: '`min_start_time` is the inclusive lower bound for run `start_time` (RFC3339 date-time).'
name: min_start_time
in: query
required: true
schema:
type: string
format: date-time
title: Min Start Time
- description: '`project_id` is the UUID of the tracing project that owns the trace.'
name: project_id
in: query
required: true
schema:
type: string
format: uuid
title: Project Id
- description: '`selects` lists which properties to include on each returned run (repeatable query parameter). Accepts any value of the `RunSelectField` enum. If omitted, only `id` is returned.'
name: selects
in: query
style: form
explode: true
schema:
type: array
items:
enum:
- ID
- NAME
- RUN_TYPE
- STATUS
- START_TIME
- END_TIME
- LATENCY_SECONDS
- FIRST_TOKEN_TIME
- ERROR
- ERROR_PREVIEW
- EXTRA
- METADATA
- EVENTS
- INPUTS
- INPUTS_PREVIEW
- OUTPUTS
- OUTPUTS_PREVIEW
- MANIFEST
- PARENT_RUN_IDS
- PROJECT_ID
- TRACE_ID
- THREAD_ID
- DOTTED_ORDER
- IS_ROOT
- REFERENCE_EXAMPLE_ID
- REFERENCE_DATASET_ID
- TOTAL_TOKENS
- PROMPT_TOKENS
- COMPLETION_TOKENS
- TOTAL_COST
- PROMPT_COST
- COMPLETION_COST
- PROMPT_TOKEN_DETAILS
- COMPLETION_TOKEN_DETAILS
- PROMPT_COST_DETAILS
- COMPLETION_COST_DETAILS
- PRICE_MODEL_ID
- TAGS
- APP_PATH
- ATTACHMENTS
- THREAD_EVALUATION_TIME
- IS_IN_DATASET
- SHARE_URL
- FEEDBACK_STATS
type: string
title: Selects
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/query.QueryTraceResponseBody'
'400':
description: Bad Request
content:
application/json:
schema:
type: object
additionalProperties:
type: string
'401':
description: Unauthorized
content:
application/json:
schema:
type: object
additionalProperties:
type: string
components:
schemas:
runs.ErrorResponse:
type: object
properties:
details:
description: Optional error details as JSON string
type: string
example: '{"field":"dataset_id","reason":"required"}'
error:
description: Error message
type: string
example: 'Invalid request: missing required fields'
runs.Run:
type: object
properties:
dotted_order:
type: string
end_time:
type: string
error:
type: string
events:
type: array
items:
type: object
additionalProperties: true
extra:
type: object
additionalProperties: true
id:
type: string
input_attachments:
type: object
additionalProperties: true
inputs:
type: object
additionalProperties: true
name:
type: string
output_attachments:
type: object
additionalProperties: true
outputs:
type: object
additionalProperties: true
parent_run_id:
type: string
reference_example_id:
type: string
run_type:
type: string
enum:
- tool
- chain
- llm
- retriever
- embedding
- prompt
- parser
serialized:
type: object
additionalProperties: true
session_id:
type: string
session_name:
type: string
start_time:
type: string
status:
type: string
tags:
type: array
items:
type: string
trace_id:
type: string
query.RunFeedbackStats:
type: object
additionalProperties:
$ref: '#/components/schemas/query.RunFeedbackStat'
query.RunPromptTokenDetails:
type: object
properties:
raw:
description: '`raw` maps each category name to its prompt-token count.'
type: object
additionalProperties:
type: integer
format: int64
query.RunAttachmentURLs:
type: object
additionalProperties:
type: string
query.QueryTraceResponseBody:
type: object
properties:
items:
description: '`items` lists runs in the trace for the requested time window, in `start_time` order.'
type: array
items:
$ref: '#/components/schemas/query.RunResponse'
query.RunCompletionTokenDetails:
type: object
properties:
raw:
description: '`raw` maps each category name to its completion-token count.'
type: object
additionalProperties:
type: integer
format: int64
query.RunFeedbackStat:
type: object
properties:
avg:
description: '`avg` is the arithmetic mean of numeric feedback scores for this key on the run, or `null` when no numeric score has been recorded (for example purely categorical feedback).'
type: number
example: 0.87
comments:
description: '`comments` is a sample of human-readable comments attached to feedback points for this key, in no particular order. May be empty; is not exhaustive when many comments exist.'
type: array
items:
type: string
example:
- good answer
- needs citation
contains_thread_feedback:
description: '`contains_thread_feedback` is true when at least one feedback point for this key was submitted at the thread level (rather than at an individual run). Always false on responses that already describe a single run in isolation.'
type: boolean
example: false
errors:
description: '`errors` is the number of feedback points recorded as errors rather than successful scores (for example an automated evaluator that raised an exception). Defaults to 0 when no errors occurred.'
type: integer
default: 0
example: 0
max:
description: '`max` is the largest numeric feedback score recorded for this key on the run, or `null` when no numeric score has been recorded.'
type: number
example: 0.95
min:
description: '`min` is the smallest numeric feedback score recorded for this key on the run, or `null` when no numeric score has been recorded.'
type: number
example: 0.8
n:
description: '`n` is the number of feedback points recorded for this key on the run. For numeric feedback this is the sample size behind `avg`, `min`, `max`, and `stdev`; for categorical feedback it is the sum of the `values` counts.'
type: integer
example: 42
sources:
description: '`sources` is a sample of feedback sources for this key. Each entry is either a plain string identifier (for example `"api"`, `"app"`, `"model"`) or a JSON object describing a synthetic source (for example `{"type": "__ls_composite_feedback"}` for a computed aggregate). Clients must tolerate both shapes.'
type: array
items: {}
stdev:
description: '`stdev` is the sample standard deviation of numeric feedback scores for this key on the run, or `null` when it cannot be computed (for example fewer than two numeric scores, or purely categorical feedback).'
type: number
example: 0.05
values:
description: '`values` is the distribution of categorical feedback labels for this key, mapping each label to its occurrence count. Empty (`{}`) for purely numeric feedback.'
type: object
additionalProperties:
type: integer
format: int64
query.RunStatus:
type: string
enum:
- SUCCESS
- ERROR
- PENDING
x-enum-varnames:
- RunStatusSuccess
- RunStatusError
- RunStatusPending
query.RunResponse:
type: object
properties:
app_path:
description: '`app_path` identifies the application code location that produced this run, if recorded.'
type: string
example: /app/chains/chat.py:invoke
attachments:
description: '`attachments` maps each attachment file name to a pre-signed HTTPS download URL.'
allOf:
- $ref: '#/components/schemas/query.RunAttachmentURLs'
example:
'{"output.png"': '"https://storage.example.com/bucket/key?X-Amz-Signature=abc"}'
completion_cost:
description: '`completion_cost` is estimated USD cost for the completion.'
type: number
example: 0.0003
completion_cost_details:
description: '`completion_cost_details` is the per-category USD breakdown of `completion_cost`. Categories mirror `completion_token_details`. Returned only when the `COMPLETION_COST_DETAILS` field is requested.'
allOf:
- $ref: '#/components/schemas/query.RunCompletionCostDetails'
completion_token_details:
description: '`completion_token_details` is the per-category breakdown of `completion_tokens`. Category names are model-specific (for example `reasoning`, `audio`). Returned only when the `COMPLETION_TOKEN_DETAILS` field is requested.'
allOf:
- $ref: '#/components/schemas/query.RunCompletionTokenDetails'
completion_tokens:
description: '`completion_tokens` is the completion-side token count.'
type: integer
example: 150
dotted_order:
description: '`dotted_order` is the hierarchical ordering key for trace trees.'
type: string
example: 20240115T103000000000Z018e4c7ea9fb7ef0a5b66ea3a82e9327.
end_time:
description: '`end_time` is when the run ended (RFC3339 date-time). JSON null if the run has not finished yet.'
type: string
format: date-time
example: '2024-01-15T10:30:01.500Z'
error:
description: '`error` is the error message when `status` indicates failure.'
type: string
example: context deadline exceeded
error_preview:
description: '`error_preview` is a truncated plain-text error snippet.'
type: string
events:
description: '`events` is the ordered list of run events (for example streaming tokens).'
type: array
items:
$ref: '#/components/schemas/query.RunEvent'
extra:
description: '`extra` is additional runtime JSON attached to the run.'
type: object
feedback_stats:
description: '`feedback_stats` aggregates feedback scores keyed by feedback key.'
allOf:
- $ref: '#/components/schemas/query.RunFeedbackStats'
first_token_time:
description: '`first_token_time` is when the first output token was produced (RFC3339 date-time), when recorded for streamed runs.'
type: string
format: date-time
example: '2024-01-15T10:30:00.312Z'
id:
description: '`id` is this run''s UUID.'
type: string
format: uuid
example: 018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327
inputs:
description: '`inputs` is the run input payload (arbitrary JSON object).'
type: object
inputs_preview:
description: '`inputs_preview` is a truncated plain-text preview of inputs.'
type: string
is_in_dataset:
description: '`is_in_dataset` is true when this run is linked to a dataset example.'
type: boolean
example: true
is_root:
description: '`is_root` is true when this run has no parent (it is the trace root).'
type: boolean
example: true
latency_seconds:
description: '`latency_seconds` is wall-clock duration from start to end in seconds.'
type: number
example: 1.523
manifest:
description: '`manifest` is the serialized configuration of the traced component (for example the model parameters, prompt template, or pipeline definition), when recorded.'
type: object
metadata:
description: '`metadata` is arbitrary user-defined JSON metadata.'
type: object
name:
description: '`name` is a human-readable label for the run (for example the model name, function name, or step name chosen when the run was traced).'
type: string
example: ChatOpenAI
outputs:
description: '`outputs` is the run output payload (arbitrary JSON object).'
type: object
outputs_preview:
description: '`outputs_preview` is a truncated plain-text preview of outputs.'
type: string
parent_run_ids:
description: '`parent_run_ids` lists ancestor run UUIDs from the trace root down to the direct parent.'
type: array
items:
type: string
format: uuid
example:
- 018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327
- a1b2c3d4-e5f6-4a5b-8c9d-0e1f2a3b4c5d
price_model_id:
description: '`price_model_id` identifies the pricing model UUID used for cost estimates, when recorded.'
type: string
format: uuid
example: e5f6a7b8-c9d0-4e1f-2a3b-4c5d6e7f8a9b
project_id:
description: '`project_id` is the tracing project UUID this run was logged to.'
type: string
format: uuid
example: 018e4c7e-a9fb-7ef0-a5b6-6ea3a82e9327
prompt_cost:
description: '`prompt_cost` is estimated USD cost for the prompt.'
type: number
example: 0.0002
prompt_cost_details:
description: '`prompt_cost_details` is the per-category USD breakdown of `prompt_cost`. Categories mirror `prompt_token_details`. Returned only when the `PROMPT_COST_DETAILS` field is requested.'
allOf:
- $ref: '#/components/schemas/query.RunPromptCostDetails'
prompt_token_details:
description: '`prompt_token_details` is the per-category breakdown of `prompt_tokens`. Category names are model-specific (for example `cache_read`, `cache_write`). Returned only when the `PROMPT_TOKEN_DETAILS` field is requested.'
allOf:
- $ref: '#/components/schemas/query.RunPromptTokenDetails'
prompt_tokens:
description: '`prompt_tokens` is the prompt-side token count.'
type: integer
example: 200
reference_dataset_id:
description: '`reference_dataset_id` is the dataset UUID for the reference example, if any.'
type: string
format: uuid
example: c3d4e5f6-a7b8-4c5d-0e1f-2a3b4c5d6e7f
reference_example_id:
description: '`reference_example_id` is the dataset example UUID this run was compared against, if any.'
type: string
format: uuid
example: b2c3d4e5
# --- truncated at 32 KB (44 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/langchain/refs/heads/main/openapi/langchain-runs-api-openapi.yml