Properties
| Name | Type | Description |
|---|---|---|
| bench_id | object | Benchmark ID; use instead of agent_name and benchmark_name. |
| agent_name | object | Use with benchmark_name to select its latest benchmark version. |
| benchmark_name | object | Use with agent_name to select the latest benchmark version. |
| bypass_ownership | boolean | Allow a supplied agent to differ from the benchmark owner within the same organization; run attribution uses the benchmark owner. |
| agent_id | object | Optional agent for ownership checks; not a benchmark selector. |
| base_model_slug | object | Registered base model; mutually exclusive with deployment_id. |
| options | object | Run settings; omission, null, and an empty object preserve defaults. |
| parent_checkpoint_id | object | Checkpoint to resume training from or evaluate; must match the selected base model. Checkpoint evaluations inherit supported token and turn limits from the source training run unless the evaluation ov |
| display_name | object | |
| deployment_id | object | Evaluate the exact hosted checkpoint and pool of a ready deployment. |
| task_split | object | Evaluate test tasks by default; all includes both test and train tasks. |
JSON Schema
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/api-evangelist/trajectory/main/json-schema/trajectory-start-eval-request-schema.json",
"title": "StartEvalRequest",
"x-generated": "2026-10-24",
"x-method": "derived",
"x-generator": "derive-json-schema.py",
"x-source": "openapi/trajectory-openapi.json#/components/schemas/StartEvalRequest",
"properties": {
"bench_id": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "Bench Id",
"description": "Benchmark ID; use instead of agent_name and benchmark_name."
},
"agent_name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "Agent Name",
"description": "Use with benchmark_name to select its latest benchmark version."
},
"benchmark_name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "Benchmark Name",
"description": "Use with agent_name to select the latest benchmark version."
},
"bypass_ownership": {
"type": "boolean",
"title": "Bypass Ownership",
"description": "Allow a supplied agent to differ from the benchmark owner within the same organization; run attribution uses the benchmark owner.",
"default": false
},
"agent_id": {
"anyOf": [
{
"type": "string",
"maxLength": 64,
"minLength": 1
},
{
"type": "null"
}
],
"title": "Agent Id",
"description": "Optional agent for ownership checks; not a benchmark selector."
},
"base_model_slug": {
"anyOf": [
{
"$ref": "#/$defs/BaseModelSlug"
},
{
"type": "null"
}
],
"description": "Registered base model; mutually exclusive with deployment_id."
},
"options": {
"anyOf": [
{
"$ref": "#/$defs/RunOptions"
},
{
"type": "null"
}
],
"description": "Run settings; omission, null, and an empty object preserve defaults."
},
"parent_checkpoint_id": {
"anyOf": [
{
"type": "string",
"maxLength": 64,
"minLength": 1
},
{
"type": "null"
}
],
"title": "Parent Checkpoint Id",
"description": "Checkpoint to resume training from or evaluate; must match the selected base model. Checkpoint evaluations inherit supported token and turn limits from the source training run unless the evaluation overrides them."
},
"display_name": {
"anyOf": [
{
"type": "string",
"maxLength": 256,
"minLength": 1
},
{
"type": "null"
}
],
"title": "Display Name"
},
"deployment_id": {
"anyOf": [
{
"type": "string",
"maxLength": 64,
"minLength": 1
},
{
"type": "null"
}
],
"title": "Deployment Id",
"description": "Evaluate the exact hosted checkpoint and pool of a ready deployment."
},
"task_split": {
"$ref": "#/$defs/EvalTaskSplit",
"description": "Evaluate test tasks by default; all includes both test and train tasks.",
"default": "test"
}
},
"additionalProperties": false,
"type": "object",
"$defs": {
"BaseModelSlug": {
"type": "string",
"enum": [
"thinkingmachines/Inkling-Small",
"Qwen/Qwen3.5-4B",
"Qwen/Qwen3.5-397B-A17B",
"Qwen/Qwen3.6-35B-A3B",
"Qwen/Qwen3.8-27B",
"qwen/qwen3-235b-a22b",
"nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16",
"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16",
"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16",
"nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B-BF16",
"openai/gpt-6.1-sol",
"openai/gpt-6-sol",
"openai/gpt-6-astra",
"openai/gpt-5.6-sol",
"openai/gpt-5.6-luna",
"anthropic/claude-opus-5.5",
"anthropic/claude-sonnet-5.5",
"openai/gpt-5-mini",
"openai/gpt-5.4-mini",
"openai/gpt-5.5",
"anthropic/claude-sonnet-4.6",
"google/gemini-3.1-pro-preview",
"z-ai/glm-5.3",
"z-ai/glm-5.3-flash",
"moonshotai/kimi-k3"
],
"title": "BaseModelSlug"
},
"EvalTaskSplit": {
"type": "string",
"enum": [
"test",
"all"
],
"title": "EvalTaskSplit"
},
"RunOptions": {
"properties": {
"intelligent_density_training": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"title": "Intelligent Density Training",
"description": "Enable intelligent density training."
},
"algorithm": {
"anyOf": [
{
"$ref": "#/$defs/TrainingAlgorithm"
},
{
"type": "null"
}
],
"description": "Training algorithm; omit to preserve the model default."
},
"train_on_truncation": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"title": "Train On Truncation",
"description": "Include truncated trajectories in training."
},
"train_on_error": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"title": "Train On Error",
"description": "Train on error-terminated trajectories, using zero when no reward is recorded."
},
"train_on_timeout": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"title": "Train On Timeout",
"description": "Train on execution and sandbox timeouts, using zero when no reward is recorded."
},
"evaluation_enabled": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"title": "Evaluation Enabled",
"description": "Enable evaluations during training."
},
"evaluation_frequency": {
"anyOf": [
{
"type": "integer",
"maximum": 100.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Evaluation Frequency",
"description": "Optimizer steps between evaluations during training: 0 disables evaluation, 1 evaluates after every step, and 2 after every other step. When enabled, the initial policy and final step are always evaluated."
},
"evaluation_max_samples": {
"anyOf": [
{
"type": "integer",
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Evaluation Max Samples",
"description": "Maximum distinct tasks per evaluation. Standalone evals run every task in the selected scope when omitted."
},
"evaluation_samples_per_task": {
"anyOf": [
{
"type": "integer",
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Evaluation Samples Per Task"
},
"evaluation_max_active_rollouts": {
"anyOf": [
{
"type": "integer",
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Evaluation Max Active Rollouts"
},
"evaluation_collection_timeout_seconds": {
"anyOf": [
{
"type": "number",
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Evaluation Collection Timeout Seconds",
"description": "Total wait for outstanding evaluation rollouts; zero stops waiting immediately."
},
"learning_rate": {
"anyOf": [
{
"type": "number",
"maximum": 0.01,
"minimum": 1e-08
},
{
"type": "null"
}
],
"title": "Learning Rate",
"description": "Base optimizer learning rate."
},
"lr_warmup_steps": {
"anyOf": [
{
"type": "integer",
"maximum": 100000.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Lr Warmup Steps",
"description": "Reported training steps used for warmup, including skipped optimizer updates."
},
"lr_hold_steps": {
"anyOf": [
{
"type": "integer",
"maximum": 100000.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Lr Hold Steps",
"description": "Training steps to hold the base rate after warmup, including skipped updates, before cosine decay."
},
"min_lr_ratio": {
"anyOf": [
{
"type": "number",
"maximum": 1.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Min Lr Ratio",
"description": "Cosine decay floor as a fraction of the base learning rate."
},
"reasoning_effort": {
"anyOf": [
{
"type": "string",
"enum": [
"off",
"on",
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
]
},
{
"type": "null"
}
],
"title": "Reasoning Effort",
"description": "Reasoning effort supported by the model; off disables thinking, on enables models without effort levels, and null preserves the model default."
},
"temperature": {
"anyOf": [
{
"type": "number",
"maximum": 2.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Temperature",
"description": "Sampling temperature; null preserves the model default."
},
"top_p": {
"anyOf": [
{
"type": "number",
"maximum": 1.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Top P",
"description": "Nucleus sampling probability; null preserves the model default."
},
"top_k": {
"anyOf": [
{
"type": "integer",
"minimum": -1.0
},
{
"type": "null"
}
],
"title": "Top K",
"description": "Sample from the top k tokens; -1 disables the limit, and null preserves the model default."
},
"evaluation_reasoning_effort": {
"anyOf": [
{
"type": "string",
"enum": [
"off",
"on",
"minimal",
"low",
"medium",
"high",
"xhigh",
"max"
]
},
{
"type": "null"
}
],
"title": "Evaluation Reasoning Effort",
"description": "Evaluation reasoning effort; null preserves the evaluation default."
},
"evaluation_temperature": {
"anyOf": [
{
"type": "number",
"maximum": 2.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Evaluation Temperature",
"description": "Evaluation sampling temperature; null preserves the evaluation default."
},
"evaluation_top_p": {
"anyOf": [
{
"type": "number",
"maximum": 1.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Evaluation Top P",
"description": "Evaluation nucleus sampling probability; null preserves the evaluation default."
},
"evaluation_top_k": {
"anyOf": [
{
"type": "integer",
"minimum": -1.0
},
{
"type": "null"
}
],
"title": "Evaluation Top K",
"description": "Evaluation top-k token limit; -1 disables the limit, and null preserves the evaluation default."
},
"num_steps": {
"anyOf": [
{
"type": "integer",
"maximum": 100000.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Num Steps",
"description": "Number of optimizer steps to run."
},
"train_batch_size": {
"anyOf": [
{
"type": "integer",
"maximum": 256.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Train Batch Size",
"description": "Eligible task-instance groups per optimizer update."
},
"group_size": {
"anyOf": [
{
"type": "integer",
"maximum": 32.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Group Size",
"description": "Rollout samples generated for each training instance; GSPO and GRPO require at least 2, while Reinforce++ requires 1."
},
"samples_per_instance": {
"anyOf": [
{
"type": "integer",
"maximum": 32.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Samples Per Instance",
"description": "Deprecated alias for group_size; use group_size for new requests.",
"deprecated": true
},
"min_samples_per_group": {
"anyOf": [
{
"type": "integer",
"maximum": 32.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Min Samples Per Group",
"description": "Minimum valid samples required for a training group; must not exceed group_size. Omit to preserve the model default, capped at the group size."
},
"max_active_rollouts": {
"anyOf": [
{
"type": "integer",
"maximum": 512.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Max Active Rollouts",
"description": "Maximum active training rollouts per run."
},
"max_steps_off_policy": {
"anyOf": [
{
"type": "integer",
"maximum": 100.0,
"minimum": 0.0
},
{
"type": "null"
}
],
"title": "Max Steps Off Policy",
"description": "Maximum optimizer steps a training rollout's starting policy may trail the current policy. Older in-flight rollouts are cancelled and older completed rollouts are discarded; 0 trains only on rollouts started from the current policy."
},
"execution_timeout_seconds": {
"anyOf": [
{
"type": "integer",
"maximum": 36000.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Execution Timeout Seconds",
"description": "Maximum execution time per training or evaluation rollout, including policy and grading. The clock starts when the harness runs, after queueing and sandbox startup. Collection timeouts must cover this budget and are not adjusted automatically."
},
"batch_collection_idle_timeout_seconds": {
"anyOf": [
{
"type": "integer",
"maximum": 38400.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Batch Collection Idle Timeout Seconds",
"description": "Maximum wall-clock time to assemble one training batch. The clock starts when collection begins and does not reset as groups or in-flight rollouts arrive. The legacy field name is retained for API compatibility. Must cover rollout execution and is not adjusted automatically."
},
"max_output_tokens_per_step": {
"anyOf": [
{
"type": "integer",
"maximum": 262144.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Max Output Tokens Per Step",
"description": "On an endpoint created for the run, maximum assistant output tokens per model step; replaces the harness request's max_tokens."
},
"max_total_tokens_per_trajectory": {
"anyOf": [
{
"type": "integer",
"maximum": 2097152.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Max Total Tokens Per Trajectory",
"description": "On an endpoint created for the run, maximum assistant output tokens across all steps of one trajectory, including steps after context compaction, so it can exceed the model's context window. Reaching this cap during a reply can end the trajectory as LIMIT_REACHED before the harness logs reward."
},
"max_turns_per_trajectory": {
"anyOf": [
{
"type": "integer",
"maximum": 500.0,
"minimum": 1.0
},
{
"type": "null"
}
],
"title": "Max Turns Per Trajectory",
"description": "Maximum agent and environment turns allowed in each trajectory."
}
},
"additionalProperties": false,
"type": "object",
"title": "RunOptions"
},
"TrainingAlgorithm": {
"type": "string",
"enum": [
"gspo",
"grpo",
"reinforce_plus_plus"
],
"title": "TrainingAlgorithm"
}
}
}Work with this as data
Every JSON Schema here is available over the APIs.io API and to AI agents over MCP.
MCP server
One button, every client — Claude, Cursor, VS Code and the rest.
https://apis.io/mcp
Tools for schemas
4 MCP tools reach this
find_json_schemasBrowse and filter every JSON Schema in the catalog.apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.resolveTurn a domain, URL or GitHub org into the provider it belongs to.find_cohortsEvery scored population of providers in the catalog.
Call it yourself
curl for this page
This JSON Schema
curl "https://apis.io/api/v1/json-schemas/trajectory-start-eval-request"
All schemas
curl "https://apis.io/api/v1/json-schemas?limit=25"Discovery needs no key. Ratings and market analysis are Pro.
Get an API key
Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.
A second provider on the same verified email joins the account you already have.