Roark · Schema

BenchmarkLeaderboardRow

CompanyVoice AITestingEvaluationBenchmarking

Properties

Name Type Description
target object
metrics array The projected cells for this row, measured under the requested condition.
View JSON Schema on GitHub

JSON Schema

roark-benchmark-leaderboard-row-schema.json Raw ↑
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://artifacts.apis.io/roark/json-schema/roark-benchmark-leaderboard-row-schema.json",
  "title": "BenchmarkLeaderboardRow",
  "x-generated": "2026-11-10",
  "x-method": "derived",
  "x-generator": "derive-json-schema.py",
  "x-source": "openapi/roark-benchmark-api-openapi.yml#/components/schemas/BenchmarkLeaderboardRow",
  "type": "object",
  "properties": {
    "target": {
      "$ref": "#/$defs/BenchmarkTarget"
    },
    "metrics": {
      "type": "array",
      "items": {
        "$ref": "#/$defs/BenchmarkResult"
      },
      "description": "The projected cells for this row, measured under the requested condition."
    }
  },
  "required": [
    "target",
    "metrics"
  ],
  "$defs": {
    "BenchmarkResult": {
      "type": "object",
      "properties": {
        "conditionKey": {
          "type": "string",
          "description": "What these numbers were measured under: from suite v1.3 an industry scenario (`healthcare`, `home-services`, `insurance`, `customer-support`), before that a flow variant. The empty string is the overall rollup across all conditions."
        },
        "conditionLabel": {
          "type": "string"
        },
        "metricKey": {
          "type": "string",
          "description": "Stable metric id, e.g. `response_time`."
        },
        "metricName": {
          "type": "string"
        },
        "metricKind": {
          "type": "string",
          "enum": [
            "NUMERIC",
            "BOOLEAN"
          ],
          "description": "Which statistics are populated on a result. NUMERIC fills p50/p95/mean/ci; BOOLEAN fills passRate and its interval."
        },
        "unitSymbol": {
          "type": [
            "string",
            "null"
          ],
          "description": "Display unit for numeric metrics, e.g. `ms`."
        },
        "n": {
          "type": "integer",
          "description": "Scored samples behind this aggregate."
        },
        "p50": {
          "type": [
            "number",
            "null"
          ]
        },
        "p95": {
          "type": [
            "number",
            "null"
          ]
        },
        "mean": {
          "type": [
            "number",
            "null"
          ]
        },
        "ciLow": {
          "type": [
            "number",
            "null"
          ],
          "description": "Low bound of the 95% CI of the mean (NUMERIC metrics)."
        },
        "ciHigh": {
          "type": [
            "number",
            "null"
          ]
        },
        "passRate": {
          "type": [
            "number",
            "null"
          ],
          "description": "Fraction passed, 0..1 (BOOLEAN metrics)."
        },
        "passRateCiLow": {
          "type": [
            "number",
            "null"
          ],
          "description": "Low bound of the 95% Wilson interval on the pass rate."
        },
        "passRateCiHigh": {
          "type": [
            "number",
            "null"
          ]
        }
      },
      "required": [
        "conditionKey",
        "conditionLabel",
        "metricKey",
        "metricName",
        "metricKind",
        "unitSymbol",
        "n",
        "p50",
        "p95",
        "mean",
        "ciLow",
        "ciHigh",
        "passRate",
        "passRateCiLow",
        "passRateCiHigh"
      ],
      "description": "One aggregate cell: a metric measured for one target under one condition"
    },
    "BenchmarkTarget": {
      "type": "object",
      "properties": {
        "targetKey": {
          "type": "string",
          "description": "Stable identity of the model/stack across sweeps. Cite this, and pass it to the target endpoints."
        },
        "targetName": {
          "type": "string",
          "description": "Human-readable name of the model/stack."
        },
        "suite": {
          "type": "string"
        },
        "suiteVersion": {
          "type": "string"
        },
        "publicationId": {
          "type": "string",
          "format": "uuid",
          "description": "Id of this one sweep. Changes every time the target is re-published, so it is a render key, not an identity."
        },
        "iterations": {
          "type": "integer",
          "description": "Calls the sweep requested per condition."
        },
        "sampleCallCount": {
          "type": "integer",
          "description": "Executed calls that contributed at least one score to these numbers."
        },
        "publishedAt": {
          "type": "string",
          "description": "When this sweep was published (ISO-8601), i.e. data freshness."
        },
        "supersededAt": {
          "type": [
            "string",
            "null"
          ],
          "description": "When a later sweep replaced this one. Null means it is the current generation."
        },
        "isCurrent": {
          "type": "boolean",
          "description": "Whether these are the numbers Roark publishes for this target today."
        }
      },
      "required": [
        "targetKey",
        "targetName",
        "suite",
        "suiteVersion",
        "publicationId",
        "iterations",
        "sampleCallCount",
        "publishedAt",
        "supersededAt",
        "isCurrent"
      ],
      "description": "A model or stack with a published result on a benchmark suite"
    }
  }
}

Work with this as data

Every JSON Schema here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for schemas

4 MCP tools reach this
  • find_json_schemasBrowse and filter every JSON Schema in the catalog.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This JSON Schema
curl "https://apis.io/api/v1/json-schemas/roark-benchmark-leaderboard-row"
All schemas
curl "https://apis.io/api/v1/json-schemas?limit=25"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

A second provider on the same verified email joins the account you already have.