Evals · JSON Structure

Evals Scorer Structure

A function producing a score from (input, output, expected?, context?) — code, heuristic, LLM judge, human, reference-based, reference-free, or pairwise.

Type: object Properties: 11 Required: 3
EvalsLLM EvaluationAI QualityBenchmarksLLM as a JudgeObservabilityAgent EvaluationRAG EvaluationTest-Driven AI

Scorer is a JSON Structure definition published by Evals, describing 11 properties, of which 3 are required. It conforms to the https://json-structure.org/meta/core/v0/# meta-schema.

Properties

id name description type implementation judge_model judge_prompt scale threshold rubric tags

Meta-schema: https://json-structure.org/meta/core/v0/#

JSON Structure

evals-scorer-structure.json Raw ↑ 
{
  "$schema": "https://json-structure.org/meta/core/v0/#",
  "$id": "https://raw.githubusercontent.com/api-evangelist/evals/refs/heads/main/json-structure/evals-scorer-structure.json",
  "name": "Scorer",
  "description": "A function producing a score from (input, output, expected?, context?) — code, heuristic, LLM judge, human, reference-based, reference-free, or pairwise.",
  "type": "object",
  "properties": {
    "id": { "type": "string" },
    "name": { "type": "string" },
    "description": { "type": "string" },
    "type": {
      "type": "string",
      "enum": [
        "code",
        "llm_judge",
        "human",
        "heuristic",
        "reference_based",
        "reference_free",
        "pairwise"
      ]
    },
    "implementation": { "type": "string" },
    "judge_model": { "type": "string" },
    "judge_prompt": { "type": "string" },
    "scale": {
      "type": "object",
      "properties": {
        "min": { "type": "number" },
        "max": { "type": "number" },
        "step": { "type": "number" },
        "kind": {
          "type": "string",
          "enum": ["continuous", "binary", "ordinal", "categorical"]
        }
      }
    },
    "threshold": { "type": "number" },
    "rubric": { "type": "string" },
    "tags": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["id", "name", "type"]
}

Work with this as data

Every JSON Structure here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for json structure

4 MCP tools reach this
  • find_json_structuresBrowse and filter every JSON Structure in the catalog.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This JSON Structure 
curl "https://apis.io/api/v1/json-structures/evals-scorer-structure"
All json structure 
curl "https://apis.io/api/v1/json-structures?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.