Judgment Labs

Judgment Labs builds the continuous-improvement stack for AI agents: tooling to trace, evaluate, monitor, and improve agent behavior in production. Its open-source Judgeval SDK (Python and TypeScript, with Go and Java clients) instruments agent frameworks and model providers to capture traces, spans, and tool calls; Agent Judges and Code Judges score behavior against natural-language rubrics and deterministic checks; and Agent Behavior Monitoring plus automations and alerts surface regressions and failure modes. The platform is accessed through the Judgeval SDKs, the judgment CLI, and a hosted MCP server, and integrates with LangGraph, OpenAI Agents SDK, Claude Agent SDK, Google ADK, Vercel AI SDK, LiveKit, Pipecat, and OpenTelemetry / OpenInference tracing pipelines. The company is backed by Lightspeed with a $32M round.

Judgment Labs is listed with 1 API on the APIs.io network. Tagged areas include Company, Agents, Artificial Intelligence, Agent Evaluation, and Observability.

Judgment Labs’ developer surface includes documentation, API reference, getting-started guide, support, engineering blog, signup flow, CLI, and 17 more developer resources.

🌐 Visit website 📡 Profile data on GitHub

36.8/100 thin ▬ flat Agent 31/100 agent ready Front door AI 5/6 prominent Heartbeat 53/100 going silent saas Full breakdown ↓
scored 2026-11-06 · rubric v0.27.0 · Show this score on your site →

1 API 1 MCP Servers 2 Agent Skills

Company Agents Artificial Intelligence Agent Evaluation Observability Tracing Monitoring LLM Developer Tools MCP

Kin Score

Kin Score How this is scored →
scored 2026-11-06 · rubric v0.27.0

Composite quality — 36.8/100 · thin

Contract Quality 0.0 / 21

Developer Ergonomics 13.4 / 17

Access Clarity 5.8 / 17

Operational Transparency 2.9 / 11

Contract Governance 1.9 / 10

Discoverability 6.4 / 9

Regulatory Posture 3.4 / 15

Agent readiness — 31/100 · agent ready

Machine-Readable Contract 0 / 18

Agentic Access Contract 0 / 10

Documented Reversibility 0 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 0 / 9

Stable Error Semantics 0 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 0 / 7

Typed Event Surface 0 / 6

Agent Skills 5 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Delegated User Identity 6 / 6

Protected Resource Metadata 5 / 5

Registration Without a Human 6 / 6

Agentic Commerce Well-Known Document 0 / 5

Pagination 5 / 5

Distinct Operations 3 / 3

MCP Tool Quality 6 / 6

Regulatory Posture applies to this provider. Its tags matched the Horizontal (data, software, accessibility, platform) regime, so Regulatory Posture carries 15 points of the composite. If this regime is wrong for your business, say so on your provider repo — the applicability map is public and we will correct it.

Create-or-Update Ergonomics could not be measured. We hold no machine-readable contract for this provider to read, so there is nothing to measure a write surface against. Excluded rather than scored zero: never-measured and measured-empty are different facts. Publishing an OpenAPI is what makes this facet — and several others — scorable at all.

The six quality facets above are damped to 85 points between them, because the conditional facet above carries the other 15. That is why each facet's contribution is shown against a damped maximum: raising a quality facet moves the composite by 85% of its nominal weight, not 100%. The full arithmetic is at apis.io/rating/.

Show this score on your site.

It updates itself when the score changes. Paste this where you link to your API docs:

<a href="https://apis.io/providers/judgment-labs/"><img src="https://apis.io/badge/judgment-labs/card.svg" alt="Judgment Labs on APIs.io — Kin Score by API Evangelist"></a>

Other styles, a README pill and a shields.io endpoint are on the badge page.

Raise your score.

Agent readiness — checks not met, largest first (exact, from this profile):

Machine-Readable Contract +18 pts — api_specs includes an OpenAPI document
Agentic Access Contract +10 pts — agentic_access artifact present (x-agentic-access), graded by author
Idempotency +9 pts — OpenAPI mutating operations declare an idempotency key — a header, a request-body property, or a named de-duplication token whose own description says so (verified); else common[].type matches /idempoten/i (documented)
Stable Error Semantics +8 pts — OpenAPI $refs a shared error schema across 4xx/5xx responses (verified), else common[].type includes ErrorCatalog / ErrorCodes / DeclineCodes
A2A Agent Card +8 pts — a2a/-a2a.yml manifest, graded on conformance.grade against A2A 1.0.0
Rate-Limit Signaling +7 pts — OpenAPI responses document X-RateLimit-* / RateLimit headers (verified), else a rate_limits artifact or RateLimits link (documented)

7 more unmet checks in the Agent Readiness table above.

Kin Score — facets with the most composite points still available (weight × headroom):

Contract Quality scores 0.0 — up to +25.0 composite points
Access Clarity scores 34.2 — up to +13.2 composite points
Contract Governance scores 18.2 — up to +9.8 composite points

Each check links to what satisfies it. Publish the artifact, or tell us where it already lives, and the score moves on the next scoring run.

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from the provider profile at artifacts.apis.io/judgment-labs/apis.yml: open an issue on github.com/api-evangelist/judgment-labs to ask a question or point us at an artifact you publish. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

Markets this provider is in provisional

The Kin Score Market layer reads a provider inside its market: its position, and how deep its integrations go. Depth is only comparable within a position. Provisional and unweighted: not part of the Kin Score or agent readiness.

Market	Position	Depth	Registered MCP server	Declares compatibility
Agent Skills (SKILL.md packages)	distribution	9.6 (thin)	-	1

Registered MCP server and declares compatibility are report only — not weighted. Undeclared means we found nothing we can read, so there is no number: we never infer integrations.

Standards implemented 4

Standards this provider implements or carries. De facto ones are profiled and measured by the API Commons; formal ones come from the standards directory and are linked by the provider's own tags. The evidence column says how each claim was established — not that it was made.

Model Context Protocol (MCP)

measured — observed by APIs.io (verified)

Agent Skills

measured — observed by APIs.io (published)

OAuth 2.0 Protected Resource Metadata

measured — observed by APIs.io (verified)

Dynamic Client Registration

measured — observed by APIs.io (published)

APIs 1

Individual APIs this provider publishes, each with its own machine-readable definition.

Judgment Platform API

The backend platform API behind Judgeval — ingests agent traces/spans, runs evaluations and judges, and serves traces, sessions, behaviors, datasets, and automations. Consumed t...

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

Judgment Labs MCP Server

The Judgment MCP server exposes production agent data — traces, sessions, behaviors, judges, projects, views, datasets, prompts, and automations — to MCP-capable AI code editors...

MCP SERVER

Agent Skills 2

Packaged agent skills for driving this provider's APIs from an AI assistant.

judgment

AGENT SKILL

mcp-server-best-practices

AGENT SKILL

Service Levels 1

Uptime the provider promises or reports, as SLA4OAI. Read from its own pages; APIs.io runs no monitors.

Judgment Labs

SLA

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Judgment Labs Authentication

apiKey · 1 scheme

SECURITY

Judgment Labs Domain Security

TLSv1.3 · HSTS · DMARC

SECURITY

Resources

Get Started 4

Portal, sign-up, and the first successful call

DeveloperPortal

DeveloperPortal

GettingStarted

GettingStarted

Login

SignUp

SignUp

Documentation 2

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Agent Surfaces 3

MCP servers, agent skills, and machine-readable catalogs

MCPServer

MCPServer

AgentSkill

AgentSkill

LLMsTxt

LLMsTxt

Design & Contract 2

Pagination, idempotency, versioning, errors, and events

Conformance

Conformance

Lifecycle

Lifecycle

Build 4

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

Packages

SDKs

CLI

Access & Security 2

Authentication, authorization, and security posture

Authentication

Authentication

DomainSecurity

DomainSecurity

Operate 3

Status, limits, changes, and where to get help

Support

Support

StatusPage

StatusPage

ChangeLog

ChangeLog

Commercial 2

Pricing, plans, and the legal terms of use

TermsOfService

TermsOfService

PrivacyPolicy

PrivacyPolicy

Company 2

The organization behind the API

Website

Website

Blog

Blog

Source (apis.yml)

aid: judgment-labs
name: Judgment Labs
description: 'Judgment Labs builds the continuous-improvement stack for AI agents: tooling to trace, evaluate, monitor, and
  improve agent behavior in production. Its open-source Judgeval SDK (Python and TypeScript, with Go and Java clients) instruments
  agent frameworks and model providers to capture traces, spans, and tool calls; Agent Judges and Code Judges score behavior
  against natural-language rubrics and deterministic checks; and Agent Behavior Monitoring plus automations and alerts surface
  regressions and failure modes. The platform is accessed through the Judgeval SDKs, the judgment CLI, and a hosted MCP server,
  and integrates with LangGraph, OpenAI Agents SDK, Claude Agent SDK, Google ADK, Vercel AI SDK, LiveKit, Pipecat, and OpenTelemetry
  / OpenInference tracing pipelines. The company is backed by Lightspeed with a $32M round.'
url: https://raw.githubusercontent.com/api-evangelist/judgment-labs/refs/heads/main/apis.yml
deliveryModel:
  model: saas
  open_source: false
  commercial: true
  callable_host: false
  label: Hosted service · you call their endpoint
  confidence: medium
  source:
  - pricing
  generated: '2026-08-28'
  method: derived
accessModel:
  pricing: unknown
  onboarding: unknown
  trial: false
  try_now: false
  public: false
  label: Unknown
  confidence: low
  source:
  - authentication
  - security
  generated: '2026-11-06'
  method: derived
image: https://www.judgmentlabs.ai/logo/full_logo_dark.svg
x-type: company
x-source: vc-portfolio
x-backed-by:
- lightspeed-venture-partners
x-tier: stub
x-tier-reason: portfolio-lead
specificationVersion: '0.23'
created: '2026-07-17'
modified: '2026-07-19'
tags:
- Company
- Agents
- Artificial Intelligence
- Agent Evaluation
- Observability
- Tracing
- Monitoring
- LLM
- Developer Tools
- MCP
apis:
- name: Judgment Platform API
  description: The backend platform API behind Judgeval — ingests agent traces/spans, runs evaluations and judges, and serves
    traces, sessions, behaviors, datasets, and automations. Consumed through the Judgeval SDKs, the judgment CLI, and the
    hosted MCP server rather than a public OpenAPI; authenticated with a Judgment API key and organization ID.
  humanURL: https://docs.judgmentlabs.ai/documentation
  baseURL: https://api.judgmentlabs.ai
  properties:
  - type: Documentation
    url: https://docs.judgmentlabs.ai/documentation
  - type: Authentication
    url: authentication/judgment-labs-authentication.yml
common:
- type: Website
  url: https://www.judgmentlabs.ai
- type: DeveloperPortal
  url: https://docs.judgmentlabs.ai/
- type: Documentation
  url: https://docs.judgmentlabs.ai/documentation
- type: APIReference
  url: https://docs.judgmentlabs.ai/sdk-reference
- type: GettingStarted
  url: https://docs.judgmentlabs.ai/documentation
- type: Support
  url: mailto:support@judgmentlabs.ai
- type: Blog
  url: https://www.judgmentlabs.ai/blogs
- type: GitHubOrganization
  url: https://github.com/JudgmentLabs
- type: Login
  url: https://app.judgmentlabs.ai/login
- type: SignUp
  url: https://app.judgmentlabs.ai/login
- type: TermsOfService
  url: https://app.judgmentlabs.ai/terms
- type: PrivacyPolicy
  url: https://app.judgmentlabs.ai/privacy
- type: StatusPage
  url: https://status.judgmentlabs.ai
- type: Packages
  url: packages/judgment-labs-packages.yml
- type: SDKs
  url: packages/judgment-labs-packages.yml
- type: CLI
  url: cli/judgment-labs-cli.yml
- type: MCPServer
  url: mcp/judgment-labs-mcp.yml
- type: AgentSkill
  url: skills/_index.yml
- type: LLMsTxt
  url: llms/judgment-labs-llms.txt
- type: ChangeLog
  url: changelog/judgment-labs-changelog.yml
- type: Authentication
  url: authentication/judgment-labs-authentication.yml
- type: Conformance
  url: conformance/judgment-labs-conformance.yml
- type: Lifecycle
  url: lifecycle/judgment-labs-lifecycle.yml
- type: DomainSecurity
  url: security/judgment-labs-domain-security.yml
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com
- FN: APIs.json
  email: info@apis.io
x-enrichment:
  date: '2026-07-19'
  status: backfilled
  pass: local-v1
  note: backfilled from .gitignore signal + verified work evidence

Judgment Labs API — frequently asked questions

Does Judgment Labs have an API?

Yes. APIs.io lists 1 API from Judgment Labs, including Judgment Platform API.

How do I authenticate with the Judgment Labs API?

Judgment Labs documents its authentication (apiKey · 1 scheme). The details are profiled on APIs.io as Judgment Labs Authentication.

Does Judgment Labs have an MCP server?

Yes. APIs.io lists 1 MCP server for Judgment Labs: The Judgment MCP server exposes production agent data — traces, sessions, behaviors, judges, projects, views, datasets, prompts, and automations — to MCP-capable AI code editors and to the in-product

Is the Judgment Labs API ready for AI agents?

Judgment Labs has a Kin Score of 36.8 out of 100 (thin) and an Agent Readiness score of 30.9 (agent-ready), scored by API Evangelist on 2026-11-06.

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/judgment-labs"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/judgment-labs/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/judgment-labs/evidence"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.