Judgment Labs

Judgment Labs builds the continuous-improvement stack for AI agents: tooling to trace, evaluate, monitor, and improve agent behavior in production. Its open-source Judgeval SDK (Python and TypeScript, with Go and Java clients) instruments agent frameworks and model providers to capture traces, spans, and tool calls; Agent Judges and Code Judges score behavior against natural-language rubrics and deterministic checks; and Agent Behavior Monitoring plus automations and alerts surface regressions and failure modes. The platform is accessed through the Judgeval SDKs, the judgment CLI, and a hosted MCP server, and integrates with LangGraph, OpenAI Agents SDK, Claude Agent SDK, Google ADK, Vercel AI SDK, LiveKit, Pipecat, and OpenTelemetry / OpenInference tracing pipelines. The company is backed by Lightspeed with a $32M round.

Judgment Labs publishes 1 API on the APIs.io network. Tagged areas include Company, Agents, Artificial Intelligence, Agent Evaluation, and Observability.

Judgment Labs’ developer surface includes documentation, API reference, getting-started guide, support, engineering blog, signup flow, CLI, and 17 more developer resources.

🌐 Visit website 📡 Source on GitHub

35.8/100 thin ▬ flat Agent 10/100 agent aware saas Full breakdown ↓
scored 2026-09-08 · rubric v0.20.0

1 APIs 1 MCP Servers

Company Agents Artificial Intelligence Agent Evaluation Observability Tracing Monitoring LLM Developer Tools MCP

Kin Score

Kin Score How this is scored →
scored 2026-09-08 · rubric v0.20.0

Composite quality — 35.8/100 · thin

Contract Quality 0.0 / 25

Developer Ergonomics 15.7 / 20

Access Clarity 6.8 / 20

Operational Transparency 3.4 / 13

Contract Governance 2.2 / 12

Discoverability 7.6 / 10

Agent readiness — 10/100 · agent aware

Machine-Readable Contract 0 / 18

Agentic Access Contract 0 / 10

Documented Reversibility 0 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 0 / 9

Stable Error Semantics 0 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 0 / 7

Typed Event Surface 0 / 6

Agent Skills 5 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Delegated User Identity 0 / 6

Protected Resource Metadata 0 / 5

Registration Without a Human 0 / 6

Agentic Commerce Well-Known Document 0 / 5

Create-or-Update Ergonomics could not be measured. We hold no machine-readable contract for this provider to read, so there is nothing to measure a write surface against. Excluded rather than scored zero: never-measured and measured-empty are different facts. Publishing an OpenAPI is what makes this facet — and several others — scorable at all.

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from github.com/api-evangelist/judgment-labs: open an issue to ask a question, or submit a pull request to add artifacts. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

APIs 1

Individual APIs this provider publishes, each with its own machine-readable definition.

Judgment Platform API

The backend platform API behind Judgeval — ingests agent traces/spans, runs evaluations and judges, and serves traces, sessions, behaviors, datasets, and automations. Consumed t...

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

Judgment Labs MCP Server

The Judgment MCP server exposes production agent data — traces, sessions, behaviors, judges, projects, views, datasets, prompts, and automations — to MCP-capable AI code editors...

MCP SERVER

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Judgment Labs Authentication

apiKey · 1 scheme

SECURITY

Judgment Labs Domain Security

TLSv1.3 · HSTS · DMARC

SECURITY

Resources

Get Started 4

Portal, sign-up, and the first successful call

DeveloperPortal

DeveloperPortal

GettingStarted

GettingStarted

Login

SignUp

SignUp

Documentation 2

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Agent Surfaces 3

MCP servers, agent skills, and machine-readable catalogs

MCPServer

MCPServer

AgentSkill

AgentSkill

LLMsTxt

LLMsTxt

Design & Contract 2

Pagination, idempotency, versioning, errors, and events

Conformance

Conformance

Lifecycle

Lifecycle

Build 4

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

Packages

SDKs

CLI

Access & Security 2

Authentication, authorization, and security posture

Authentication

Authentication

DomainSecurity

DomainSecurity

Operate 3

Status, limits, changes, and where to get help

Support

Support

StatusPage

StatusPage

ChangeLog

ChangeLog

Commercial 2

Pricing, plans, and the legal terms of use

TermsOfService

TermsOfService

PrivacyPolicy

PrivacyPolicy

Company 2

The organization behind the API

Website

Website

Blog

Blog

Source (apis.yml)

aid: judgment-labs
name: Judgment Labs
description: 'Judgment Labs builds the continuous-improvement stack for AI agents: tooling to trace, evaluate, monitor, and
  improve agent behavior in production. Its open-source Judgeval SDK (Python and TypeScript, with Go and Java clients) instruments
  agent frameworks and model providers to capture traces, spans, and tool calls; Agent Judges and Code Judges score behavior
  against natural-language rubrics and deterministic checks; and Agent Behavior Monitoring plus automations and alerts surface
  regressions and failure modes. The platform is accessed through the Judgeval SDKs, the judgment CLI, and a hosted MCP server,
  and integrates with LangGraph, OpenAI Agents SDK, Claude Agent SDK, Google ADK, Vercel AI SDK, LiveKit, Pipecat, and OpenTelemetry
  / OpenInference tracing pipelines. The company is backed by Lightspeed with a $32M round.'
url: https://raw.githubusercontent.com/api-evangelist/judgment-labs/refs/heads/main/apis.yml
deliveryModel:
  model: saas
  open_source: false
  commercial: true
  callable_host: false
  label: Hosted service · you call their endpoint
  confidence: medium
  source:
  - pricing
  generated: '2026-08-28'
  method: derived
accessModel:
  pricing: unknown
  onboarding: unknown
  trial: false
  try_now: false
  public: false
  label: Unknown
  confidence: low
  source:
  - authentication
  - security
  generated: '2026-09-03'
  method: derived
image: https://www.judgmentlabs.ai/logo/full_logo_dark.svg
x-type: company
x-source: vc-portfolio
x-backed-by:
- lightspeed-venture-partners
x-tier: stub
x-tier-reason: portfolio-lead
specificationVersion: '0.23'
created: '2026-07-17'
modified: '2026-07-19'
tags:
- Company
- Agents
- Artificial Intelligence
- Agent Evaluation
- Observability
- Tracing
- Monitoring
- LLM
- Developer Tools
- MCP
apis:
- name: Judgment Platform API
  description: The backend platform API behind Judgeval — ingests agent traces/spans, runs evaluations and judges, and serves
    traces, sessions, behaviors, datasets, and automations. Consumed through the Judgeval SDKs, the judgment CLI, and the
    hosted MCP server rather than a public OpenAPI; authenticated with a Judgment API key and organization ID.
  humanURL: https://docs.judgmentlabs.ai/documentation
  baseURL: https://api.judgmentlabs.ai
  properties:
  - type: Documentation
    url: https://docs.judgmentlabs.ai/documentation
  - type: Authentication
    url: authentication/judgment-labs-authentication.yml
common:
- type: Website
  url: https://www.judgmentlabs.ai
- type: DeveloperPortal
  url: https://docs.judgmentlabs.ai/
- type: Documentation
  url: https://docs.judgmentlabs.ai/documentation
- type: APIReference
  url: https://docs.judgmentlabs.ai/sdk-reference
- type: GettingStarted
  url: https://docs.judgmentlabs.ai/documentation
- type: Support
  url: mailto:support@judgmentlabs.ai
- type: Blog
  url: https://www.judgmentlabs.ai/blogs
- type: GitHubOrganization
  url: https://github.com/JudgmentLabs
- type: Login
  url: https://app.judgmentlabs.ai/login
- type: SignUp
  url: https://app.judgmentlabs.ai/login
- type: TermsOfService
  url: https://app.judgmentlabs.ai/terms
- type: PrivacyPolicy
  url: https://app.judgmentlabs.ai/privacy
- type: StatusPage
  url: https://status.judgmentlabs.ai
- type: Packages
  url: packages/judgment-labs-packages.yml
- type: SDKs
  url: packages/judgment-labs-packages.yml
- type: CLI
  url: cli/judgment-labs-cli.yml
- type: MCPServer
  url: mcp/judgment-labs-mcp.yml
- type: AgentSkill
  url: skills/_index.yml
- type: LLMsTxt
  url: llms/judgment-labs-llms.txt
- type: ChangeLog
  url: changelog/judgment-labs-changelog.yml
- type: Authentication
  url: authentication/judgment-labs-authentication.yml
- type: Conformance
  url: conformance/judgment-labs-conformance.yml
- type: Lifecycle
  url: lifecycle/judgment-labs-lifecycle.yml
- type: DomainSecurity
  url: security/judgment-labs-domain-security.yml
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com
- FN: APIs.json
  email: info@apis.io
x-enrichment:
  date: '2026-07-19'
  status: backfilled
  pass: local-v1
  note: backfilled from .gitignore signal + verified work evidence

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/judgment-labs"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/judgment-labs/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/judgment-labs/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.