Pi Labs

Pi Labs (withpi.ai) is an applied-AI company, backed by Accel and later acquired by Microsoft, that builds tooling to evaluate and enhance AI applications. Its core product, Pi Scorer, is a deterministic, roughly 200ms foundation model that scores any text against a rubric of natural-language questions — reported to be about 500x more efficient than using an LLM as a judge. Teams use it to calibrate rubrics, gate and rank LLM outputs, and close the loop across training, inference, and feedback. The scoring engine is exposed as an OpenAPI-compatible REST API at api.withpi.ai with official Python and TypeScript SDKs (both published as "withpi"), a copilot for building scoring specs, and a Promptfoo integration (Pi Scorer).

Pi Labs publishes 1 API on the APIs.io network. Tagged areas include Company, Artificial Intelligence, Evaluation, LLM, and Scoring.

Pi Labs’ developer surface includes documentation, API reference, getting-started guide, changelog, authentication, and 8 more developer resources.

🌐 Visit website 📡 Source on GitHub
16.9/100 emerging ▬ flat Agent 3/100 human only library Full breakdown ↓
scored 2026-09-08 · rubric v0.20.0
1 APIs
CompanyArtificial IntelligenceEvaluationLLMScoringMachine-LearningDeveloper Tools

Kin Score

Kin Score Kin Score How this is scored →
scored 2026-09-08 · rubric v0.20.0
Composite quality — 16.9/100 · emerging
Contract Quality 0.0 / 25
Developer Ergonomics 8.0 / 20
Access Clarity 0.0 / 20
Operational Transparency 1.4 / 13
Contract Governance 0.0 / 12
Discoverability 7.6 / 10
Agent readiness — 3/100 · human only
Machine-Readable Contract 0 / 18
Agentic Access Contract 0 / 10
Documented Reversibility 0 / 6
MCP Server 0 / 12
Machine-Readable Auth 10 / 10
Idempotency 0 / 9
Stable Error Semantics 0 / 8
Request/Response Examples 0 / 7
Rate-Limit Signaling 0 / 7
Typed Event Surface 0 / 6
Agent Skills 5 / 5
Well-Known Catalog 0 / 4
Consent & Bot Identity 0 / 3
A2A Agent Card 0 / 8
Dry-Run / Simulate Mode 0 / 4
Delegated User Identity 0 / 6
Protected Resource Metadata 0 / 5
Registration Without a Human 0 / 6
Agentic Commerce Well-Known Document 0 / 5
Create-or-Update Ergonomics could not be measured. We hold no machine-readable contract for this provider to read, so there is nothing to measure a write surface against. Excluded rather than scored zero: never-measured and measured-empty are different facts. Publishing an OpenAPI is what makes this facet — and several others — scorable at all.
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from github.com/api-evangelist/pi-labs: open an issue to ask a question, or submit a pull request to add artifacts. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

APIs 1

Individual APIs this provider publishes, each with its own machine-readable definition.

Pi Scoring API

OpenAPI-compatible REST API for scoring, calibrating, and optimizing LLM outputs against rubrics of natural-language questions. Authenticated with an API key (WITHPI_API_KEY) se...

Security Posture 1

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Pi Labs Authentication

apiKey · 1 scheme

SECURITY

Resources

Get Started 2

Portal, sign-up, and the first successful call

DeveloperPortal
DeveloperPortal
 GettingStarted
GettingStarted

Documentation 2

Reference material describing how the API behaves

Documentation
Documentation
 APIReference
APIReference

Agent Surfaces 2

MCP servers, agent skills, and machine-readable catalogs

AgentSkill
AgentSkill
 LLMsTxt
LLMsTxt

Design & Contract 1

Pagination, idempotency, versioning, errors, and events

Conventions
Conventions

Build 3

SDKs, sample code, and the tooling you integrate with

GitHubOrganization
GitHubOrganization
 Packages
Packages
 SDKs
SDKs

Access & Security 1

Authentication, authorization, and security posture

Authentication
Authentication

Operate 1

Status, limits, changes, and where to get help

ChangeLog
ChangeLog

Other 1

Properties that don't map to a standard resource type

ParentCompany
ParentCompany

Source (apis.yml)

apis.yml Raw ↑ 
aid: pi-labs
deliveryModel:
  model: library
  open_source: false
  commercial: false
  callable_host: false
  label: Library · distributed through package registries, no callable service
  confidence: low
  source:
  - none
  generated: '2026-08-28'
  method: derived
accessModel:
  pricing: unknown
  onboarding: unknown
  trial: false
  try_now: false
  public: false
  label: Unknown
  confidence: low
  source:
  - authentication
  generated: '2026-09-03'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/pi-labs.png
name: Pi Labs
description: Pi Labs (withpi.ai) is an applied-AI company, backed by Accel and later acquired by Microsoft, that builds tooling
  to evaluate and enhance AI applications. Its core product, Pi Scorer, is a deterministic, roughly 200ms foundation model
  that scores any text against a rubric of natural-language questions — reported to be about 500x more efficient than using
  an LLM as a judge. Teams use it to calibrate rubrics, gate and rank LLM outputs, and close the loop across training, inference,
  and feedback. The scoring engine is exposed as an OpenAPI-compatible REST API at api.withpi.ai with official Python and
  TypeScript SDKs (both published as "withpi"), a copilot for building scoring specs, and a Promptfoo integration (Pi Scorer).
url: https://raw.githubusercontent.com/api-evangelist/pi-labs/refs/heads/main/apis.yml
x-type: company
x-source: vc-portfolio
x-backed-by:
- accel
x-tier: stub
x-tier-reason: portfolio-lead
specificationVersion: '0.23'
created: '2026-07-17'
modified: '2026-07-20'
tags:
- Company
- Artificial Intelligence
- Evaluation
- LLM
- Scoring
- Machine-Learning
- Developer Tools
tags_raw:
- Company
- AI
- Evaluation
- LLM
- Scoring
- Machine Learning
- Developer Tools
apis:
- name: Pi Scoring API
  description: OpenAPI-compatible REST API for scoring, calibrating, and optimizing LLM outputs against rubrics of natural-language
    questions. Authenticated with an API key (WITHPI_API_KEY) sent as an Authorization bearer header.
  humanURL: https://code.withpi.ai
  baseURL: https://api.withpi.ai
  tags:
  - Scoring
  - Evaluation
  - LLM
  properties:
  - type: Documentation
    url: https://code.withpi.ai
  - type: Authentication
    url: authentication/pi-labs-authentication.yml
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com
- FN: APIs.json
  email: info@apis.io
common:
- type: ParentCompany
  url: https://apis.io/providers/microsoft/
  name: Microsoft
  x-relationship: acquired
  x-evidence: ai) is an applied-AI company, backed by Accel and later acquired by Microsoft, that builds tooling to evaluate
    and enhance AI applications.
- type: DeveloperPortal
  url: https://withpi.ai
- type: Documentation
  url: https://code.withpi.ai
- type: APIReference
  url: https://code.withpi.ai
- type: GettingStarted
  url: https://code.withpi.ai/quickstart
- type: GitHubOrganization
  url: https://github.com/withpi
- type: ChangeLog
  url: https://withpi.ai/release-notes
- type: Packages
  url: packages/pi-labs-packages.yml
- type: SDKs
  url: packages/pi-labs-packages.yml
- type: Authentication
  url: authentication/pi-labs-authentication.yml
- type: Conventions
  url: conventions/pi-labs-conventions.yml
- type: AgentSkill
  url: skills/_index.yml
- type: LLMsTxt
  url: llms/pi-labs-llms.txt
x-enrichment:
  date: '2026-07-20'
  status: enriched
  artifacts_added: 7
  pass: local-v1

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this
  • find_providersBrowse and filter every provider in the catalog.
  • get_provider_artifactsEvery artifact this provider publishes, grouped by type.
  • get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
  • get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
  • get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
  • get_provider_ratingPRO — composite, band, trend and facet scores.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This provider 
curl "https://apis.io/api/v1/providers/pi-labs"
All providers 
curl "https://apis.io/api/v1/providers?limit=25"
Every operation they expose 
curl "https://apis.io/api/v1/providers/pi-labs/operations?limit=25"
How their score was established 
curl "https://apis.io/api/v1/providers/pi-labs/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.