Pi Labs

Pi Labs (withpi.ai) is an applied-AI company, backed by Accel and later acquired by Microsoft, that builds tooling to evaluate and enhance AI applications. Its core product, Pi Scorer, is a deterministic, roughly 200ms foundation model that scores any text against a rubric of natural-language questions — reported to be about 500x more efficient than using an LLM as a judge. Teams use it to calibrate rubrics, gate and rank LLM outputs, and close the loop across training, inference, and feedback. The scoring engine is exposed as an OpenAPI-compatible REST API at api.withpi.ai with official Python and TypeScript SDKs (both published as "withpi"), a copilot for building scoring specs, and a Promptfoo integration (Pi Scorer).

Pi Labs publishes 1 API on the APIs.io network. Tagged areas include Company, AI, Evaluation, LLM, and Scoring.

Pi Labs’ developer surface includes documentation, API reference, getting-started guide, changelog, authentication, and 7 more developer resources.

🌐 Visit website 📡 Source on GitHub

23.7/100 emerging ▬ flat Agent 14/100 human only Full breakdown ↓
scored 2026-07-27 · rubric v0.5

AccessSelf serve

1 APIs

CompanyAIEvaluationLLMScoringMachine LearningDeveloper Tools

Kin Score

Kin Score How this is scored →
scored 2026-07-27 · rubric v0.5

Composite quality — 23.7/100 · emerging

Contract Quality 0.0 / 25

Developer Ergonomics 11.7 / 20

Commercial Clarity 0.0 / 20

Operational Transparency 2.7 / 13

Governance 0.0 / 12

Discoverability 9.3 / 10

Agent readiness — 14/100 · human only

Machine-Readable Contract 0 / 18

Agentic Access Contract 0 / 15

MCP Server 0 / 12

Machine-Readable Auth 10 / 10

Idempotency 0 / 9

Stable Error Semantics 0 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 0 / 7

Typed Event Surface 0 / 6

Agent Skills 5 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. This rating is computed from github.com/api-evangelist/pi-labs: open an issue to ask a question, or submit a pull request to add artifacts. Want it done for you? Prioritized profiling — $2,500 →

APIs 1

Individual APIs this provider publishes, each with its own machine-readable definition.

Get Started 2

Portal, sign-up, and the first successful call

DeveloperPortal

DeveloperPortal

GettingStarted

GettingStarted

Documentation 2

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Agent Surfaces 2

MCP servers, agent skills, and machine-readable catalogs

AgentSkill

AgentSkill

LLMsTxt

LLMsTxt

Design & Contract 1

Pagination, idempotency, versioning, errors, and events

Conventions

Conventions

Build 3

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

GitHubOrganization

Packages

Packages

SDKs

SDKs

Access & Security 1

Authentication, authorization, and security posture

Authentication

Authentication

Operate 1

Status, limits, changes, and where to get help

ChangeLog

ChangeLog

Source (apis.yml)

aid: pi-labs
accessModel:
  pricing: unknown
  onboarding: self-serve
  trial: false
  try_now: false
  public: false
  label: Self-serve signup
  confidence: medium
  source:
  - authentication
  generated: '2026-07-22'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/pi-labs.png
name: Pi Labs
description: Pi Labs (withpi.ai) is an applied-AI company, backed by Accel and later acquired by Microsoft, that builds tooling
  to evaluate and enhance AI applications. Its core product, Pi Scorer, is a deterministic, roughly 200ms foundation model
  that scores any text against a rubric of natural-language questions — reported to be about 500x more efficient than using
  an LLM as a judge. Teams use it to calibrate rubrics, gate and rank LLM outputs, and close the loop across training, inference,
  and feedback. The scoring engine is exposed as an OpenAPI-compatible REST API at api.withpi.ai with official Python and
  TypeScript SDKs (both published as "withpi"), a copilot for building scoring specs, and a Promptfoo integration (Pi Scorer).
url: https://raw.githubusercontent.com/api-evangelist/pi-labs/refs/heads/main/apis.yml
x-type: company
x-source: vc-portfolio
x-backed-by:
- accel
x-tier: stub
x-tier-reason: portfolio-lead
specificationVersion: '0.20'
created: '2026-07-17'
modified: '2026-07-20'
tags:
- Company
- AI
- Evaluation
- LLM
- Scoring
- Machine Learning
- Developer Tools
apis:
- name: Pi Scoring API
  description: OpenAPI-compatible REST API for scoring, calibrating, and optimizing LLM outputs against rubrics of natural-language
    questions. Authenticated with an API key (WITHPI_API_KEY) sent as an Authorization bearer header.
  humanURL: https://code.withpi.ai
  baseURL: https://api.withpi.ai
  tags:
  - Scoring
  - Evaluation
  - LLM
  properties:
  - type: Documentation
    url: https://code.withpi.ai
  - type: Authentication
    url: authentication/pi-labs-authentication.yml
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com
- FN: APIs.json
  email: info@apis.io
common:
- type: DeveloperPortal
  url: https://withpi.ai
- type: Documentation
  url: https://code.withpi.ai
- type: APIReference
  url: https://code.withpi.ai
- type: GettingStarted
  url: https://code.withpi.ai/quickstart
- type: GitHubOrganization
  url: https://github.com/withpi
- type: ChangeLog
  url: https://withpi.ai/release-notes
- type: Packages
  url: packages/pi-labs-packages.yml
- type: SDKs
  url: packages/pi-labs-packages.yml
- type: Authentication
  url: authentication/pi-labs-authentication.yml
- type: Conventions
  url: conventions/pi-labs-conventions.yml
- type: AgentSkill
  url: skills/_index.yml
- type: LLMsTxt
  url: llms/pi-labs-llms.txt
x-enrichment:
  date: '2026-07-20'
  status: enriched
  artifacts_added: 7
  pass: local-v1