UpTrain website screenshot

UpTrain

UpTrain is an open-source (Apache-2.0) unified platform to evaluate and improve generative AI and LLM applications. It ships a Python framework plus a managed evaluation API that grades responses against 20+ preconfigured checks - context relevance, factual accuracy, response completeness, hallucination, tonality, prompt injection and more - and performs root cause analysis on failure cases.

UpTrain publishes 6 APIs on the APIs.io network, including Auth API, Checksets API, Datasets API, and 3 more. Tagged areas include Artificial Intelligence, LLM, Evaluation, LLM Evaluation, and Observability.

UpTrain’s developer surface includes authentication, documentation, and 8 more developer resources.

🌐 Visit website 📡 Source on GitHub
34.2/100 thin ▬ flat Agent 20/100 agent aware saas Full breakdown ↓
scored 2026-09-14 · rubric v0.22.0
AccessFreemiumSelf serve⚡ Free to try
1 APIs
Artificial IntelligenceLLMEvaluationLLM EvaluationObservabilityOpen-Source

Kin Score

Kin Score Kin Score How this is scored →
scored 2026-09-14 · rubric v0.22.0
Composite quality — 34.2/100 · thin
Contract Quality 12.1 / 23
Developer Ergonomics 5.1 / 18
Access Clarity 5.2 / 18
Operational Transparency 2.8 / 12
Contract Governance 0.0 / 11
Discoverability 6.2 / 9
Create-or-Update Ergonomics 0.0 / 10
Agent readiness — 20/100 · agent aware
Machine-Readable Contract 18 / 18
Agentic Access Contract 10 / 10
Documented Reversibility 0 / 6
MCP Server 0 / 12
Machine-Readable Auth 10 / 10
Idempotency 0 / 9
Stable Error Semantics 0 / 8
Request/Response Examples 0 / 7
Rate-Limit Signaling 7 / 7
Typed Event Surface 0 / 6
Agent Skills 0 / 5
Well-Known Catalog 0 / 4
Consent & Bot Identity 0 / 3
A2A Agent Card 0 / 8
Dry-Run / Simulate Mode 0 / 4
Delegated User Identity 0 / 6
Protected Resource Metadata 0 / 5
Registration Without a Human 0 / 6
Agentic Commerce Well-Known Document 0 / 5
Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it carries 10 points of the composite. It is scored from the published contracts themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in front of it, and the first time that check is skipped a duplicate record is created. Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet, not penalised by it.
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from github.com/api-evangelist/uptrain: open an issue to ask a question, or submit a pull request to add artifacts. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

APIs 6

Individual APIs this provider publishes, each with its own machine-readable definition.

UpTrain Auth API

The Auth API from UpTrain — 1 operation(s) for auth.

UpTrain Checksets API

The Checksets API from UpTrain — 1 operation(s) for checksets.

UpTrain Datasets API

The Datasets API from UpTrain — 1 operation(s) for datasets.

UpTrain Evaluation API

The Evaluation API from UpTrain — 3 operation(s) for evaluation.

UpTrain Root Cause Analysis API

The Root Cause Analysis API from UpTrain — 1 operation(s) for root cause analysis.

UpTrain Runs API

The Runs API from UpTrain — 3 operation(s) for runs.

Open Collections 8

Open, tool-agnostic API collections (OpenAPI-derived and Bruno).

API Collection

OPEN COLLECTION

UpTrain Managed Evaluation Auth API

OPEN COLLECTION

UpTrain Managed Evaluation Auth Checksets API

OPEN COLLECTION

UpTrain Managed Evaluation Auth Datasets API

OPEN COLLECTION

UpTrain Managed Auth Evaluation API

OPEN COLLECTION

UpTrain Managed Evaluation Auth Root Cause Analysis API

OPEN COLLECTION

UpTrain Managed Evaluation Auth Runs API

OPEN COLLECTION

UpTrain Managed Evaluation API

OPEN COLLECTION

Scroll for all 8

Pricing Plans 1

Published pricing tiers and plan structures.

Uptrain Plans Pricing

2 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Uptrain Rate Limits

2 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Uptrain Finops

FINOPS

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Uptrain Authentication

apiKey · 1 scheme

SECURITY

Uptrain Domain Security

TLSv1.3 · HSTS

SECURITY

Agentic Access 1

Recommended x-agentic-access execution contracts for AI agents.

Uptrain Agentic Access

11 operations · 6 acting

11 operations · 6 acting

AGENTIC

Resources

Documentation 1

Reference material describing how the API behaves

Documentation
Documentation

Agent Surfaces 1

MCP servers, agent skills, and machine-readable catalogs

AgenticAccess
AgenticAccess

Build 1

SDKs, sample code, and the tooling you integrate with

GitHubOrganization
GitHubOrganization

Access & Security 2

Authentication, authorization, and security posture

DomainSecurity
DomainSecurity
 Authentication
Authentication

Operate 1

Status, limits, changes, and where to get help

RateLimits
RateLimits

Commercial 2

Pricing, plans, and the legal terms of use

Plans
Plans
 FinOps
FinOps

Company 2

The organization behind the API

LinkedIn
LinkedIn
 Website
Website

Source (apis.yml)

apis.yml Raw ↑ 
aid: uptrain
url: https://raw.githubusercontent.com/api-evangelist/uptrain/refs/heads/main/apis.yml
name: UpTrain
kind: company
description: UpTrain is an open-source (Apache-2.0) unified platform to evaluate and improve generative AI and LLM applications.
  It ships a Python framework plus a managed evaluation API that grades responses against 20+ preconfigured checks - context
  relevance, factual accuracy, response completeness, hallucination, tonality, prompt injection and more - and performs root
  cause analysis on failure cases.
deliveryModel:
  model: saas
  open_source: false
  commercial: true
  callable_host: true
  label: Hosted service · you call their endpoint
  confidence: high
  source:
  - openapi
  - pricing
  generated: '2026-08-28'
  method: derived
accessModel:
  pricing: freemium
  onboarding: self-serve
  trial: false
  try_now: true
  public: false
  label: Freemium · Self-serve signup
  confidence: medium
  source:
  - plans
  - authentication
  - security
  generated: '2026-09-03'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/uptrain.png
tags:
- Artificial Intelligence
- LLM
- Evaluation
- LLM Evaluation
- Observability
- Open-Source
tags_raw:
- AI
- LLM
- Evaluation
- LLM Evaluation
- Observability
- Open Source
created: '2026-06-20'
modified: '2026-06-20'
specificationVersion: '0.23'
apis:
- aid: uptrain:uptrain-auth-api
  name: UpTrain Auth API
  description: The Auth API from UpTrain — 1 operation(s) for auth.
  humanURL: https://docs.uptrain.ai/
  baseURL: https://demo.uptrain.ai/api/public
  tags:
  - Authentication
  tags_raw:
  - Auth
  properties:
  - type: OpenAPI
    url: openapi/uptrain-auth-api-openapi.yml
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/introduction
  - type: APIReference
    url: https://docs.uptrain.ai/predefined-evaluations/overview
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/quickstart
  - type: Documentation
    url: https://docs.uptrain.ai/predefined-evaluations/overview
- aid: uptrain:uptrain-checksets-api
  name: UpTrain Checksets API
  description: The Checksets API from UpTrain — 1 operation(s) for checksets.
  humanURL: https://docs.uptrain.ai/
  baseURL: https://demo.uptrain.ai/api/public
  tags:
  - Checksets
  properties:
  - type: OpenAPI
    url: openapi/uptrain-checksets-api-openapi.yml
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/introduction
  - type: APIReference
    url: https://docs.uptrain.ai/predefined-evaluations/overview
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/quickstart
  - type: Documentation
    url: https://docs.uptrain.ai/predefined-evaluations/overview
- aid: uptrain:uptrain-datasets-api
  name: UpTrain Datasets API
  description: The Datasets API from UpTrain — 1 operation(s) for datasets.
  humanURL: https://docs.uptrain.ai/
  baseURL: https://demo.uptrain.ai/api/public
  tags:
  - Datasets
  properties:
  - type: OpenAPI
    url: openapi/uptrain-datasets-api-openapi.yml
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/introduction
  - type: APIReference
    url: https://docs.uptrain.ai/predefined-evaluations/overview
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/quickstart
  - type: Documentation
    url: https://docs.uptrain.ai/predefined-evaluations/overview
- aid: uptrain:uptrain-evaluation-api
  name: UpTrain Evaluation API
  description: The Evaluation API from UpTrain — 3 operation(s) for evaluation.
  humanURL: https://docs.uptrain.ai/
  baseURL: https://demo.uptrain.ai/api/public
  tags:
  - Evaluation
  properties:
  - type: OpenAPI
    url: openapi/uptrain-evaluation-api-openapi.yml
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/introduction
  - type: APIReference
    url: https://docs.uptrain.ai/predefined-evaluations/overview
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/quickstart
  - type: Documentation
    url: https://docs.uptrain.ai/predefined-evaluations/overview
- aid: uptrain:uptrain-root-cause-analysis-api
  name: UpTrain Root Cause Analysis API
  description: The Root Cause Analysis API from UpTrain — 1 operation(s) for root cause analysis.
  humanURL: https://docs.uptrain.ai/
  baseURL: https://demo.uptrain.ai/api/public
  tags:
  - Root Cause Analysis
  properties:
  - type: OpenAPI
    url: openapi/uptrain-root-cause-analysis-api-openapi.yml
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/introduction
  - type: APIReference
    url: https://docs.uptrain.ai/predefined-evaluations/overview
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/quickstart
  - type: Documentation
    url: https://docs.uptrain.ai/predefined-evaluations/overview
- aid: uptrain:uptrain-runs-api
  name: UpTrain Runs API
  description: The Runs API from UpTrain — 3 operation(s) for runs.
  humanURL: https://docs.uptrain.ai/
  baseURL: https://demo.uptrain.ai/api/public
  tags:
  - Runs
  properties:
  - type: OpenAPI
    url: openapi/uptrain-runs-api-openapi.yml
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/introduction
  - type: APIReference
    url: https://docs.uptrain.ai/predefined-evaluations/overview
  - type: Documentation
    url: https://docs.uptrain.ai/getting-started/quickstart
  - type: Documentation
    url: https://docs.uptrain.ai/predefined-evaluations/overview
common:
- type: AgenticAccess
  url: agentic-access/uptrain-agentic-access.yml
- type: DomainSecurity
  url: security/uptrain-domain-security.yml
- type: Authentication
  url: authentication/uptrain-authentication.yml
- type: GitHubOrganization
  url: https://github.com/uptrain-ai
- type: LinkedIn
  url: https://www.linkedin.com/company/uptrain-ai
- type: Website
  url: https://uptrain.ai/
- type: Documentation
  url: https://docs.uptrain.ai/
- type: Plans
  url: plans/uptrain-plans-pricing.yml
- type: RateLimits
  url: rate-limits/uptrain-rate-limits.yml
- type: FinOps
  url: finops/uptrain-finops.yml
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this
  • find_providersBrowse and filter every provider in the catalog.
  • get_provider_artifactsEvery artifact this provider publishes, grouped by type.
  • get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
  • get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
  • get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
  • get_provider_ratingPRO — composite, band, trend and facet scores.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This provider 
curl "https://apis.io/api/v1/providers/uptrain"
All providers 
curl "https://apis.io/api/v1/providers?limit=25"
Every operation they expose 
curl "https://apis.io/api/v1/providers/uptrain/operations?limit=25"
How their score was established 
curl "https://apis.io/api/v1/providers/uptrain/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.