Contextual AI

Contextual AI LMUnit API

Evaluate model responses with natural-language unit tests.

Documentation GitHub OpenAPI

Operations 1

POST /lmunit LMUnit #

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this

find_apisBrowse and filter every API in the catalog.
get_api_artifactsOne API's artifacts, grouped by type.
get_openapiThe primary OpenAPI for this API.
find_similar_apisAPIs that look like this one.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This API

curl "https://apis.io/api/v1/apis/contextual-ai-lmunit-api"

All apis

curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

openapi: 3.2.0
info:
  title: Contextual AI Platform Agents LM Unit API
  description: REST API for the Contextual AI enterprise RAG platform. Provides agents (create / query grounded RAG agents), datastores and documents (ingest and manage the knowledge corpus), and standalone component APIs - Generate (grounded generation with the GLM), Rerank (instruction-following reranker), Parse (document parsing into AI-ready markdown), and LMUnit (natural-language unit-test evaluation). All endpoints authenticate with a Bearer API key.
  termsOfService: https://contextual.ai/terms-of-service/
  contact:
    name: Contextual AI Support
    url: https://docs.contextual.ai
    email: support@contextual.ai
  version: '1.0'
servers:
- url: https://api.contextual.ai/v1
  description: Contextual AI production API
security:
- bearerAuth: []
tags:
- name: LMUnit
  description: Evaluate model responses with natural-language unit tests.
paths:
  /lmunit:
    post:
      operationId: lmunit
      tags:
      - LMUnit
      summary: LMUnit
      description: Evaluate a model response against a natural-language unit test. Total input is limited to 7000 tokens. Returns a score from 1 (strongly fails) to 5 (strongly passes).
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/LMUnitRequest'
      responses:
        '200':
          description: Evaluation score.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/LMUnitResponse'
        '422':
          $ref: '#/components/responses/ValidationError'
components:
  schemas:
    HTTPValidationError:
      type: object
      properties:
        detail:
          type: array
          items:
            $ref: '#/components/schemas/ValidationErrorDetail'
    LMUnitRequest:
      type: object
      required:
      - query
      - response
      - unit_test
      properties:
        query:
          type: string
          description: The prompt to which the model responds.
        response:
          type: string
          description: The model response to evaluate.
        unit_test:
          type: string
          description: Natural-language statement to evaluate the response against.
    LMUnitResponse:
      type: object
      properties:
        score:
          type: number
          minimum: 1
          maximum: 5
          description: Continuous score from 1 (strongly fails) to 5 (strongly passes).
    ValidationErrorDetail:
      type: object
      properties:
        loc:
          type: array
          items:
            oneOf:
            - type: string
            - type: integer
        msg:
          type: string
        type:
          type: string
  responses:
    ValidationError:
      description: Validation error.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/HTTPValidationError'
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: Contextual AI API key supplied as a Bearer token.