Amigo

Amigo Evaluations API

The Evaluations API from Amigo — 4 operation(s) for evaluations.

GitHub OpenAPI
Business capability
Agent Quality & Operations AE-100.60

Operations 7

POST /v1/{workspace_id}/production-eval-definitions Create Production Eval Definition #
GET /v1/{workspace_id}/production-eval-definitions List Production Eval Definitions #
GET /v1/{workspace_id}/production-eval-definitions/{definition_id} Get Production Eval Definition #
PATCH /v1/{workspace_id}/production-eval-definitions/{definition_id} Update Production Eval Definition #
DELETE /v1/{workspace_id}/production-eval-definitions/{definition_id} Delete Production Eval Definition #
POST /v1/{workspace_id}/calls/{conversation_id}/evaluate Evaluate Call #
GET /v1/{workspace_id}/calls/{conversation_id}/eval-results Get Call Eval Results #

Documentation

📖
Documentation
https://docs.amigo.ai/developer-guide/classic-api
📖
Documentation
https://docs.amigo.ai/developer-guide/platform-api

Specifications

⚙
OpenAPI
https://artifacts.apis.io/amigo/openapi/amigo-evaluations-api-openapi.yml

Schemas & Data

📊
JSONSchema
https://artifacts.apis.io/amigo/json-schema/amigo-production-eval-definition-schema.json
📊
JSONSchema
https://artifacts.apis.io/amigo/json-schema/amigo-update-production-eval-definition-request-schema.json
📊
JSONSchema
https://artifacts.apis.io/amigo/json-schema/amigo-create-production-eval-definition-request-schema.json
📊
JSONSchema
https://artifacts.apis.io/amigo/json-schema/amigo-paginated-response-production-eval-definition-schema.json
📊
JSONSchema
https://artifacts.apis.io/amigo/json-schema/amigo-call-eval-results-response-schema.json

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/amigo:amigo-evaluations-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

amigo-evaluations-api-openapi.yml Raw ↑ 
openapi: 3.2.0
info:
  title: Platform Evaluations API
  description: Management API for the Amigo platform. Provides CRUD operations for workspaces, API keys, and resources.
  version: 1.0.0
servers:
- url: https://api.platform.amigo.ai
  description: Production
security:
- BearerAuth: []
tags:
- name: Evaluations
paths:
  /v1/{workspace_id}/production-eval-definitions:
    post:
      tags:
      - Evaluations
      summary: Create Production Eval Definition
      operationId: create-production-eval-definition
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateProductionEvalDefinitionRequest'
      responses:
        '201':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ProductionEvalDefinition'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
    get:
      tags:
      - Evaluations
      summary: List Production Eval Definitions
      operationId: list-production-eval-definitions
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
      - name: service_id
        in: query
        required: false
        schema:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Service Id
      - name: active
        in: query
        required: false
        schema:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Active
      - name: limit
        in: query
        required: false
        schema:
          type: integer
          maximum: 100
          exclusiveMinimum: 0
          default: 10
          title: Limit
      - name: continuation_token
        in: query
        required: false
        schema:
          type: integer
          default: 0
          title: Continuation Token
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/PaginatedResponse_ProductionEvalDefinition_'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
  /v1/{workspace_id}/production-eval-definitions/{definition_id}:
    get:
      tags:
      - Evaluations
      summary: Get Production Eval Definition
      operationId: get-production-eval-definition
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
      - name: definition_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Definition Id
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ProductionEvalDefinition'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
    patch:
      tags:
      - Evaluations
      summary: Update Production Eval Definition
      operationId: update-production-eval-definition
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
      - name: definition_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Definition Id
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/UpdateProductionEvalDefinitionRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ProductionEvalDefinition'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
    delete:
      tags:
      - Evaluations
      summary: Delete Production Eval Definition
      operationId: delete-production-eval-definition
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
      - name: definition_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Definition Id
      responses:
        '204':
          description: Successful Response
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
  /v1/{workspace_id}/calls/{conversation_id}/evaluate:
    post:
      tags:
      - Evaluations
      summary: Evaluate Call
      description: 'Run the workspace''s active eval definitions against one completed call and

        persist the verdicts. Synchronous (the caller waits on the judge); the eager

        post-call trigger is a separate change.'
      operationId: evaluate-call
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
      - name: conversation_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Conversation Id
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CallEvalResultsResponse'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
  /v1/{workspace_id}/calls/{conversation_id}/eval-results:
    get:
      tags:
      - Evaluations
      summary: Get Call Eval Results
      operationId: get-call-eval-results
      parameters:
      - name: workspace_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Workspace Id
      - name: conversation_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          title: Conversation Id
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CallEvalResultsResponse'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
components:
  schemas:
    CallEvalResultsResponse:
      properties:
        conversation_id:
          type: string
          format: uuid
          title: Conversation Id
        results:
          items:
            $ref: '#/components/schemas/CallEvalResult'
          type: array
          title: Results
      type: object
      required:
      - conversation_id
      - results
      title: CallEvalResultsResponse
    CallEvalResult:
      properties:
        id:
          type: string
          format: uuid
          title: Id
        workspace_id:
          type: string
          format: uuid
          title: Workspace Id
        service_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Service Id
        conversation_id:
          type: string
          format: uuid
          title: Conversation Id
        call_sid:
          anyOf:
          - type: string
          - type: 'null'
          title: Call Sid
        eval_key:
          type: string
          maxLength: 128
          title: Eval Key
        eval_type:
          type: string
          enum:
          - assertion
          - metric
          title: Eval Type
        assertion_kind:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Assertion Kind
        metric_key:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Metric Key
        status:
          type: string
          enum:
          - passed
          - failed
          - pending
          - skipped
          - error
          title: Status
        passed:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Passed
        score:
          anyOf:
          - type: number
          - type: 'null'
          title: Score
        expected:
          title: Expected
        actual:
          title: Actual
        rationale:
          anyOf:
          - type: string
          - type: 'null'
          title: Rationale
        justification:
          anyOf:
          - type: string
          - type: 'null'
          title: Justification
        cited_turns:
          items: {}
          type: array
          title: Cited Turns
        computed_at:
          anyOf:
          - type: string
            format: date-time
          - type: 'null'
          title: Computed At
        created_at:
          anyOf:
          - type: string
            format: date-time
          - type: 'null'
          title: Created At
      type: object
      required:
      - workspace_id
      - conversation_id
      - eval_key
      - eval_type
      - status
      title: CallEvalResult
      description: 'One eval verdict for one production call, keyed by ``conversation_id``.

        Mirrors the sim eval-result shape. ``justification`` and ``cited_turns`` come

        from the justified ai_query metric compute. PHI: ``justification`` / ``actual``

        can echo live transcript content; never log them.'
    ProductionEvalDefinition:
      properties:
        id:
          type: string
          format: uuid
          title: Id
        workspace_id:
          type: string
          format: uuid
          title: Workspace Id
        service_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Service Id
        eval_key:
          type: string
          maxLength: 128
          title: Eval Key
        eval_type:
          type: string
          enum:
          - assertion
          - metric
          title: Eval Type
        assertion_kind:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Assertion Kind
        metric_key:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Metric Key
        expected:
          title: Expected
        params:
          additionalProperties: true
          type: object
          title: Params
        active:
          type: boolean
          title: Active
          default: true
        created_at:
          anyOf:
          - type: string
            format: date-time
          - type: 'null'
          title: Created At
        updated_at:
          anyOf:
          - type: string
            format: date-time
          - type: 'null'
          title: Updated At
      type: object
      required:
      - workspace_id
      - eval_key
      - eval_type
      title: ProductionEvalDefinition
      description: 'An eval (assertion or metric-eval) that runs against a workspace''s live

        calls — the production analogue of a sim case''s ``evals``. ``service_id``

        None applies to every service in the workspace. ``expected`` is JSON: a

        phrase string for assertions, or a ``{gte|lte|equals|...}`` threshold for

        metric-evals.'
    UpdateProductionEvalDefinitionRequest:
      properties:
        service_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Service Id
        eval_key:
          anyOf:
          - type: string
            maxLength: 128
            minLength: 1
          - type: 'null'
          title: Eval Key
        eval_type:
          anyOf:
          - type: string
            enum:
            - assertion
            - metric
          - type: 'null'
          title: Eval Type
        assertion_kind:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Assertion Kind
        metric_key:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Metric Key
        expected:
          title: Expected
        params:
          anyOf:
          - additionalProperties: true
            type: object
          - type: 'null'
          title: Params
        active:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Active
      type: object
      title: UpdateProductionEvalDefinitionRequest
    CreateProductionEvalDefinitionRequest:
      properties:
        eval_key:
          type: string
          maxLength: 128
          minLength: 1
          title: Eval Key
        eval_type:
          type: string
          enum:
          - assertion
          - metric
          title: Eval Type
        service_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Service Id
        assertion_kind:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Assertion Kind
        metric_key:
          anyOf:
          - type: string
            maxLength: 128
          - type: 'null'
          title: Metric Key
        expected:
          title: Expected
        params:
          additionalProperties: true
          type: object
          title: Params
        active:
          type: boolean
          title: Active
          default: true
      type: object
      required:
      - eval_key
      - eval_type
      title: CreateProductionEvalDefinitionRequest
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
            - type: string
            - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
        input:
          title: Input
        ctx:
          type: object
          title: Context
      type: object
      required:
      - loc
      - msg
      - type
      title: ValidationError
    PaginatedResponse_ProductionEvalDefinition_:
      properties:
        items:
          items:
            $ref: '#/components/schemas/ProductionEvalDefinition'
          type: array
          title: Items
        has_more:
          type: boolean
          title: Has More
        continuation_token:
          anyOf:
          - type: integer
          - type: 'null'
          title: Continuation Token
        total:
          anyOf:
          - type: integer
          - type: 'null'
          title: Total
      type: object
      required:
      - items
      - has_more
      title: PaginatedResponse[ProductionEvalDefinition]
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
      description: API key issued via `POST /v1/{workspace_id}/api-keys`. Pass the returned `api_key` value as a Bearer token.