Okareo

Okareo Checks API

Create, generate, validate, and manage Checks.

GitHub OpenAPI

Operations 6

POST /v0/check_validate Check Validate #
POST /v0/check_generate Check Generate #
POST /v0/check_create_or_update Check Create Or Update #
DELETE /v0/check/{check_id} Check Delete #
GET /v0/check/{check_id} Get Check #
GET /v0/checks Get All Checks #

Specifications

⚙
OpenAPI
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/openapi/okareo-checks-api-openapi.yml

Schemas & Data

📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/json-schema/okareo-datapoint-list-item-schema.json
📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/json-schema/okareo-scenario-set-generate-schema.json
📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/json-schema/okareo-datapoint-filter-item-schema.json
📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/json-schema/okareo-driver-model-schema-schema.json
📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/json-schema/okareo-test-run-item-schema.json
📊
JSONSchema
https://raw.githubusercontent.com/api-evangelist/okareo/refs/heads/main/json-schema/okareo-test-run-payload-v2-schema.json

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/okareo:okareo-checks-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

okareo-checks-api-openapi.yml Raw ↑ 
openapi: 3.2.0
info:
  title: Okareo Checks API
  version: 0.1.0
  description: Create, generate, validate, and manage Checks.
servers:
- url: https://api.okareo.com
tags:
- name: Checks
  description: Create, generate, validate, and manage Checks.
paths:
  /v0/check_validate:
    post:
      tags:
      - Checks
      summary: Check Validate
      description: 'Validate a check without persisting.

        For code-based checks validates code_contents; for model-based checks validates prompt_template.

        Returns valid=True/False and an optional message.'
      operationId: check_validate_v0_check_validate_post
      parameters:
      - name: api-key
        in: header
        required: true
        schema:
          type: string
          title: Api-Key
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CheckValidateRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CheckValidateResponse'
        '400':
          description: Input data is incorrect
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Okareo API token has failed authentication
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Data is not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Input data is invalid
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
  /v0/check_generate:
    post:
      tags:
      - Checks
      summary: Check Generate
      description: 'Generate code for an evaluator for testing.

        Returns:

        the evaluator object with the generated code'
      operationId: check_generate_v0_check_generate_post
      parameters:
      - name: api-key
        in: header
        required: true
        schema:
          type: string
          title: Api-Key
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/EvaluatorSpecRequest'
      responses:
        '201':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluatorGenerateResponse'
        '400':
          description: Input data is incorrect
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Okareo API token has failed authentication
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Data is not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Input data is invalid
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
  /v0/check_create_or_update:
    post:
      tags:
      - Checks
      summary: Check Create Or Update
      description: 'Create or get check


        Returns:

        the check object with its ID'
      operationId: check_create_or_update_v0_check_create_or_update_post
      parameters:
      - name: api-key
        in: header
        required: true
        schema:
          type: string
          title: Api-Key
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CheckCreateUpdateSchema'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluatorDetailedResponse'
        '400':
          description: Input data is incorrect
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Okareo API token has failed authentication
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Data is not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Input data is invalid
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
  /v0/check/{check_id}:
    delete:
      tags:
      - Checks
      summary: Check Delete
      description: 'Deletes a check


        Raises:

        HTTPException: 404 if check_id is not found


        Returns: 204 status code on successful deletion'
      operationId: check_delete_v0_check__check_id__delete
      parameters:
      - name: check_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          description: The ID of the Check to delete
          title: Check Id
        description: The ID of the Check to delete
      - name: api-key
        in: header
        required: true
        schema:
          type: string
          title: Api-Key
      requestBody:
        required: true
        content:
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/Body_check_delete_v0_check__check_id__delete'
      responses:
        '204':
          description: Successful Response
        '400':
          description: Input data is incorrect
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Okareo API token has failed authentication
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Data is not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Input data is invalid
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
    get:
      tags:
      - Checks
      summary: Get Check
      description: 'Get a check


        Raises:

        HTTPException: 404 if check_id is not found


        Returns: the check'
      operationId: get_check_v0_check__check_id__get
      parameters:
      - name: check_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          description: The ID of the Check to get
          title: Check Id
        description: The ID of the Check to get
      - name: api-key
        in: header
        required: true
        schema:
          type: string
          title: Api-Key
      responses:
        '201':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluatorDetailedResponse'
        '400':
          description: Input data is incorrect
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Okareo API token has failed authentication
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Data is not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Input data is invalid
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
  /v0/checks:
    get:
      tags:
      - Checks
      summary: Get All Checks
      description: 'Get a list of checks for this organization


        Returns:

        a list of requested checks'
      operationId: get_all_checks_v0_checks_get
      parameters:
      - name: all-versions
        in: query
        required: false
        schema:
          type: boolean
          default: false
          title: All-Versions
      - name: api-key
        in: header
        required: true
        schema:
          type: string
          title: Api-Key
      responses:
        '201':
          description: Successful Response
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/EvaluatorBriefResponse'
                title: Response Get All Checks V0 Checks Get
        '400':
          description: Input data is incorrect
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '401':
          description: Okareo API token has failed authentication
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '404':
          description: Data is not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
        '422':
          description: Input data is invalid
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
components:
  schemas:
    ErrorResponse:
      properties:
        detail:
          type: string
          title: Detail
      type: object
      required:
      - detail
      title: ErrorResponse
    EvaluatorBriefResponse:
      properties:
        id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Id
        name:
          anyOf:
          - type: string
          - type: 'null'
          title: Name
        description:
          anyOf:
          - type: string
          - type: 'null'
          title: Description
          default: ''
        output_data_type:
          anyOf:
          - type: string
          - type: 'null'
          title: Output Data Type
          default: ''
        time_created:
          anyOf:
          - type: string
            format: date-time
          - type: 'null'
          title: Time Created
        check_config:
          anyOf:
          - additionalProperties: true
            type: object
          - type: 'null'
          title: Check Config
        is_predefined:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Is Predefined
          default: false
        version:
          anyOf:
          - type: integer
          - type: 'null'
          title: Version
        tags:
          anyOf:
          - items:
              type: string
            type: array
          - type: 'null'
          title: Tags
      type: object
      title: EvaluatorBriefResponse
    CheckValidateResponse:
      properties:
        valid:
          type: boolean
          title: Valid
          description: True if the check passed validation.
        message:
          anyOf:
          - type: string
          - type: 'null'
          title: Message
          description: Error or warning message when valid is False.
      type: object
      required:
      - valid
      title: CheckValidateResponse
      description: Response from POST /check_validate.
    CheckValidateRequest:
      properties:
        check_type:
          type: string
          enum:
          - code
          - model
          - audio
          title: Check Type
          description: 'Type of check: code (code-based), model (model-based judge prompt), or audio (LLM audio judge).'
        check_config:
          additionalProperties:
            type: string
          type: object
          title: Check Config
          description: 'Check config: code_contents for code-based, prompt_template for model-based or audio.'
        output_data_type:
          anyOf:
          - type: string
          - type: 'null'
          title: Output Data Type
          description: Output data type (e.g. pass_fail, score, analysis). Used for frontend symmetry.
      type: object
      required:
      - check_type
      - check_config
      title: CheckValidateRequest
      description: Request body for POST /check_validate. Validates check code or prompt without persisting.
    Body_check_delete_v0_check__check_id__delete:
      properties:
        name:
          type: string
          title: Name
          description: Name of the Check to delete
      type: object
      required:
      - name
      title: Body_check_delete_v0_check__check_id__delete
    CheckCreateUpdateSchema:
      properties:
        check_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Check Id
          description: When provided, update existing check by ID
        name:
          anyOf:
          - type: string
          - type: 'null'
          title: Name
        description:
          anyOf:
          - type: string
          - type: 'null'
          title: Description
        check_config:
          anyOf:
          - additionalProperties:
              anyOf:
              - type: string
              - type: boolean
            type: object
          - type: 'null'
          title: Check Config
        project_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Project Id
          description: ID of the project
        tags:
          anyOf:
          - items:
              type: string
            type: array
          - type: 'null'
          title: Tags
      type: object
      title: CheckCreateUpdateSchema
    EvaluatorGenerateResponse:
      properties:
        name:
          anyOf:
          - type: string
          - type: 'null'
          title: Name
        description:
          anyOf:
          - type: string
          - type: 'null'
          title: Description
        requires_scenario_input:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Requires Scenario Input
        requires_scenario_result:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Requires Scenario Result
        output_data_type:
          anyOf:
          - type: string
          - type: 'null'
          title: Output Data Type
        generated_code:
          anyOf:
          - type: string
          - type: 'null'
          title: Generated Code
        generated_prompt:
          anyOf:
          - type: string
          - type: 'null'
          title: Generated Prompt
        warning:
          anyOf:
          - type: string
          - type: 'null'
          title: Warning
      type: object
      title: EvaluatorGenerateResponse
    EvaluatorDetailedResponse:
      properties:
        id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Id
        project_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Project Id
        name:
          anyOf:
          - type: string
          - type: 'null'
          title: Name
        description:
          anyOf:
          - type: string
          - type: 'null'
          title: Description
          default: ''
        requires_scenario_input:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Requires Scenario Input
        requires_scenario_result:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Requires Scenario Result
        output_data_type:
          anyOf:
          - type: string
          - type: 'null'
          title: Output Data Type
          default: ''
        code_contents:
          anyOf:
          - type: string
          - type: 'null'
          title: Code Contents
          default: ''
        time_created:
          anyOf:
          - type: string
            format: date-time
          - type: 'null'
          title: Time Created
        warning:
          anyOf:
          - type: string
          - type: 'null'
          title: Warning
        check_config:
          anyOf:
          - additionalProperties: true
            type: object
          - type: 'null'
          title: Check Config
        is_predefined:
          anyOf:
          - type: boolean
          - type: 'null'
          title: Is Predefined
          default: false
        version:
          anyOf:
          - type: integer
          - type: 'null'
          title: Version
        tags:
          anyOf:
          - items:
              type: string
            type: array
          - type: 'null'
          title: Tags
      type: object
      title: EvaluatorDetailedResponse
    EvaluatorSpecRequest:
      properties:
        name:
          anyOf:
          - type: string
          - type: 'null'
          title: Name
          description: Name of the evaluator
        user_input:
          anyOf:
          - type: string
            maxLength: 50000
          - type: 'null'
          title: User Input
          description: Primary user instruction for generation. When set, used instead of description for the LLM prompt.
        description:
          anyOf:
          - type: string
          - type: 'null'
          title: Description
          description: "Description for the evaluator.\n            When this request is sent to generate an evaluator, this field will be used to generate it (or user_input if set)."
        requires_scenario_input:
          type: boolean
          title: Requires Scenario Input
          description: Whether the evaluator requires scenario input
          default: false
        requires_scenario_result:
          type: boolean
          title: Requires Scenario Result
          description: Whether the evaluator requires scenario expected result
          default: false
        output_data_type:
          anyOf:
          - type: string
          - type: 'null'
          title: Output Data Type
          description: Evaluator output data type (i.e., bool, int, float)
        project_id:
          anyOf:
          - type: string
            format: uuid
          - type: 'null'
          title: Project Id
          description: ID for the project
        check_type:
          anyOf:
          - type: string
          - type: 'null'
          title: Check Type
          description: model or code based check
          default: code
        prior_code:
          anyOf:
          - type: string
            maxLength: 50000
          - type: 'null'
          title: Prior Code
          description: Existing code to refine (code branch). When set, LLM is asked to modify/improve it.
        prior_prompt:
          anyOf:
          - type: string
            maxLength: 50000
          - type: 'null'
          title: Prior Prompt
          description: Existing prompt to refine (model branch). When set, LLM is asked to revise it.
      type: object
      title: EvaluatorSpecRequest