Bernstein Agent Comparison API

openapi: 3.2.0
info:
  title: Bernstein Task Server Agent Comparison API
  description: 'Bernstein REST API - the open-source governance layer for AI agents, one git worktree per task.


    ## Authentication


    Authentication is ENABLED by default. Include a Bearer token in all requests:


    ```

    Authorization: Bearer

    ```


    To run without auth (development only), set `BERNSTEIN_AUTH_DISABLED=1` - this logs a loud warning and passes every request through.


    Public endpoints (no auth required): `/health`, `/health/ready`, `/health/live`, `/ready`, `/alive`, `/.well-known/agent.json`, `/docs`, `/openapi.json`, and the auth-flow endpoints (`/auth/login`, `/auth/oidc/callback`, etc.).


    Webhook and hook endpoints (`/webhook`, `/webhooks/*`, `/hooks/{session_id}`) authenticate via HMAC-SHA256 signatures - they do NOT accept Bearer tokens.


    ## Base URL


    Default: `http://127.0.0.1:8052`. Override with env vars `BERNSTEIN_HOST` and `BERNSTEIN_PORT`.


    ## Error Format


    All errors return JSON with a `detail` field:


    ```json

    {"detail": "Task not found: task-xyz"}

    ```


    | Status | Meaning |

    |--------|---------|

    | 400 | Bad request (validation error) |

    | 401 | Unauthorized (missing/invalid token) |

    | 403 | Forbidden (IP not in allowlist) |

    | 404 | Resource not found |

    | 409 | Conflict (task already in terminal state) |

    | 429 | Rate limited - respect the `Retry-After` header |

    | 500 | Internal server error |'
  version: 3.22.0
  license:
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0
  contact:
    name: Alex Chernysh
    url: https://github.com/sipyourdrink-ltd/bernstein
servers:
- url: http://{host}:{port}
  description: Task server started by the Bernstein CLI on the operator's own machine. There is no hosted instance; substitute the host and port of your install.
  variables:
    host:
      default: 127.0.0.1
      description: Interface the task server binds to. Loopback by default.
    port:
      default: '8052'
      description: Task server port. Override with BERNSTEIN_PORT.
tags:
- name: agent-comparison
paths:
  /agents/comparison:
    get:
      tags:
      - agent-comparison
      summary: Get Agent Comparison
      description: 'Return per-(adapter, model) performance comparison metrics.


        Aggregates data from all agent sessions in the current run:

        success rate, average completion time, cost per task, and

        quality gate pass rate.


        Returns:

        JSON list of :class:`AgentMetrics` objects sorted by adapter

        then model.'
      operationId: get_agent_comparison_agents_comparison_get
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                items:
                  $ref: '#/components/schemas/AgentMetrics'
                type: array
                title: Response Get Agent Comparison Agents Comparison Get
  /api/v1/agents/comparison:
    get:
      tags:
      - agent-comparison
      summary: Get Agent Comparison
      description: 'Return per-(adapter, model) performance comparison metrics.


        Aggregates data from all agent sessions in the current run:

        success rate, average completion time, cost per task, and

        quality gate pass rate.


        Returns:

        JSON list of :class:`AgentMetrics` objects sorted by adapter

        then model.'
      operationId: get_agent_comparison_api_v1_agents_comparison_get
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                items:
                  $ref: '#/components/schemas/AgentMetrics'
                type: array
                title: Response Get Agent Comparison Api V1 Agents Comparison Get
components:
  schemas:
    AgentMetrics:
      properties:
        adapter:
          type: string
          title: Adapter
        model:
          type: string
          title: Model
        total_tasks:
          type: integer
          title: Total Tasks
          default: 0
        succeeded:
          type: integer
          title: Succeeded
          default: 0
        failed:
          type: integer
          title: Failed
          default: 0
        avg_completion_secs:
          type: number
          title: Avg Completion Secs
          default: 0
        total_cost_usd:
          type: number
          title: Total Cost Usd
          default: 0
        quality_gate_pass_rate:
          type: number
          title: Quality Gate Pass Rate
          default: 1
        success_rate:
          type: number
          title: Success Rate
          description: Fraction of tasks that succeeded (0.0-1.0).
          readOnly: true
        cost_per_task:
          type: number
          title: Cost Per Task
          description: Average cost per task in USD.
          readOnly: true
      type: object
      required:
      - adapter
      - model
      - success_rate
      - cost_per_task
      title: AgentMetrics
      description: Aggregated performance metrics for a single (adapter, model) pair.