Triton Inference Server

Triton Inference Server Statistics API

Server and model inference statistics

GitHub OpenAPI

Operations 3

GET /v2/models/{model_name}/stats Triton Inference Server Get Model Inference Statistics #
GET /v2/models/{model_name}/versions/{model_version}/stats Triton Inference Server Get Model Version Inference Statistics #
GET /v2/models/stats Triton Inference Server Get Statistics for All Models #

Documentation

📖
Documentation
https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md
📖
Documentation
https://github.com/triton-inference-server/server/blob/main/docs/user_guide/metrics.md

Specifications

⚙
OpenAPI
https://artifacts.apis.io/triton/openapi/triton-statistics-api-openapi.yml

Other Resources

🔗
PostmanCollection
https://www.postman.com/nvidia-triton
🔗
Metrics Format
https://prometheus.io/docs/instrumenting/exposition_formats/

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/triton:triton-statistics-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

triton-statistics-api-openapi.yml Raw ↑ 
openapi: 3.2.0
info:
  title: Triton Inference Server NVIDIA Triton Inference Server…
  description: RESTful API for the NVIDIA Triton Inference Server, implementing the KServe V2 inference protocol with Triton-specific extensions. Provides endpoints for model inference, health checks, server and model metadata, model repository management, statistics, tracing, logging, and system shared memory management.
  version: '2.0'
  contact:
    name: NVIDIA Triton Team
    url: https://github.com/triton-inference-server/server
    email: triton@nvidia.com
  license:
    name: BSD 3-Clause
    url: https://github.com/triton-inference-server/server/blob/main/LICENSE
servers:
- url: http://localhost:8000
  description: Triton HTTP endpoint (default)
- url: http://{host}:{port}
  description: Custom Triton HTTP endpoint
  variables:
    host:
      default: localhost
      description: Triton server hostname or IP
    port:
      default: '8000'
      description: Triton HTTP port
tags:
- name: Statistics
  description: Server and model inference statistics
paths:
  /v2/models/{model_name}/stats:
    get:
      operationId: modelStatistics
      summary: Triton Inference Server Get Model Inference Statistics
      description: Retrieve inference statistics for a specific model including request count, execution count, and cumulative timing information. This is a Triton extension to the KServe protocol.
      tags:
      - Statistics
      parameters:
      - $ref: '#/components/parameters/modelName'
      responses:
        '200':
          description: Model statistics returned successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/StatisticsResponse'
        '400':
          description: Error retrieving model statistics
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
  /v2/models/{model_name}/versions/{model_version}/stats:
    get:
      operationId: modelVersionStatistics
      summary: Triton Inference Server Get Model Version Inference Statistics
      description: Retrieve inference statistics for a specific version of a model.
      tags:
      - Statistics
      parameters:
      - $ref: '#/components/parameters/modelName'
      - $ref: '#/components/parameters/modelVersion'
      responses:
        '200':
          description: Model version statistics returned successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/StatisticsResponse'
        '400':
          description: Error retrieving model version statistics
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
  /v2/models/stats:
    get:
      operationId: allModelStatistics
      summary: Triton Inference Server Get Statistics for All Models
      description: Retrieve inference statistics for all loaded models.
      tags:
      - Statistics
      responses:
        '200':
          description: Statistics for all models returned successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/StatisticsResponse'
        '400':
          description: Error retrieving statistics
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
components:
  schemas:
    ModelStatistics:
      type: object
      properties:
        name:
          type: string
          description: Model name
        version:
          type: string
          description: Model version
        last_inference:
          type: integer
          description: Timestamp of the last inference (milliseconds since epoch)
        inference_count:
          type: integer
          description: Total number of inferences performed
        execution_count:
          type: integer
          description: Total number of executions (batch-level)
        inference_stats:
          $ref: '#/components/schemas/InferenceStatistics'
        batch_stats:
          type: array
          description: Statistics broken down by batch size
          items:
            type: object
            properties:
              batch_size:
                type: integer
              compute_input:
                $ref: '#/components/schemas/StatisticsDuration'
              compute_infer:
                $ref: '#/components/schemas/StatisticsDuration'
              compute_output:
                $ref: '#/components/schemas/StatisticsDuration'
        memory_usage:
          type: array
          description: Memory usage details
          items:
            type: object
            properties:
              type:
                type: string
              id:
                type: integer
              byte_size:
                type: integer
    InferenceStatistics:
      type: object
      properties:
        success:
          $ref: '#/components/schemas/StatisticsDuration'
        fail:
          $ref: '#/components/schemas/StatisticsDuration'
        queue:
          $ref: '#/components/schemas/StatisticsDuration'
        compute_input:
          $ref: '#/components/schemas/StatisticsDuration'
        compute_infer:
          $ref: '#/components/schemas/StatisticsDuration'
        compute_output:
          $ref: '#/components/schemas/StatisticsDuration'
        cache_hit:
          $ref: '#/components/schemas/StatisticsDuration'
        cache_miss:
          $ref: '#/components/schemas/StatisticsDuration'
    StatisticsDuration:
      type: object
      properties:
        count:
          type: integer
          description: Number of occurrences
        ns:
          type: integer
          description: Cumulative duration in nanoseconds
    ErrorResponse:
      type: object
      properties:
        error:
          type: string
          description: Error message describing what went wrong
    StatisticsResponse:
      type: object
      properties:
        model_stats:
          type: array
          description: Statistics for each model
          items:
            $ref: '#/components/schemas/ModelStatistics'
  parameters:
    modelVersion:
      name: model_version
      in: path
      required: true
      description: Version of the model
      schema:
        type: string
    modelName:
      name: model_name
      in: path
      required: true
      description: Name of the model
      schema:
        type: string
externalDocs:
  description: Triton Inference Server Protocol Documentation
  url: https://github.com/triton-inference-server/server/blob/main/docs/protocol/README.md