Cohere Detokenize API

The Cohere Detokenize API converts a sequence of token IDs back into their corresponding text string using the tokenizer for a specified model. It is the inverse operation of the Tokenize API and is useful for inspecting model outputs at the token level, debugging tokenization behavior, and reconstructing text from token representations.

OpenAPI Specification

cohere-detokenize-api-openapi.yml Raw ↑
openapi: 3.1.0
info:
  title: Cohere Chat Detokenize API
  description: The Cohere Chat API enables developers to integrate large language model text generation capabilities into their applications through a conversational interface. It supports multi-turn conversations, tool use with JSON schema definitions, retrieval-augmented generation, and streaming responses. The API is available via the v2 endpoint and works with Cohere's Command family of models.
  version: '2.0'
  contact:
    name: Cohere Support
    url: https://support.cohere.com
  termsOfService: https://cohere.com/terms-of-use
servers:
- url: https://api.cohere.com
  description: Cohere Production Server
security:
- bearerAuth: []
tags:
- name: Detokenize
  description: Endpoints for converting token IDs back into text using a specified Cohere model's tokenizer.
paths:
  /v1/detokenize:
    post:
      operationId: detokenize
      summary: Detokenize token IDs
      description: Takes a list of token IDs and converts them back into their corresponding text string using byte-pair encoding for the specified model. This is the inverse operation of the Tokenize endpoint.
      tags:
      - Detokenize
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/DetokenizeRequest'
      responses:
        '200':
          description: Successful detokenization response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/DetokenizeResponse'
        '400':
          description: Bad request due to invalid parameters
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '401':
          description: Unauthorized due to missing or invalid API key
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
components:
  schemas:
    DetokenizeRequest:
      type: object
      required:
      - tokens
      - model
      properties:
        tokens:
          type: array
          description: The list of token IDs to be detokenized.
          items:
            type: integer
        model:
          type: string
          description: The name of the model whose tokenizer will be used to detokenize the input tokens.
          example: command-r-plus
    Error:
      type: object
      properties:
        message:
          type: string
          description: A human-readable error message describing what went wrong.
    DetokenizeResponse:
      type: object
      properties:
        text:
          type: string
          description: The text string resulting from detokenizing the input token IDs.
        meta:
          type: object
          description: Metadata about the API request.
          properties:
            api_version:
              type: object
              properties:
                version:
                  type: string
                  description: The API version used for the request.
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: Bearer authentication using a Cohere API key. Pass the API key in the Authorization header as Bearer <token>.
externalDocs:
  description: Cohere Chat API Documentation
  url: https://docs.cohere.com/reference/chat