Compoid Tokenize API
The Tokenize API from Compoid — 1 operation(s) for tokenize.
The Tokenize API from Compoid — 1 operation(s) for tokenize.
Every API here is available over the APIs.io API and to AI agents over MCP.
One button, every client — Claude, Cursor, VS Code and the rest.
https://apis.io/mcp
find_apisBrowse and filter every API in the catalog.get_api_artifactsOne API's artifacts, grouped by type.get_openapiThe primary OpenAPI for this API.find_similar_apisAPIs that look like this one.apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.resolveTurn a domain, URL or GitHub org into the provider it belongs to.find_cohortsEvery scored population of providers in the catalog.curl "https://apis.io/api/v1/apis/compoid-com:compoid-com-tokenize-api"
curl "https://apis.io/api/v1/apis?limit=25"
Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.
Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.
A second provider on the same verified email joins the account you already have.
openapi: 3.2.0
info:
title: Compoid Com Tokenize API
version: 0.1.0
x-refined-note:
- x-original-title differs across the merged source definitions and was not carried
description: 'Operations tagged Tokenize across 2 of this provider''s published API definitions: compoid-com-openapi.json, compoid-com-openapi.yml. Each path carries the servers of the definition it was published in.'
servers:
- url: https://api.compoid.com
description: Compoid AI inference host (paths carry the /v1 prefix)
tags:
- name: Tokenize
paths:
/tokenize:
post:
summary: Tokenize
operationId: tokenize_tokenize_post
requestBody:
content:
application/json:
schema:
anyOf:
- $ref: '#/components/schemas/TokenizeCompletionRequest'
- $ref: '#/components/schemas/TokenizeChatRequest'
title: Request
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema: {}
'400':
description: Bad Request
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'404':
description: Not Found
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'500':
description: Internal Server Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'501':
description: Not Implemented
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
tags:
- Tokenize
components:
schemas:
CustomChatCompletionContentSimpleAudioParam:
properties:
audio_url:
anyOf:
- type: string
- type: 'null'
title: Audio Url
additionalProperties: true
type: object
title: CustomChatCompletionContentSimpleAudioParam
description: "A simpler version of the param that only accepts a plain audio_url.\n\nExample:\n{\n \"audio_url\": \"https://example.com/audio.mp3\"\n}"
ChatCompletionContentPartAudioEmbedsParam:
properties:
audio_embeds:
anyOf:
- type: string
- additionalProperties:
anyOf:
- type: string
- items:
anyOf:
- type: integer
- type: number
type: array
type: object
- type: 'null'
title: Audio Embeds
type:
type: string
const: audio_embeds
title: Type
uuid:
anyOf:
- type: string
- type: 'null'
title: Uuid
additionalProperties: true
type: object
required:
- type
title: ChatCompletionContentPartAudioEmbedsParam
vllm__entrypoints__generate__base__protocol__FunctionDefinition:
properties:
name:
type: string
title: Name
description:
anyOf:
- type: string
- type: 'null'
title: Description
parameters:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Parameters
strict:
anyOf:
- type: boolean
- type: 'null'
title: Strict
defer_loading:
anyOf:
- type: boolean
- type: 'null'
title: Defer Loading
additionalProperties: true
type: object
required:
- name
title: FunctionDefinition
ChatCompletionFunctionMessageParam:
properties:
content:
anyOf:
- type: string
- type: 'null'
title: Content
name:
type: string
title: Name
role:
type: string
const: function
title: Role
additionalProperties: true
type: object
required:
- content
- name
- role
title: ChatCompletionFunctionMessageParam
ChatCompletionAssistantMessageParam:
properties:
role:
type: string
const: assistant
title: Role
audio:
anyOf:
- $ref: '#/components/schemas/Audio'
- type: 'null'
content:
anyOf:
- type: string
- items:
anyOf:
- $ref: '#/components/schemas/ChatCompletionContentPartTextParam'
- $ref: '#/components/schemas/ChatCompletionContentPartRefusalParam'
type: array
- type: 'null'
title: Content
function_call:
anyOf:
- $ref: '#/components/schemas/FunctionCall'
- type: 'null'
name:
type: string
title: Name
refusal:
anyOf:
- type: string
- type: 'null'
title: Refusal
tool_calls:
items:
anyOf:
- $ref: '#/components/schemas/ChatCompletionMessageFunctionToolCallParam'
- $ref: '#/components/schemas/ChatCompletionMessageCustomToolCallParam'
type: array
title: Tool Calls
additionalProperties: true
type: object
required:
- role
title: ChatCompletionAssistantMessageParam
description: Messages sent by the model in response to user messages.
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
openai__types__shared_params__function_definition__FunctionDefinition:
properties:
name:
type: string
title: Name
description:
type: string
title: Description
parameters:
additionalProperties: true
type: object
title: Parameters
strict:
anyOf:
- type: boolean
- type: 'null'
title: Strict
additionalProperties: true
type: object
required:
- name
title: FunctionDefinition
Audio:
properties:
id:
type: string
title: Id
additionalProperties: true
type: object
required:
- id
title: Audio
description: 'Data about a previous audio response from the model.
[Learn more](https://developers.openai.com/api/docs/guides/audio).'
CustomChatCompletionContentSimpleVideoParam:
properties:
video_url:
anyOf:
- type: string
- type: 'null'
title: Video Url
uuid:
anyOf:
- type: string
- type: 'null'
title: Uuid
additionalProperties: true
type: object
title: CustomChatCompletionContentSimpleVideoParam
description: "A simpler version of the param that only accepts a plain audio_url.\n\nExample:\n{\n \"video_url\": \"https://example.com/video.mp4\"\n}"
ChatCompletionDeveloperMessageParam:
properties:
content:
anyOf:
- type: string
- items:
$ref: '#/components/schemas/ChatCompletionContentPartTextParam'
type: array
title: Content
role:
type: string
const: developer
title: Role
name:
type: string
title: Name
additionalProperties: true
type: object
required:
- content
- role
title: ChatCompletionDeveloperMessageParam
description: 'Developer-provided instructions that the model should follow, regardless of
messages sent by the user. With o1 models and newer, `developer` messages
replace the previous `system` messages.'
ChatCompletionContentPartVideoParam:
properties:
video_url:
$ref: '#/components/schemas/VideoURL'
type:
type: string
const: video_url
title: Type
additionalProperties: true
type: object
required:
- video_url
- type
title: ChatCompletionContentPartVideoParam
CustomChatCompletionContentToolReferenceParam:
properties:
name:
type: string
title: Name
type:
type: string
const: tool_reference
title: Type
additionalProperties: true
type: object
title: CustomChatCompletionContentToolReferenceParam
description: "A tool reference content param that only accepts a plain tool name.\n\nExample:\n{\n \"name\": \"get_weather\",\n \"type\": \"tool_reference\"\n}"
Role:
type: string
enum:
- user
- assistant
- system
- developer
- tool
title: Role
description: The role of a message author (mirrors ``chat::Role``).
ChatCompletionMessageCustomToolCallParam:
properties:
id:
type: string
title: Id
custom:
$ref: '#/components/schemas/Custom'
type:
type: string
const: custom
title: Type
additionalProperties: true
type: object
required:
- id
- custom
- type
title: ChatCompletionMessageCustomToolCallParam
description: A call to a custom tool created by the model.
ChatCompletionMessageFunctionToolCallParam:
properties:
id:
type: string
title: Id
function:
$ref: '#/components/schemas/Function'
type:
type: string
const: function
title: Type
additionalProperties: true
type: object
required:
- id
- function
- type
title: ChatCompletionMessageFunctionToolCallParam
description: A call to a function tool created by the model.
ChatCompletionContentPartInputAudioParam:
properties:
input_audio:
$ref: '#/components/schemas/InputAudio'
type:
type: string
const: input_audio
title: Type
prompt_cache_breakpoint:
$ref: '#/components/schemas/PromptCacheBreakpoint'
additionalProperties: true
type: object
required:
- input_audio
- type
title: ChatCompletionContentPartInputAudioParam
description: Learn about [audio inputs](https://developers.openai.com/api/docs/guides/audio).
ErrorResponse:
properties:
error:
$ref: '#/components/schemas/ErrorInfo'
additionalProperties: true
type: object
required:
- error
title: ErrorResponse
FunctionCall:
properties:
arguments:
type: string
title: Arguments
name:
type: string
title: Name
additionalProperties: true
type: object
required:
- arguments
- name
title: FunctionCall
description: 'Deprecated and replaced by `tool_calls`.
The name and arguments of a function that should be called, as generated by the model.'
ChatCompletionContentPartImageEmbedsParam:
properties:
image_embeds:
anyOf:
- type: string
- additionalProperties:
anyOf:
- type: string
- items:
anyOf:
- type: integer
- type: number
type: array
type: object
- type: 'null'
title: Image Embeds
type:
type: string
const: image_embeds
title: Type
uuid:
anyOf:
- type: string
- type: 'null'
title: Uuid
additionalProperties: true
type: object
required:
- type
title: ChatCompletionContentPartImageEmbedsParam
ChatCompletionContentPartTextParam:
properties:
text:
type: string
title: Text
type:
type: string
const: text
title: Type
prompt_cache_breakpoint:
$ref: '#/components/schemas/PromptCacheBreakpoint'
additionalProperties: true
type: object
required:
- text
- type
title: ChatCompletionContentPartTextParam
description: Learn about [text inputs](https://developers.openai.com/api/docs/guides/text).
ChatCompletionFunctionToolParam:
properties:
function:
$ref: '#/components/schemas/openai__types__shared_params__function_definition__FunctionDefinition'
type:
type: string
const: function
title: Type
additionalProperties: true
type: object
required:
- function
- type
title: ChatCompletionFunctionToolParam
description: A function tool that can be used to generate a response.
ChatCompletionContentPartAudioParam:
properties:
audio_url:
$ref: '#/components/schemas/AudioURL'
type:
type: string
const: audio_url
title: Type
additionalProperties: true
type: object
required:
- audio_url
- type
title: ChatCompletionContentPartAudioParam
ChatCompletionContentPartVideoEmbedsParam:
properties:
video_embeds:
anyOf:
- type: string
- additionalProperties:
anyOf:
- type: string
- items:
anyOf:
- type: integer
- type: number
type: array
type: object
- type: 'null'
title: Video Embeds
type:
type: string
const: video_embeds
title: Type
uuid:
anyOf:
- type: string
- type: 'null'
title: Uuid
additionalProperties: true
type: object
required:
- type
title: ChatCompletionContentPartVideoEmbedsParam
CustomChatCompletionMessageParam:
properties:
role:
type: string
title: Role
content:
anyOf:
- type: string
- items:
anyOf:
- $ref: '#/components/schemas/ChatCompletionContentPartTextParam'
- $ref: '#/components/schemas/ChatCompletionContentPartImageParam'
- $ref: '#/components/schemas/ChatCompletionContentPartInputAudioParam'
- $ref: '#/components/schemas/File'
- $ref: '#/components/schemas/ChatCompletionContentPartAudioParam'
- $ref: '#/components/schemas/ChatCompletionContentPartVideoParam'
- $ref: '#/components/schemas/ChatCompletionContentPartRefusalParam'
- $ref: '#/components/schemas/CustomChatCompletionContentSimpleImageParam'
- $ref: '#/components/schemas/ChatCompletionContentPartImageEmbedsParam'
- $ref: '#/components/schemas/ChatCompletionContentPartAudioEmbedsParam'
- $ref: '#/components/schemas/ChatCompletionContentPartVideoEmbedsParam'
- $ref: '#/components/schemas/ChatCompletionContentPartPromptEmbedsParam'
- $ref: '#/components/schemas/CustomChatCompletionContentSimpleAudioParam'
- $ref: '#/components/schemas/CustomChatCompletionContentSimpleVideoParam'
- $ref: '#/components/schemas/CustomChatCompletionContentToolReferenceParam'
- type: string
- $ref: '#/components/schemas/CustomThinkCompletionContentParam'
type: array
title: Content
name:
type: string
title: Name
tool_call_id:
anyOf:
- type: string
- type: 'null'
title: Tool Call Id
tool_calls:
anyOf:
- items:
$ref: '#/components/schemas/ChatCompletionMessageFunctionToolCallParam'
type: array
- type: 'null'
title: Tool Calls
reasoning:
anyOf:
- type: string
- type: 'null'
title: Reasoning
tools:
anyOf:
- items:
$ref: '#/components/schemas/ChatCompletionFunctionToolParam'
type: array
- type: 'null'
title: Tools
task:
anyOf:
- type: string
- type: 'null'
title: Task
additionalProperties: true
type: object
required:
- role
title: CustomChatCompletionMessageParam
description: Enables custom roles in the Chat Completion API.
PromptCacheBreakpoint:
properties:
mode:
type: string
const: explicit
title: Mode
additionalProperties: true
type: object
required:
- mode
title: PromptCacheBreakpoint
description: 'Marks the exact end of a reusable prompt prefix.
The breakpoint inherits its TTL from the request''s `prompt_cache_options.ttl`; the boundary is not rounded to a token block.'
ChatCompletionContentPartImageParam:
properties:
image_url:
$ref: '#/components/schemas/ImageURL'
type:
type: string
const: image_url
title: Type
prompt_cache_breakpoint:
$ref: '#/components/schemas/PromptCacheBreakpoint'
additionalProperties: true
type: object
required:
- image_url
- type
title: ChatCompletionContentPartImageParam
description: Learn about [image inputs](https://developers.openai.com/api/docs/guides/images-vision).
openai_harmony__Content:
properties: {}
type: object
title: Content
ChatCompletionContentPartRefusalParam:
properties:
refusal:
type: string
title: Refusal
type:
type: string
const: refusal
title: Type
additionalProperties: true
type: object
required:
- refusal
- type
title: ChatCompletionContentPartRefusalParam
CustomChatCompletionContentSimpleImageParam:
properties:
image_url:
anyOf:
- type: string
- type: 'null'
title: Image Url
uuid:
anyOf:
- type: string
- type: 'null'
title: Uuid
additionalProperties: true
type: object
title: CustomChatCompletionContentSimpleImageParam
description: "A simpler version of the param that only accepts a plain image_url.\nThis is supported by OpenAI API, although it is not documented.\n\nExample:\n{\n \"image_url\": \"https://example.com/image.jpg\"\n}"
FilePromptCacheBreakpoint:
properties:
mode:
type: string
const: explicit
title: Mode
additionalProperties: true
type: object
required:
- mode
title: FilePromptCacheBreakpoint
description: 'Marks the exact end of a reusable prompt prefix.
The breakpoint inherits its TTL from the request''s `prompt_cache_options.ttl`; the boundary is not rounded to a token block.'
Author:
properties:
role:
$ref: '#/components/schemas/Role'
name:
anyOf:
- type: string
- type: 'null'
title: Name
type: object
required:
- role
title: Author
Function:
properties:
arguments:
type: string
title: Arguments
name:
type: string
title: Name
additionalProperties: true
type: object
required:
- arguments
- name
title: Function
description: The function that the model called.
ChatCompletionToolsParam:
properties:
type:
type: string
const: function
title: Type
default: function
function:
$ref: '#/components/schemas/vllm__entrypoints__generate__base__protocol__FunctionDefinition'
defer_loading:
anyOf:
- type: boolean
- type: 'null'
title: Defer Loading
additionalProperties: true
type: object
required:
- function
title: ChatCompletionToolsParam
AudioURL:
properties:
url:
type: string
title: Url
additionalProperties: true
type: object
required:
- url
title: AudioURL
ChatCompletionUserMessageParam:
properties:
content:
anyOf:
- type: string
- items:
anyOf:
- $ref: '#/components/schemas/ChatCompletionContentPartTextParam'
- $ref: '#/components/schemas/ChatCompletionContentPartImageParam'
- $ref: '#/components/schemas/ChatCompletionContentPartInputAudioParam'
- $ref: '#/components/schemas/File'
type: array
title: Content
role:
type: string
const: user
title: Role
name:
type: string
title: Name
additionalProperties: true
type: object
required:
- content
- role
title: ChatCompletionUserMessageParam
description: 'Messages sent by an end user, containing prompts or additional context
information.'
Custom:
properties:
input:
type: string
title: Input
name:
type: string
title: Name
additionalProperties: true
type: object
required:
- input
- name
title: Custom
description: The custom tool that the model called.
CustomThinkCompletionContentParam:
properties:
thinking:
type: string
title: Thinking
closed:
type: boolean
title: Closed
type:
type: string
const: thinking
title: Type
additionalProperties: true
type: object
required:
- thinking
- type
title: CustomThinkCompletionContentParam
description: "A Think Completion Content Param that accepts a plain text and a boolean.\n\nExample:\n{\n \"thinking\": \"I am thinking about the answer\",\n \"closed\": True,\n \"type\": \"thinking\"\n}"
ImageURL:
properties:
url:
type: string
title: Url
detail:
type: string
enum:
- auto
- low
- high
- original
title: Detail
additionalProperties: true
type: object
required:
- url
title: ImageURL
ChatCompletionContentPartPromptEmbedsParam:
properties:
data:
type: string
title: Data
type:
type: string
const: prompt_embeds
title: Type
additionalProperties: true
type: object
required:
- data
- type
title: ChatCompletionContentPartPromptEmbedsParam
openai_harmony__Message:
properties:
author:
$ref: '#/components/schemas/Author'
content:
items:
$ref: '#/components/schemas/openai_harmony__Content'
type: array
title: Content
channel:
anyOf:
- type: string
- type: 'null'
title: Channel
recipient:
anyOf:
- type: string
- type: 'null'
title: Recipient
content_type:
anyOf:
- type: string
- type: 'null'
title: Content Type
type: object
required:
- author
title: Message
FileFile:
properties:
file_data:
type: string
title: File Data
file_id:
type: string
title: File Id
filename:
type: string
title: Filename
additionalProperties: true
type: object
title: FileFile
VideoURL:
properties:
url:
type: string
title: Url
additionalProperties: true
type: object
required:
- url
title: VideoURL
ChatCompletionSystemMessageParam:
properties:
content:
anyOf:
- type: string
- items:
$ref: '#/components/schemas/ChatCompletionContentPartTextParam'
type: array
title: Content
role:
type: string
const: system
title: Role
name:
type: string
title: Name
additionalProperties: true
type: object
required:
- content
- role
title: ChatCompletionSystemMessageParam
description: 'Developer-provided instructions that the model should follow, regardless of
messages sent by the user. With o1 models and newer, use `developer` messages
for this purpose instead.'
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
input:
title: Input
ctx:
type: object
title: Context
type: object
required:
- loc
- msg
- type
title: ValidationError
ErrorInfo:
properties:
message:
type: string
title: Message
type:
type: string
title: Type
param:
anyOf:
- type: string
- type: 'null'
title: Param
code:
type: integer
title: Code
additionalProperties: true
type: object
required:
- message
- type
- code
title: ErrorInfo
File:
properties:
file:
$ref: '#/components/schemas/FileFile'
type:
type: string
const: file
title: Type
prompt_cache_breakpoint:
$ref: '#/components/schemas/FilePromptCacheBreakpoint'
additionalProperties: true
type: object
required:
- file
- type
title: File
description: Learn about [file inputs](https://developers.openai.com/api/docs/guides/text) for text generation.
InputAudio:
properties:
data:
type: string
title: Data
format:
type: string
enum:
- wav
- mp3
title: Format
additionalProperties: true
type: object
required:
- data
- format
title: InputAudio
TokenizeChatRequest:
properties:
model:
anyOf:
- type: string
- type: 'null'
title: Model
messages:
items:
anyOf:
- $ref: '#/components/schemas/ChatCompletionDeveloperMessageParam'
- $ref: '#/components/schemas/ChatCompletionSystemMessageParam'
- $ref: '#/components/schemas/ChatCompletionUserMessageParam'
- $ref: '#/components/schemas/ChatCompletionAssistantMessageParam'
- $ref: '#/components/schemas/ChatCompletionToolMessageParam'
- $ref: '#/components/schemas/ChatCompletionFunctionMessageParam'
- $ref: '#/components/schemas/CustomChatCompletionMessageParam'
- $ref: '#/components/schemas/openai_harmony__Message'
type: array
title: Messages
add_generation_prompt:
type: boolean
title: Add Generation Prompt
description: If true, the generation prompt will be added to the chat template. This is a parameter used by chat template in tokenizer config of the model.
default: true
return_token_strs:
anyOf:
- type: boolean
- type: 'null'
title: Return Token Strs
description: If true, also return the token strings corresponding to the token ids.
default: false
continue_final_message:
type: boolean
title: Continue Final Message
description: If this is set, the chat will be formatted so that the final message in the chat is open-ended, without any EOS tokens. The model will continue this message rather than starting a new one. This allows you to "prefill" part of the model's response for it. Cannot be used at the same time as `add_generation_prompt`.
default: false
add_special_tokens:
type: boolean
title: Add Special Tokens
description: If true, special tokens (e.g. BOS) will be added to the prompt on top of what is added by the chat template. For most models, the chat template takes care of adding the special tokens so this should be set to false (as is the default).
default: false
chat_template:
anyOf:
- type: string
- type: 'null'
title: Chat Template
description: A Jinja template to use for this conversion. As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one.
chat_template_kwargs:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Chat Template Kwargs
description: Additional keyword args to pass to the template renderer. Will be accessible by the chat template.
media_io_kwargs:
anyOf:
- additionalProperties:
additionalProperties: true
type: object
type: object
- type: 'null'
title: Media Io Kwargs
description: Additional kwargs to pass to the media IO connectors, keyed by modality. Merged with engine-level media_io_kwargs.
mm_processor_kwargs:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Mm Processor Kwargs
description: Additional kwargs to pass to the HF processor.
tools:
anyOf:
- items:
$ref: '#/components/schemas/ChatCompletionToolsParam'
type: array
- type: 'null'
title: Tools
description: A list of tools the model may call.
additionalProperties: true
type: object
required:
- messages
title: TokenizeChatRequest
TokenizeCompletionRequest:
properties:
model:
anyOf:
- type: string
- type: 'null'
title: Model
prompt:
type: string
title: Prompt
add_special_tokens:
type: boolean
title: Add Special Tokens
description: If true (the default), special tokens (e.g. BOS) will be added to the prompt.
default: true
return_token_strs:
anyOf:
- type: boolean
- type: 'null'
title: Return Token Strs
description: If true, also return the token strings corresponding to the token ids.
default: false
additionalProperties: true
type: object
required:
- prompt
title: TokenizeCompletionRequest
ChatCompletionToolMessageParam:
properties:
content:
anyOf:
- type: string
- items:
$ref: '#/components/schemas/ChatCompletionContentPartTextParam'
type: array
title: Content
role:
type: string
const: tool
title: Role
tool_call_id:
type: string
title: Tool Call Id
additionalProperties: true
type: object
required:
- content
- role
- tool_call_id
title: ChatCompletionToolMessageParam
x-refined-from:
- compoid-com-openapi.json
- compoid-com-openapi.yml