LiteLLM

LiteLLM is an open-source Python SDK and proxy server providing a unified OpenAI-compatible interface to 100+ LLM providers.

20 APIs 0 Features

Gateways

APIs

LiteLLM Chat Completions API

Provides an OpenAI-compatible /chat/completions endpoint that routes requests to 100+ LLM providers with unified request and response formatting, streaming support, cost trackin...

LiteLLM Completions API

Provides an OpenAI-compatible /completions endpoint for text completion requests routed through the LiteLLM proxy to supported LLM providers.

LiteLLM Responses API

Provides an OpenAI-compatible /responses endpoint supporting the Responses API specification, including conversation history compression via /responses/compact.

LiteLLM Embeddings API

Provides an OpenAI-compatible /embeddings endpoint for generating text embeddings across multiple providers including OpenAI, Cohere, HuggingFace, and Bedrock with unified forma...

LiteLLM Image Generation API

Provides OpenAI-compatible /images/generations, /images/edits, and /images/variations endpoints for image generation and manipulation routed through the LiteLLM proxy.

LiteLLM Audio API

Provides OpenAI-compatible /audio/transcriptions and /audio/speech endpoints for audio transcription and text-to-speech conversion across supported providers.

LiteLLM Moderations API

Provides an OpenAI-compatible /moderations endpoint for content moderation across supported providers through the LiteLLM proxy.

LiteLLM Batches API

Provides an OpenAI-compatible /batches endpoint for batch processing operations, enabling bulk request handling across LLM providers.

LiteLLM Files API

Provides an OpenAI-compatible /files endpoint for file management operations used in conjunction with fine-tuning and batch processing.

LiteLLM Fine-Tuning API

Provides an OpenAI-compatible /fine_tuning endpoint for model fine-tuning operations across supported providers through the LiteLLM proxy.

LiteLLM Rerank API

Provides a /rerank endpoint for document reranking operations, supporting providers like Cohere through the LiteLLM proxy with a unified interface.

LiteLLM Vector Stores API

Provides /vector_stores endpoints for creating and managing vector stores, file operations within vector stores, and search functionality for retrieval-augmented generation (RAG...

LiteLLM Anthropic Messages API

Provides Anthropic-compatible /v1/messages and /v1/messages/count_tokens endpoints for native Anthropic API format support through the LiteLLM proxy.

LiteLLM Realtime API

Provides /realtime WebSocket endpoints for real-time model interactions with load balancing and guardrails support across providers.

LiteLLM MCP API

Provides /mcp endpoints for Model Context Protocol (MCP) integration, enabling LLMs to interact with external tools and APIs through OpenAPI specifications.

LiteLLM OCR API

Provides an /ocr endpoint for optical character recognition, enabling text extraction from images through supported providers via the LiteLLM proxy.

LiteLLM Guardrails API

Provides /guardrails/apply_guardrail endpoint for applying configured content filtering and safety guardrails to LLM requests and responses.

LiteLLM Evals API

Provides /evals endpoints for the Evaluations API, enabling measurement and benchmarking of model performance through the LiteLLM proxy.

LiteLLM A2A Agent Gateway API

Provides /a2a endpoints for the Agent-to-Agent (A2A) gateway, enabling agent registration, publishing, and inter-agent communication.

LiteLLM Videos API

Provides /videos endpoints for video generation and handling through supported providers like RunwayML via the LiteLLM proxy.

Collections

LiteLLM Proxy API

OPEN

Pricing Plans

Litellm Plans Pricing

3 plans

PLANS

Rate Limits

Litellm Rate Limits

5 limits

RATE LIMITS

FinOps

Litellm Finops

FINOPS

Resources

Sources

opencollection: 1.0.0
info:
  name: LiteLLM Proxy API
  version: 1.0.0
items:
- info:
    name: Create a chat completion
    type: http
  http:
    method: POST
    url: http://localhost:4000/chat/completions
  docs: Create a chat completion
- info:
    name: Create a text completion
    type: http
  http:
    method: POST
    url: http://localhost:4000/completions
  docs: Create a text completion
- info:
    name: Create an embedding vector
    type: http
  http:
    method: POST
    url: http://localhost:4000/embeddings
  docs: Create an embedding vector
- info:
    name: Transcribe audio to text
    type: http
  http:
    method: POST
    url: http://localhost:4000/audio/transcriptions
  docs: Transcribe audio to text
- info:
    name: Synthesize speech from text
    type: http
  http:
    method: POST
    url: http://localhost:4000/audio/speech
  docs: Synthesize speech from text
- info:
    name: Generate images
    type: http
  http:
    method: POST
    url: http://localhost:4000/images/generations
  docs: Generate images
- info:
    name: Edit an image
    type: http
  http:
    method: POST
    url: http://localhost:4000/images/edits
  docs: Edit an image
- info:
    name: Generate image variations
    type: http
  http:
    method: POST
    url: http://localhost:4000/images/variations
  docs: Generate image variations
- info:
    name: Classify text for moderation
    type: http
  http:
    method: POST
    url: http://localhost:4000/moderations
  docs: Classify text for moderation
- info:
    name: Create a fine-tuning job
    type: http
  http:
    method: POST
    url: http://localhost:4000/fine_tuning/jobs
  docs: Create a fine-tuning job
- info:
    name: Create a batch job
    type: http
  http:
    method: POST
    url: http://localhost:4000/batches
  docs: Create a batch job
- info:
    name: Create an assistant
    type: http
  http:
    method: POST
    url: http://localhost:4000/assistants
  docs: Create an assistant
- info:
    name: Create a vector store
    type: http
  http:
    method: POST
    url: http://localhost:4000/vector_stores
  docs: Create a vector store
- info:
    name: Search a vector store
    type: http
  http:
    method: POST
    url: http://localhost:4000/vector_stores/search
  docs: Search a vector store
- info:
    name: Create a message (Anthropic-compatible)
    type: http
  http:
    method: POST
    url: http://localhost:4000/v1/messages
  docs: Create a message (Anthropic-compatible)
- info:
    name: Count tokens for a message
    type: http
  http:
    method: POST
    url: http://localhost:4000/v1/messages/count_tokens
  docs: Count tokens for a message
- info:
    name: Run OCR on an image
    type: http
  http:
    method: POST
    url: http://localhost:4000/ocr
  docs: Run OCR on an image
- info:
    name: Rerank documents against a query
    type: http
  http:
    method: POST
    url: http://localhost:4000/rerank
  docs: Rerank documents against a query
- info:
    name: Ingest documents into a RAG index
    type: http
  http:
    method: POST
    url: http://localhost:4000/rag/ingest
  docs: Ingest documents into a RAG index
- info:
    name: Query a RAG index
    type: http
  http:
    method: POST
    url: http://localhost:4000/rag/query
  docs: Query a RAG index
bundled: true