NVIDIA NIM · Rate Limits
Nvidia Nim Rate Limits
Nvidia Nim Rate Limits is the machine-readable rate-limit profile for NVIDIA NIM on the APIs.io network, in the API Commons Rate Limits shape.
It captures 6 rate-limit definitions.
This profile was written by API Evangelist and has not been verified against a page the provider published; treat every limit as unconfirmed until a dated transcription replaces it.
The profile also includes 7 backoff/retry policies defined.
Tagged areas include Artificial Intelligence, Inference, Microservices, LLM, and Foundation Models.
6 Limits
Method:
generated, dated 2026-05-25.
Written by API Evangelist on 2026-05-25 and not verified against a page the provider published. Treat every limit as unconfirmed until a dated transcription replaces it.
View Source
Artificial IntelligenceInferenceMicroservicesLLMFoundation ModelsGPUKubernetesNVIDIAOpenAI-Compatible
Policies
Hosted Developer RPM
Free developer tier soft rate limit on /v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/ranking.
Hosted Developer Credits
1,000 free inference credits granted on NVIDIA Developer Program signup, consumed across all hosted models.
Hosted Concurrent Requests
Concurrent in-flight requests against the shared hosted endpoint.
SSE Keepalive
Streaming connections idle longer than ~60s may be closed by the gateway.
Per-request Output Cap
Per-request output token cap on the hosted endpoint (model-dependent; some models support higher).
Hosted Input Size
Maximum input tokens on the hosted endpoint (model-dependent — long-context Llama 3.1/3.3 and Nemotron variants accept up to ~128K).
Self-hosted Container Concurrency
Self-hosted NIM containers accept as many concurrent requests as the GPU and TensorRT-LLM batching configuration allow. Operators tune via NIM_MAX_BATCH_SIZE and similar env vars.
Work with this as data
Every rate limit here is available over the APIs.io API and to AI agents over MCP.