CentML · Rate Limits

Centml Rate Limits

CentML's serverless inference API enforces per-account rate limits expressed as requests per minute and tokens per minute, which vary by model and account tier. Dedicated deployments are governed by the capacity of the provisioned GPU hardware and configured autoscaling (min/max replicas) rather than shared account-level token limits. Specific per-model limit values are not reconciled in this artifact.

Centml Rate Limits is the machine-readable rate-limit profile for CentML on the APIs.io network, conforming to the API Commons Rate Limits specification.

It captures 4 rate-limit definitions, measuring requests, tokens, and replicas.

The profile also includes 2 backoff/retry policies defined and response codes documented for throttled.

Tagged areas include AI, LLM, Inference, Serverless, and GPU.

4 Limits Throttle: 429
AILLMInferenceServerlessGPURate LimitingQuotasThrottling

Limits

Requests Per Minute (RPM) account
requests
see provider documentation
Per-model RPM on serverless endpoints, varies by tier and model.
Tokens Per Minute (TPM) account
tokens
see provider documentation
Per-model TPM on serverless endpoints, varies by tier and model.
Concurrent Requests account
requests
see provider documentation
Concurrency permitted against serverless endpoints, varies by tier.
Dedicated Replica Capacity deployment
replicas
configured via min/max replicas
Dedicated deployment throughput is bounded by provisioned GPU hardware and autoscaling settings, not shared account token limits.

Policies

Tiered Limits
Limits raise as accounts move from free / trial to paid usage and via Enterprise agreements.
Backoff Strategy
Clients should implement exponential backoff with jitter and honor Retry-After on 429 responses.

Sources