Crawlbase · Rate Limits

Crawlbase Rate Limits

Crawlbase governs throughput primarily by concurrent request threads and by your plan's monthly credit / request allowance rather than a single global per-minute cap. The default documented ceiling on the Crawling API is roughly 20 requests per second (the same default applies to the proxy interface at smartproxy.crawlbase.com); higher thread counts are unlocked by higher-tier subscriptions. Only requests that return pc_status 200 count against a pay-as-you-go quota - failed requests (timeouts, blocks, target 5xx) are free. Scroll-heavy JavaScript requests are metered by processing time (first 8 seconds = 1 request, each additional 5 seconds = 1 more request).

Crawlbase Rate Limits is the machine-readable rate-limit profile for Crawlbase on the APIs.io network, conforming to the API Commons Rate Limits specification.

It captures 4 rate-limit definitions, measuring requests, threads, and seconds.

The profile also includes 3 backoff/retry policies defined and response codes documented for throttled.

Tagged areas include Web Scraping, Web Crawling, Proxy, Rate Limiting, and Quotas.

4 Limits Throttle: 429
Web ScrapingWeb CrawlingProxyRate LimitingQuotas

Limits

Crawling API Request Rate account
requests
~20 requests/second (default)
Documented default ceiling; the proxy interface (smartproxy.crawlbase.com) shares the same default.
Concurrent Threads account
threads
per plan
Higher subscription tiers unlock more concurrent request threads.
Monthly Credit / Request Allowance account
requests
per plan
Pay-as-you-go and subscription tiers meter monthly successful requests / credits. Only pc_status 200 responses are charged.
Scroll Processing Metering request
seconds
1 request per first 8s, +1 per additional 5s
JavaScript scroll requests are billed by server processing time rather than a flat per-request cost.

Policies

Charge on Success Only
Only requests that return pc_status 200 count against quota; failed requests (timeouts, blocks, target 5xx) are free.
Backoff Strategy
Clients should implement exponential backoff with jitter and honor Retry-After on 429 responses.
Async Queueing
The Crawling API async=true mode queues requests and returns a request id, smoothing bursts and delivering results to a callback webhook.

Sources

Work with this as data

Every rate limit here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for rate limits

4 MCP tools reach this
  • find_rate_limitsBrowse and filter every rate limit in the catalog.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools

Call it yourself

curl for this page
This rate limit 
curl "https://apis.io/api/v1/rate-limits/crawlbase-rate-limits"
All rate limits 
curl "https://apis.io/api/v1/rate-limits?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no email required.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.