AI Crawler Index · Rate Limits
Pathwren Rate Limits
Pathwren Rate Limits is the rate-limit profile for AI Crawler Index on the APIs.io network.
The provider publishes no rate limits that could be read; the profile records that absence rather than inventing a ceiling.
Limit state is signalled in the rate_limit_headers, observed_on, observed_at, and http_status response headers.
Tagged areas include AI crawlers, web crawlers, Robots.txt, User Agents, and Bot Detection.
0 Limits
rate_limit_headersobserved_onobserved_athttp_status
Read from the provider's live responses on 2026-09-01 — www.pathwren.workers.dev/.well-known/api-onboarding.
View Source
Limits
AI Crawler Index publishes no rate limits that could be read; the profile records that absence rather than inventing a ceiling.
AI crawlersweb crawlersRobots.txtUser AgentsBot DetectionGPTBotClaudeBotcrawler IP rangesllms-txtOpen DataA2A
Work with this as data
Every rate limit here is available over the APIs.io API and to AI agents over MCP.