AI Crawler Index

Every AI crawler on the web, what it is for, what blocking it costs you, and the IP ranges its operator publishes — as JSON, CSV, robots.txt and regex. 56 crawlers from 30 operators, 1887 IPv4 and 1056 IPv6 prefixes mirrored from 12 operator-published endpoints and refreshed every six hours. Read-only static files: no key, no signup, no rate limit, CORS open, CC0.

AI Crawler Index publishes 5 API contracts indexed on the APIs.io network, including Bulk API, Crawlers API, Ip Ranges API, and 2 more. Tagged areas include AI crawlers, web crawlers, Robots.txt, User Agents, and Bot Detection.

The AI Crawler Index catalog on APIs.io includes 1 Spectral governance ruleset.

AI Crawler Index’s developer surface includes pricing, documentation, support, changelog, and 19 more developer resources.

🌐 Visit website 📡 Profile data on GitHub

47.0/100 developing ▬ flat Agent 47/100 agent ready Heartbeat 97/100 alive Full breakdown ↓
scored 2026-11-02 · rubric v0.26.0 · Show this score on your site →

AccessFreeOpen⚡ Free to try

1 published contract 1 MCP Servers 4 Agent Skills

AI crawlersweb crawlersRobots.txt User Agents Bot DetectionGPTBotClaudeBotcrawler IP rangesllms-txt Open Data A2A

Kin Score

Kin Score How this is scored →
scored 2026-11-02 · rubric v0.26.0

Composite quality — 47.0/100 · developing

Contract Quality 9.7 / 21

Developer Ergonomics 9.6 / 17

Access Clarity 5.4 / 17

Operational Transparency 4.7 / 11

Contract Governance 3.2 / 10

Discoverability 6.2 / 9

Regulatory Posture 3.4 / 15

Agent readiness — 47/100 · agent ready

Machine-Readable Contract 18 / 18

Agentic Access Contract 0 / 10

Documented Reversibility 6 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 9 / 9

Stable Error Semantics 8 / 8

Request/Response Examples 7 / 7

Rate-Limit Signaling 7 / 7

Typed Event Surface 0 / 6

Agent Skills 5 / 5

Well-Known Catalog 4 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 8 / 8

Dry-Run / Simulate Mode 4 / 4

Delegated User Identity 0 / 6

Protected Resource Metadata 0 / 5

Registration Without a Human 0 / 6

Agentic Commerce Well-Known Document 0 / 5

Pagination 5 / 5

Distinct Operations 3 / 3

MCP Tool Quality 6 / 6

Regulatory Posture applies to this provider. Its tags matched the Government & Public Sector regime, so Regulatory Posture carries 15 points of the composite. If this regime is wrong for your business, say so on your provider repo — the applicability map is public and we will correct it.

Create-or-Update Ergonomics does not apply to this provider. The published contracts declare no write operations, and a read-only API cannot create-or-update. The facet is excluded from this provider's denominator entirely — not scored zero. A reference or data API is not deficient for being unable to upsert.

The six quality facets above are damped to 85 points between them, because the conditional facet above carries the other 15. That is why each facet's contribution is shown against a damped maximum: raising a quality facet moves the composite by 85% of its nominal weight, not 100%. The full arithmetic is at apis.io/rating/.

Show this score on your site.

It updates itself when the score changes. Paste this where you link to your API docs:

<a href="https://apis.io/providers/pathwren/"><img src="https://apis.io/badge/pathwren/card.svg" alt="AI Crawler Index on APIs.io — Kin Score by API Evangelist"></a>

Other styles, a README pill and a shields.io endpoint are on the badge page.

Raise your score.

Agent readiness — checks not met, largest first (exact, from this profile):

Agentic Access Contract +10 pts — agentic_access artifact present (x-agentic-access), graded by author
Registration Without a Human +6 pts — a `registration_endpoint` in the served RFC 8414 / OIDC discovery document
Delegated User Identity +6 pts — a served discovery document listing authorization_code in grant_types_supported (served), else an oauth2 authorizationCode flow or openIdConnect declared in an OpenAPI (documented)
Typed Event Surface +6 pts — AsyncAPI document, OR webhooks/callbacks objects in the as-published OpenAPI, OR common[].type includes Webhooks
Protected Resource Metadata +5 pts — a served /.well-known/oauth-protected-resource carrying `resource` (documented), and naming its authorization_servers (verified)
Agentic Commerce Well-Known Document +5 pts — a parsing /.well-known/ucp.json or acp.json declaring a version and at least one service — `self` when its endpoints are on the provider's own domain, `platform` when they are on a vendor's

1 more unmet check in the Agent Readiness table above.

Kin Score — facets with the most composite points still available (weight × headroom):

Access Clarity scores 31.6 — up to +13.7 composite points
Contract Quality scores 45.6 — up to +13.6 composite points
Developer Ergonomics scores 56.5 — up to +8.7 composite points

Each check links to what satisfies it. Publish the artifact, or tell us where it already lives, and the score moves on the next scoring run.

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from the provider profile at artifacts.apis.io/pathwren/apis.yml: open an issue on github.com/api-evangelist/pathwren to ask a question or point us at an artifact you publish. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

Standards implemented 4

Standards this provider implements or carries. De facto ones are profiled and measured by the API Commons; formal ones come from the standards directory and are linked by the provider's own tags. The evidence column says how each claim was established — not that it was made.

Agent2Agent

measured — observed by APIs.io (conformant)

llms.txt

tagged — the provider's own APIs.json carries the standard as a tag; implementation not verified

Model Context Protocol (MCP)

measured — observed by APIs.io (verified)

OpenAPI

measured — observed by APIs.io (published)

APIs 5

Individual APIs this provider publishes, each with its own machine-readable definition.

AI Crawler Index Bulk API

The whole dataset in several shapes.

AI Crawler Index Crawlers API

One record per crawler.

AI Crawler Index Ip Ranges API

Operator-published prefixes, normalised.

AI Crawler Index Robots API

Ready-made robots.txt policy files.

AI Crawler Index Status API

Freshness of the upstream sources.

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

AI Crawler Index MCP Server

Remote MCP server at www.pathwren.workers.dev over streamable HTTP; 9 tools listed.

MCP SERVER

Agent Skills 4

Packaged agent skills for driving this provider's APIs from an AI assistant.

build-robots-policy

AGENT SKILL

identify-crawler

AGENT SKILL

monitor-index-freshness

AGENT SKILL

verify-crawler-ip

AGENT SKILL

Pricing Plans 1

Published pricing tiers and plan structures.

Pathwren Plans Pricing

1 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Pathwren Rate Limits

0 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Pathwren Finops

FINOPS

Spectral Rules 1

Spectral governance rulesets for linting and validating these APIs.

AI Crawler Index API Rules

54 rules · 43 errors 6 warnings 5 info

SPECTRAL

OpenAPI Overlays 1

Overlays applied on top of this provider's contracts. Each card says who wrote it: a document the provider publishes, or one API Evangelist derived or generated.

Pathwren Ai Crawler Index Overlay

5 actions · update · pathwren-ai-crawler-index-openapi.json

GENERATED

JSON Schema 1

Standalone JSON Schema definitions for this provider's data models.

Crawler

17 properties

JSON SCHEMA

Security Posture 3

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Pathwren Authentication

0 schemes

SECURITY

Pathwren Domain Security

TLSv1.3 · HSTS · DMARC

SECURITY

Pathwren Vulnerability Disclosure

security.txt · contact published

SECURITY

Resources

Get Started 1

Portal, sign-up, and the first successful call

DeveloperPortal

DeveloperPortal

Documentation 1

Reference material describing how the API behaves

Documentation

Documentation

Agent Surfaces 4

MCP servers, agent skills, and machine-readable catalogs

WellKnown

LLMsTxt

MCPServer

AgentSkill

Design & Contract 2

Pagination, idempotency, versioning, errors, and events

Conformance

Conformance

Lifecycle

Lifecycle

Build 1

SDKs, sample code, and the tooling you integrate with

Packages

Packages

Access & Security 4

Authentication, authorization, and security posture

SecurityTxt

SecurityTxt

Security

Security

VulnerabilityDisclosure

VulnerabilityDisclosure

DomainSecurity

DomainSecurity

Operate 4

Status, limits, changes, and where to get help

RateLimits

Support

StatusPage

ChangeLog

Commercial 3

Pricing, plans, and the legal terms of use

Pricing

Pricing

License

License

Plans

Plans

Company 1

The organization behind the API

Website

Website

Other 2

Properties that don't map to a standard resource type

Overlay

Overlay

AgentCard

AgentCard

Source (apis.yml)

aid: pathwren
name: AI Crawler Index
type: Index
description: 'Every AI crawler on the web, what it is for, what blocking it costs you, and the IP ranges its operator publishes
  — as JSON, CSV, robots.txt and regex. 56 crawlers from 30 operators, 1887 IPv4 and 1056 IPv6 prefixes mirrored from 12 operator-published
  endpoints and refreshed every six hours. Read-only static files: no key, no signup, no rate limit, CORS open, CC0.'
accessModel:
  pricing: free
  onboarding: open
  trial: false
  try_now: true
  public: true
  gated: none
  gated_source:
  - onboarding
  label: Free · Open access
  confidence: high
  source:
  - plans
  - authentication
  - rate-limits
  - security
  generated: '2026-11-02'
  method: derived
image: https://www.pathwren.workers.dev/icon.png
url: https://www.pathwren.workers.dev/apis.json
specificationVersion: '1.0'
created: '2026-08-31'
modified: '2026-09-16'
tags:
- AI crawlers
- web crawlers
- Robots.txt
- User Agents
- Bot Detection
- GPTBot
- ClaudeBot
- crawler IP ranges
- llms-txt
- Open Data
- A2A
tags_raw:
- AI crawlers
- web crawlers
- robots.txt
- user agents
- bot detection
- GPTBot
- ClaudeBot
- crawler IP ranges
- llms.txt
- open data
- A2A
maintainers:
- FN: Pathwren
  email: pathwren@tutamail.com
  url: https://www.pathwren.workers.dev/about.html
apis:
- aid: pathwren:pathwren-bulk-api
  name: AI Crawler Index Bulk API
  description: The whole dataset in several shapes.
  humanURL: https://www.pathwren.workers.dev/api.html?s=ai-crawler-index&c=apis-json
  tags:
  - Bulk
  tags_raw:
  - bulk
  properties:
  - type: OpenAPI
    url: openapi/pathwren-bulk-api-openapi.yml
  - type: x-openapi-yaml
    url: https://www.pathwren.workers.dev/openapi.yaml
  - type: Swagger
    url: https://www.pathwren.workers.dev/swagger.json
  - type: Documentation
    url: https://www.pathwren.workers.dev/api.html
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.xml
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.json
  - type: x-mcp-server
    url: https://www.pathwren.workers.dev/mcp
  - type: x-mcp-manifest
    url: https://www.pathwren.workers.dev/.well-known/mcp.json
  - type: x-onboarding
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: x-plugin-manifest
    url: https://www.pathwren.workers.dev/.well-known/ai-plugin.json
  - type: x-api-catalog
    url: https://www.pathwren.workers.dev/.well-known/api-catalog
  - type: x-llms-txt
    url: https://www.pathwren.workers.dev/llms.txt
  - type: x-bulk-data
    url: https://www.pathwren.workers.dev/data/agents.json
  - type: x-ip-ranges
    url: https://www.pathwren.workers.dev/ip-ranges/all.json
  - type: x-security-txt
    url: https://www.pathwren.workers.dev/.well-known/security.txt
  - type: x-sitemap
    url: https://www.pathwren.workers.dev/sitemap.xml
  - type: StatusPage
    url: https://www.pathwren.workers.dev/status.html
  - type: License
    url: https://creativecommons.org/publicdomain/zero/1.0/
  - type: JSONSchema
    url: json-schema/pathwren-crawler.schema.json
  - type: Authentication
    url: authentication/pathwren-authentication.yml
  - type: Conventions
    url: conventions/pathwren-conventions.yml
  - type: ErrorCatalog
    url: errors/pathwren-problem-types.yml
  - type: RateLimits
    url: rate-limits/pathwren-rate-limits.yml
  - type: DataModel
    url: data-model/pathwren-data-model.yml
  - type: ToolCrosswalk
    url: mcp/pathwren-tool-crosswalk.yml
  - type: GettingStarted
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: APIReference
    url: https://www.pathwren.workers.dev/api.html
  - type: ChangeLog
    url: https://www.pathwren.workers.dev/changelog.html
- aid: pathwren:pathwren-crawlers-api
  name: AI Crawler Index Crawlers API
  description: One record per crawler.
  humanURL: https://www.pathwren.workers.dev/api.html?s=ai-crawler-index&c=apis-json
  tags:
  - Crawlers
  tags_raw:
  - crawlers
  properties:
  - type: OpenAPI
    url: openapi/pathwren-crawlers-api-openapi.yml
  - type: x-openapi-yaml
    url: https://www.pathwren.workers.dev/openapi.yaml
  - type: Swagger
    url: https://www.pathwren.workers.dev/swagger.json
  - type: Documentation
    url: https://www.pathwren.workers.dev/api.html
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.xml
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.json
  - type: x-mcp-server
    url: https://www.pathwren.workers.dev/mcp
  - type: x-mcp-manifest
    url: https://www.pathwren.workers.dev/.well-known/mcp.json
  - type: x-onboarding
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: x-plugin-manifest
    url: https://www.pathwren.workers.dev/.well-known/ai-plugin.json
  - type: x-api-catalog
    url: https://www.pathwren.workers.dev/.well-known/api-catalog
  - type: x-llms-txt
    url: https://www.pathwren.workers.dev/llms.txt
  - type: x-bulk-data
    url: https://www.pathwren.workers.dev/data/agents.json
  - type: x-ip-ranges
    url: https://www.pathwren.workers.dev/ip-ranges/all.json
  - type: x-security-txt
    url: https://www.pathwren.workers.dev/.well-known/security.txt
  - type: x-sitemap
    url: https://www.pathwren.workers.dev/sitemap.xml
  - type: StatusPage
    url: https://www.pathwren.workers.dev/status.html
  - type: License
    url: https://creativecommons.org/publicdomain/zero/1.0/
  - type: JSONSchema
    url: json-schema/pathwren-crawler.schema.json
  - type: Authentication
    url: authentication/pathwren-authentication.yml
  - type: Conventions
    url: conventions/pathwren-conventions.yml
  - type: ErrorCatalog
    url: errors/pathwren-problem-types.yml
  - type: RateLimits
    url: rate-limits/pathwren-rate-limits.yml
  - type: DataModel
    url: data-model/pathwren-data-model.yml
  - type: ToolCrosswalk
    url: mcp/pathwren-tool-crosswalk.yml
  - type: GettingStarted
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: APIReference
    url: https://www.pathwren.workers.dev/api.html
  - type: ChangeLog
    url: https://www.pathwren.workers.dev/changelog.html
- aid: pathwren:pathwren-ip-ranges-api
  name: AI Crawler Index Ip Ranges API
  description: Operator-published prefixes, normalised.
  humanURL: https://www.pathwren.workers.dev/api.html?s=ai-crawler-index&c=apis-json
  tags:
  - IP Ranges
  tags_raw:
  - ip-ranges
  properties:
  - type: OpenAPI
    url: openapi/pathwren-ip-ranges-api-openapi.yml
  - type: x-openapi-yaml
    url: https://www.pathwren.workers.dev/openapi.yaml
  - type: Swagger
    url: https://www.pathwren.workers.dev/swagger.json
  - type: Documentation
    url: https://www.pathwren.workers.dev/api.html
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.xml
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.json
  - type: x-mcp-server
    url: https://www.pathwren.workers.dev/mcp
  - type: x-mcp-manifest
    url: https://www.pathwren.workers.dev/.well-known/mcp.json
  - type: x-onboarding
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: x-plugin-manifest
    url: https://www.pathwren.workers.dev/.well-known/ai-plugin.json
  - type: x-api-catalog
    url: https://www.pathwren.workers.dev/.well-known/api-catalog
  - type: x-llms-txt
    url: https://www.pathwren.workers.dev/llms.txt
  - type: x-bulk-data
    url: https://www.pathwren.workers.dev/data/agents.json
  - type: x-ip-ranges
    url: https://www.pathwren.workers.dev/ip-ranges/all.json
  - type: x-security-txt
    url: https://www.pathwren.workers.dev/.well-known/security.txt
  - type: x-sitemap
    url: https://www.pathwren.workers.dev/sitemap.xml
  - type: StatusPage
    url: https://www.pathwren.workers.dev/status.html
  - type: License
    url: https://creativecommons.org/publicdomain/zero/1.0/
  - type: JSONSchema
    url: json-schema/pathwren-crawler.schema.json
  - type: Authentication
    url: authentication/pathwren-authentication.yml
  - type: Conventions
    url: conventions/pathwren-conventions.yml
  - type: ErrorCatalog
    url: errors/pathwren-problem-types.yml
  - type: RateLimits
    url: rate-limits/pathwren-rate-limits.yml
  - type: DataModel
    url: data-model/pathwren-data-model.yml
  - type: ToolCrosswalk
    url: mcp/pathwren-tool-crosswalk.yml
  - type: GettingStarted
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: APIReference
    url: https://www.pathwren.workers.dev/api.html
  - type: ChangeLog
    url: https://www.pathwren.workers.dev/changelog.html
- aid: pathwren:pathwren-robots-api
  name: AI Crawler Index Robots API
  description: Ready-made robots.txt policy files.
  humanURL: https://www.pathwren.workers.dev/api.html?s=ai-crawler-index&c=apis-json
  tags:
  - Robots
  tags_raw:
  - robots
  properties:
  - type: OpenAPI
    url: openapi/pathwren-robots-api-openapi.yml
  - type: x-openapi-yaml
    url: https://www.pathwren.workers.dev/openapi.yaml
  - type: Swagger
    url: https://www.pathwren.workers.dev/swagger.json
  - type: Documentation
    url: https://www.pathwren.workers.dev/api.html
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.xml
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.json
  - type: x-mcp-server
    url: https://www.pathwren.workers.dev/mcp
  - type: x-mcp-manifest
    url: https://www.pathwren.workers.dev/.well-known/mcp.json
  - type: x-onboarding
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: x-plugin-manifest
    url: https://www.pathwren.workers.dev/.well-known/ai-plugin.json
  - type: x-api-catalog
    url: https://www.pathwren.workers.dev/.well-known/api-catalog
  - type: x-llms-txt
    url: https://www.pathwren.workers.dev/llms.txt
  - type: x-bulk-data
    url: https://www.pathwren.workers.dev/data/agents.json
  - type: x-ip-ranges
    url: https://www.pathwren.workers.dev/ip-ranges/all.json
  - type: x-security-txt
    url: https://www.pathwren.workers.dev/.well-known/security.txt
  - type: x-sitemap
    url: https://www.pathwren.workers.dev/sitemap.xml
  - type: StatusPage
    url: https://www.pathwren.workers.dev/status.html
  - type: License
    url: https://creativecommons.org/publicdomain/zero/1.0/
  - type: JSONSchema
    url: json-schema/pathwren-crawler.schema.json
  - type: Authentication
    url: authentication/pathwren-authentication.yml
  - type: Conventions
    url: conventions/pathwren-conventions.yml
  - type: ErrorCatalog
    url: errors/pathwren-problem-types.yml
  - type: RateLimits
    url: rate-limits/pathwren-rate-limits.yml
  - type: DataModel
    url: data-model/pathwren-data-model.yml
  - type: ToolCrosswalk
    url: mcp/pathwren-tool-crosswalk.yml
  - type: GettingStarted
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: APIReference
    url: https://www.pathwren.workers.dev/api.html
  - type: ChangeLog
    url: https://www.pathwren.workers.dev/changelog.html
- aid: pathwren:pathwren-status-api
  name: AI Crawler Index Status API
  description: Freshness of the upstream sources.
  humanURL: https://www.pathwren.workers.dev/api.html?s=ai-crawler-index&c=apis-json
  tags:
  - Status
  tags_raw:
  - status
  properties:
  - type: OpenAPI
    url: openapi/pathwren-status-api-openapi.yml
  - type: x-openapi-yaml
    url: https://www.pathwren.workers.dev/openapi.yaml
  - type: Swagger
    url: https://www.pathwren.workers.dev/swagger.json
  - type: Documentation
    url: https://www.pathwren.workers.dev/api.html
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.xml
  - type: Feed
    url: https://www.pathwren.workers.dev/feed.json
  - type: x-mcp-server
    url: https://www.pathwren.workers.dev/mcp
  - type: x-mcp-manifest
    url: https://www.pathwren.workers.dev/.well-known/mcp.json
  - type: x-onboarding
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: x-plugin-manifest
    url: https://www.pathwren.workers.dev/.well-known/ai-plugin.json
  - type: x-api-catalog
    url: https://www.pathwren.workers.dev/.well-known/api-catalog
  - type: x-llms-txt
    url: https://www.pathwren.workers.dev/llms.txt
  - type: x-bulk-data
    url: https://www.pathwren.workers.dev/data/agents.json
  - type: x-ip-ranges
    url: https://www.pathwren.workers.dev/ip-ranges/all.json
  - type: x-security-txt
    url: https://www.pathwren.workers.dev/.well-known/security.txt
  - type: x-sitemap
    url: https://www.pathwren.workers.dev/sitemap.xml
  - type: StatusPage
    url: https://www.pathwren.workers.dev/status.html
  - type: License
    url: https://creativecommons.org/publicdomain/zero/1.0/
  - type: JSONSchema
    url: json-schema/pathwren-crawler.schema.json
  - type: Authentication
    url: authentication/pathwren-authentication.yml
  - type: Conventions
    url: conventions/pathwren-conventions.yml
  - type: ErrorCatalog
    url: errors/pathwren-problem-types.yml
  - type: RateLimits
    url: rate-limits/pathwren-rate-limits.yml
  - type: DataModel
    url: data-model/pathwren-data-model.yml
  - type: ToolCrosswalk
    url: mcp/pathwren-tool-crosswalk.yml
  - type: GettingStarted
    url: https://www.pathwren.workers.dev/.well-known/api-onboarding
  - type: APIReference
    url: https://www.pathwren.workers.dev/api.html
  - type: ChangeLog
    url: https://www.pathwren.workers.dev/changelog.html
common:
- type: RateLimits
  url: rate-limits/pathwren-rate-limits.yml
- type: Pricing
  url: https://www.pathwren.workers.dev/.well-known/x402
- type: Overlay
  url: overlays/pathwren-ai-crawler-index-overlay.yaml
- type: Website
  url: https://pathwren.workers.dev
- type: DeveloperPortal
  url: https://www.pathwren.workers.dev/
- type: Documentation
  url: https://www.pathwren.workers.dev/api.html
- type: Support
  url: https://www.pathwren.workers.dev/about.html
- type: License
  url: https://creativecommons.org/publicdomain/zero/1.0/
- type: StatusPage
  url: https://www.pathwren.workers.dev/status.html
- type: ChangeLog
  url: changelog/pathwren-changelog.yml
- type: WellKnown
  url: well-known/pathwren-well-known.yml
- type: SecurityTxt
  url: well-known/pathwren-security.txt
- type: Security
  url: security/pathwren-vulnerability-disclosure.yml
- type: VulnerabilityDisclosure
  url: security/pathwren-vulnerability-disclosure.yml
- type: DomainSecurity
  url: security/pathwren-domain-security.yml
- type: LLMsTxt
  url: llms/pathwren-llms.txt
- type: MCPServer
  url: mcp/pathwren-mcp.yml
- type: AgentCard
  url: a2a/pathwren-a2a.yml
- type: AgentSkill
  url: skills/_index.yml
- type: Packages
  url: packages/pathwren-packages.yml
- type: Conformance
  url: conformance/pathwren-conformance.yml
- type: Lifecycle
  url: lifecycle/pathwren-lifecycle.yml
- type: Plans
  url: plans/pathwren-plans-pricing.yml
x-enrichment:
  date: '2026-09-01'
  status: enriched
  artifacts_added: 38
  pass: local-v1
x-a2a:
  verified: true
  probed: '2026-09-26'
  url: https://www.pathwren.workers.dev/.well-known/agent-card.json
  grade: conformant
  protocol_version: '1.0'
  control: 404

AI Crawler Index API — frequently asked questions

Does AI Crawler Index have an API?

Yes. APIs.io lists 5 APIs from AI Crawler Index, including AI Crawler Index Bulk API, AI Crawler Index Crawlers API and AI Crawler Index Ip Ranges API, and 2 more.

Is there an OpenAPI specification for the AI Crawler Index API?

Yes. APIs.io indexes 5 OpenAPI specifications for AI Crawler Index, each linked for download from this page.

How do I authenticate with the AI Crawler Index API?

AI Crawler Index uses bearer-token authentication (0 schemes). The details are profiled on APIs.io as Pathwren Authentication.

How much does the AI Crawler Index API cost?

APIs.io lists 1 pricing plan for AI Crawler Index, with each plan's price, limits and inclusions under Pricing Plans on this page.

Does AI Crawler Index have an MCP server?

Yes. APIs.io lists 1 MCP server for AI Crawler Index: Remote MCP server at www.pathwren.workers.dev over streamable HTTP; 9 tools listed.

Is the AI Crawler Index API ready for AI agents?

AI Crawler Index has a Kin Score of 47.0 out of 100 (developing) and an Agent Readiness score of 47.0 (agent-ready), scored by API Evangelist on 2026-11-02.

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/pathwren"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/pathwren/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/pathwren/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.