Crawl4AI is an open-source, Apache-2.0 web crawler and scraper built to turn any URL into clean, LLM-ready data — Markdown, typed JSON, screenshots, PDFs, or a map of every URL on a domain. Operated by CONTEXT4AI PTE LTD of Singapore and created by Hossein Tohidi (@unclecode), the project pairs a 79,000-star Python library and self-hostable Docker API server with a hosted Cloud API at gate.crawl4ai.com that serves scrape, search, answer, extract and bulk-job endpoints from three regions, plus a live remote MCP server that exposes the same capabilities to agents as five native tools. Self-hosting is free and unmetered; the hosted tier is free at 5,000 searches and 1,000 scrapes a month.
Crawl4AI publishes 4 API contracts indexed on the APIs.io network, including Crawl API, and 3 more. Tagged areas include AI Automation, Web Crawling, Web Scraping, Data Extraction, and Search.
The Crawl4AI catalog on APIs.io includes 1 event-driven AsyncAPI specification and 1 Spectral governance ruleset.
Crawl4AI’s developer surface includes documentation, API reference, getting-started guide, engineering blog, support, pricing, signup flow, and 35 more developer resources.
The questions people put to an LLM, and the jobs they hand an agent, that Crawl4AI's APIs answer — each one linked to the operation that does it.
Generated by API Evangelist
Regulatory Posture applies to this provider. Its tags matched the
Horizontal (data, software, accessibility, platform) regime, so
Regulatory Posture carries 15 points of the composite.
If this regime is wrong for your business, say so on your
provider repo — the
applicability map is public and we will correct it.
Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it
carries 10 points of the composite. It is scored from the published contracts
themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already
holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in
front of it, and the first time that check is skipped a duplicate record is created.
Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet,
not penalised by it.
The six quality facets above are damped to 75 points between them,
because the conditional facet above carries the other
25. That is why each facet's contribution is shown against a damped
maximum: raising a quality facet moves the composite by 75% of its nominal
weight, not 100%. The full arithmetic is at apis.io/rating/.
Show this score on your site.
It updates itself when the score changes. Paste this where you link to your API docs:
<a href="https://apis.io/providers/crawl4ai/"><img src="https://apis.io/badge/crawl4ai/card.svg" alt="Crawl4AI on APIs.io — Kin Score by API Evangelist"></a>
Other styles, a README pill and a shields.io endpoint are on the badge page.
Raise your score.
Agent readiness — checks not met, largest first (exact, from this profile):
Idempotency+9 pts — OpenAPI mutating operations declare an idempotency key — a header, a request-body property, or a named de-duplication token whose own description says so (verified); else common[].type matches /idempoten/i (documented)
A2A Agent Card+8 pts — a2a/-a2a.yml manifest, graded on conformance.grade against A2A 1.0.0
Request/Response Examples+7 pts — share of operations carrying request/response examples; else an Examples / Code Samples link
Delegated User Identity+6 pts — a served discovery document listing authorization_code in grant_types_supported (served), else an oauth2 authorizationCode flow or openIdConnect declared in an OpenAPI (documented)
6 more unmet checks in the Agent Readiness table above.
Kin Score — facets with the most composite points still available (weight × headroom):
Standards this provider implements or carries. De facto ones are profiled and measured by the API Commons; formal ones come from the standards directory and are linked by the provider's own tags. The evidence column says how each claim was established — not that it was made.
The hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clean Markdown or HTML, GET /search runs a browser-free multi-engine web s...
The versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POST /v1/markdown, /v1/screenshot (screenshot and PDF capture), /v1/extra...
The Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image, serving on port 11235 on infrastructure the operator runs. Exposes /cra...
Crawl4AI ships a hosted, streamable-HTTP MCP server at https://gate.crawl4ai.com/mcp that exposes the Cloud API as five native agent tools. tools/list answers ANONYMOUSLY — the ...
Overlays applied on top of this provider's contracts. Each card says who wrote it: a document the provider publishes, or one API Evangelist derived or generated.
aid: crawl4ai
name: Crawl4AI
description: Crawl4AI is an open-source, Apache-2.0 web crawler and scraper built to turn any URL into clean, LLM-ready data
— Markdown, typed JSON, screenshots, PDFs, or a map of every URL on a domain. Operated by CONTEXT4AI PTE LTD of Singapore
and created by Hossein Tohidi (@unclecode), the project pairs a 79,000-star Python library and self-hostable Docker API
server with a hosted Cloud API at gate.crawl4ai.com that serves scrape, search, answer, extract and bulk-job endpoints from
three regions, plus a live remote MCP server that exposes the same capabilities to agents as five native tools. Self-hosting
is free and unmetered; the hosted tier is free at 5,000 searches and 1,000 scrapes a month.
type: Index
deliveryModel:
model: open-source-and-hosted
open_source: true
commercial: true
callable_host: true
label: Apache-2.0 open source, plus a hosted commercial API
confidence: high
source:
- https://pypi.org/project/crawl4ai/
- https://gate.crawl4ai.com/
- https://gate.crawl4ai.com/legal/#terms
generated: '2026-08-29'
method: searched
accessModel:
pricing: freemium
onboarding: self-service
trial: true
try_now: true
public: true
label: Freemium — instant 24-hour key, no signup; $7/mo Supporter tier
confidence: high
source:
- https://gate.crawl4ai.com/ (#pricing)
- https://gate.crawl4ai.com/llms.txt
generated: '2026-08-29'
method: searched
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/crawl4ai.png
x-docs-platform: mkdocs
x-docs-platform-confidence: medium
x-docs-platform-checked: 2026-09-24
tags:
- AI Automation
- Web Crawling
- Web Scraping
- Data Extraction
- Search
- LLM Tooling
- Agents
- MCP
- Open Source
tags_raw:
- AI Automation
- Web Crawling
- Web Scraping
- Data Extraction
- Search
- LLM Tooling
- Agents
- MCP
- Open Source
- Open-Source
url: https://raw.githubusercontent.com/api-evangelist/crawl4ai/refs/heads/main/apis.yml
created: '2026-03-27'
modified: '2026-09-16'
specificationVersion: '0.23'
apis:
- aid: crawl4ai:crawl4ai
name: Crawl4AI Cloud API
description: 'The hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clean
Markdown or HTML, GET /search runs a browser-free multi-engine web search, GET /answer returns a direct answer with sources
(experimental), POST /extract pulls typed JSON out of a page with an instruction and/or JSON Schema, POST /scrape/batch
streams up to 50 URLs as NDJSON, and POST /scrape/jobs drains lists of up to 10,000 URLs in the background. Requests route
to the nearest of three regions (EU, Singapore, US). No OpenAPI is published for this surface.'
humanURL: https://gate.crawl4ai.com/docs
baseURL: https://gate.crawl4ai.com
tags:
- AI Automation
- Web Crawling
- Search
properties:
- type: Documentation
url: https://gate.crawl4ai.com/docs
- type: APIReference
url: https://gate.crawl4ai.com/docs
- type: LLMsTxt
url: llms/crawl4ai-llms.txt
- type: MCPServer
url: mcp/crawl4ai-mcp.yml
- type: ToolCrosswalk
url: mcp/crawl4ai-tool-crosswalk.yml
- aid: crawl4ai:crawl4ai-v1
name: Crawl4AI Cloud v1 API
description: The versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POST
/v1/markdown, /v1/screenshot (screenshot and PDF capture), /v1/extract (auto, LLM or reusable CSS schema), /v1/map (domain
URL discovery with BM25 relevance scoring), /v1/crawl/site (whole-site crawl up to 1,000 pages) and /v1/crawl (the full
40-field CrawlerRunConfig), each with an async variant, a job API and webhook callbacks. Authenticated with X-API-Key.
No OpenAPI is published; the reference lives in the provider's own agent skill.
humanURL: https://github.com/unclecode/crawl4ai-cloud-sdk/blob/main/python/claude-plugin/skills/crawl4ai/reference/endpoints.md
baseURL: https://api.crawl4ai.com
tags:
- AI Automation
- Web Crawling
- Data Extraction
properties:
- type: APIReference
url: skills/reference/crawl4ai-endpoints.md
- type: ErrorCatalog
url: skills/reference/crawl4ai-errors.md
- type: SourceCode
url: https://github.com/unclecode/crawl4ai-cloud-sdk
- aid: crawl4ai:crawl4ai-docker
name: Crawl4AI Self-Hosted API
description: 'The Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image,
serving on port 11235 on infrastructure the operator runs. Exposes /crawl, /crawl/stream, /crawl/job, /md, /llm, /screenshot,
/pdf, /html, /token, /health, /artifacts/{id}, /hooks/info, a monitoring API and an MCP bridge at /mcp/sse, /mcp/ws and
/mcp/schema. Since 0.9.0 it is secure-by-default: authentication on, loopback bind without a token, and a strict request
trust boundary.'
humanURL: https://docs.crawl4ai.com/core/self-hosting/
baseURL: http://{host}:11235
baseURLNote: Templated on purpose — this server runs on the customer's own host. Crawl4AI operates no instance of it.
tags:
- AI Automation
- Web Crawling
- Open Source
tags_raw:
- AI Automation
- Web Crawling
- Open Source
- Open-Source
properties:
- type: Documentation
url: https://docs.crawl4ai.com/core/self-hosting/
- type: SourceCode
url: https://github.com/unclecode/crawl4ai/tree/main/deploy/docker
- type: ContainerImage
url: https://hub.docker.com/r/unclecode/crawl4ai
- aid: crawl4ai:crawl4ai-crawl-api
name: Crawl4AI Crawl API
description: The Crawl API from Crawl4AI — 2 operation(s) for crawl.
humanURL: https://gate.crawl4ai.com/docs
baseURL: https://gate.crawl4ai.com
tags:
- Crawl
properties:
- type: OpenAPI
url: openapi/crawl4ai-crawl-api-openapi.yml
common:
- type: License
name: Apache-2.0
url: https://github.com/unclecode/crawl4ai-cloud-sdk/blob/main/LICENSE
- type: Website
url: https://crawl4ai.com
- type: DeveloperPortal
url: https://gate.crawl4ai.com/
- type: Documentation
url: https://docs.crawl4ai.com
- type: APIReference
url: https://gate.crawl4ai.com/docs
- type: GettingStarted
url: https://docs.crawl4ai.com/core/quickstart/
- type: Blog
url: https://docs.crawl4ai.com/blog/
- type: Support
url: https://discord.gg/jP8KfhDhyN
- type: GitHubOrganization
url: https://github.com/unclecode
- type: SourceCode
url: https://github.com/unclecode/crawl4ai
- type: Roadmap
url: https://github.com/unclecode/crawl4ai/blob/main/ROADMAP.md
- type: LinkedIn
url: https://www.linkedin.com/company/crawl4ai
- type: Integrations
url: https://docs.crawl4ai.com/marketplace/
- type: Pricing
url: https://gate.crawl4ai.com/#pricing
- type: SignUp
url: https://gate.crawl4ai.com/dashboard/
- type: TermsOfService
url: https://gate.crawl4ai.com/legal/#terms
- type: PrivacyPolicy
url: https://gate.crawl4ai.com/legal/#privacy
- type: StatusPage
url: https://unclecode.github.io/crawl4ai-status/
- type: Plans
url: plans/crawl4ai-plans-pricing.yml
- type: RateLimits
url: rate-limits/crawl4ai-rate-limits.yml
- type: FinOps
url: finops/crawl4ai-finops.yml
- type: OpenAPI
url: openapi/_original/crawl4ai-platform-gateway-openapi.json
- type: Overlay
url: overlays/crawl4ai-platform-gateway-overlay.yaml
- type: LLMsTxt
url: llms/crawl4ai-llms.txt
- type: MCPServer
url: mcp/crawl4ai-mcp.yml
- type: ToolCrosswalk
url: mcp/crawl4ai-tool-crosswalk.yml
- type: AgentSkill
url: skills/_index.yml
- type: Packages
url: packages/crawl4ai-packages.yml
- type: SDKs
url: packages/crawl4ai-packages.yml
- type: CLI
url: cli/crawl4ai-cli.yml
- type: Authentication
url: authentication/crawl4ai-authentication.yml
- type: ErrorCatalog
url: errors/crawl4ai-problem-types.yml
- type: Conventions
url: conventions/crawl4ai-conventions.yml
- type: Conformance
url: conformance/crawl4ai-conformance.yml
- type: Lifecycle
url: lifecycle/crawl4ai-lifecycle.yml
- type: Deprecation
url: lifecycle/crawl4ai-lifecycle.yml
- type: ChangeLog
url: changelog/crawl4ai-changelog.yml
- type: Webhooks
url: asyncapi/crawl4ai-webhooks.yml
- type: Sandbox
url: sandbox/crawl4ai-sandbox.yml
- type: DataModel
url: data-model/crawl4ai-data-model.yml
- type: DomainSecurity
url: security/crawl4ai-domain-security.yml
- type: VulnerabilityDisclosure
url: security/crawl4ai-vulnerability-disclosure.yml
- type: Security
url: security/crawl4ai-vulnerability-disclosure.yml
integrations:
- name: Crawl4AI
maintainers:
- FN: Kin Lane
email: kin@apievangelist.com
x-enrichment:
date: '2026-08-29'
status: enriched
artifacts_added: 27
pass: local-v1
Crawl4AI API — frequently asked questions
Does Crawl4AI have an API?
Yes. APIs.io lists 4 APIs from Crawl4AI, including Crawl4AI Cloud API, Crawl4AI Cloud v1 API and Crawl4AI Self-Hosted API, and 1 more.
Is there an OpenAPI specification for the Crawl4AI API?
Yes. APIs.io indexes 1 OpenAPI specification for Crawl4AI, each linked for download from this page.
How do I authenticate with the Crawl4AI API?
Crawl4AI uses bearer-token authentication (3 schemes). The details are profiled on APIs.io as Crawl4Ai Authentication.
How much does the Crawl4AI API cost?
Crawl4AI publishes its prices; its Free plan is $0 · no card as of 2026-10-28. APIs.io lists 5 pricing plans for Crawl4AI, with each plan's price, limits and inclusions under Pricing on this page.
Does Crawl4AI have an MCP server?
Yes. APIs.io lists 1 MCP server for Crawl4AI: Crawl4AI ships a hosted, streamable-HTTP MCP server at https://gate.crawl4ai.com/mcp that exposes the Cloud API as five native agent tools. tools/list answers ANONYMOUSLY — the full tool set with comp
Is the Crawl4AI API ready for AI agents?
Crawl4AI has a Kin Score of 70.3 out of 100 (exemplar) and an Agent Readiness score of 38.7 (agent-ready), scored by API Evangelist on 2026-11-06.
Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.
Get an API key
Free tier, no form to fill in. Signing in shares your email address with us — we
store it to create your key and to recognise you if you sign in with another
provider. See our Privacy Policy and
Terms.