Crawl4AI is an open-source, Apache-2.0 web crawler and scraper built to turn any URL into clean, LLM-ready data — Markdown, typed JSON, screenshots, PDFs, or a map of every URL on a domain. Operated by CONTEXT4AI PTE LTD of Singapore and created by Hossein Tohidi (@unclecode), the project pairs a 79,000-star Python library and self-hostable Docker API server with a hosted Cloud API at gate.crawl4ai.com that serves scrape, search, answer, extract and bulk-job endpoints from three regions, plus a live remote MCP server that exposes the same capabilities to agents as five native tools. Self-hosting is free and unmetered; the hosted tier is free at 5,000 searches and 1,000 scrapes a month.
Crawl4AI publishes 4 APIs on the APIs.io network, including Crawl API, and 3 more. Tagged areas include AI Automation, Web Crawling, Web Scraping, Data Extraction, and Search.
The Crawl4AI catalog on APIs.io includes 1 event-driven AsyncAPI specification.
Crawl4AI’s developer surface includes documentation, API reference, getting-started guide, engineering blog, support, pricing, signup flow, and 35 more developer resources.
Regulatory Posture applies to this provider. Its tags matched the
Horizontal (data, software, accessibility, platform) regime, so
Regulatory Posture carries 15 points of the composite.
If this regime is wrong for your business, say so on your
provider repo — the
applicability map is public and we will correct it.
Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it
carries 10 points of the composite. It is scored from the published contracts
themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already
holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in
front of it, and the first time that check is skipped a duplicate record is created.
Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet,
not penalised by it.
The six quality facets above are damped to 75 points between them,
because the conditional facet above carries the other
25. That is why each facet's contribution is shown against a damped
maximum: raising a quality facet moves the composite by 75% of its nominal
weight, not 100%. The full arithmetic is at apis.io/rating/.
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from github.com/api-evangelist/crawl4ai: open an issue to ask a question, or submit a pull request to add artifacts.
Submit an artifact on GitHub — free →Manage your own listing — the Influence plan, $499/mo →
The hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clean Markdown or HTML, GET /search runs a browser-free multi-engine web s...
The versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POST /v1/markdown, /v1/screenshot (screenshot and PDF capture), /v1/extra...
The Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image, serving on port 11235 on infrastructure the operator runs. Exposes /cra...
Crawl4AI ships a hosted, streamable-HTTP MCP server at https://gate.crawl4ai.com/mcp that exposes the Cloud API as five native agent tools. tools/list answers ANONYMOUSLY — the ...
aid: crawl4ai
name: Crawl4AI
description: Crawl4AI is an open-source, Apache-2.0 web crawler and scraper built to turn any URL into clean, LLM-ready data
— Markdown, typed JSON, screenshots, PDFs, or a map of every URL on a domain. Operated by CONTEXT4AI PTE LTD of Singapore
and created by Hossein Tohidi (@unclecode), the project pairs a 79,000-star Python library and self-hostable Docker API
server with a hosted Cloud API at gate.crawl4ai.com that serves scrape, search, answer, extract and bulk-job endpoints from
three regions, plus a live remote MCP server that exposes the same capabilities to agents as five native tools. Self-hosting
is free and unmetered; the hosted tier is free at 5,000 searches and 1,000 scrapes a month.
type: Index
deliveryModel:
model: open-source-and-hosted
open_source: true
commercial: true
callable_host: true
label: Apache-2.0 open source, plus a hosted commercial API
confidence: high
source:
- https://pypi.org/project/crawl4ai/
- https://gate.crawl4ai.com/
- https://gate.crawl4ai.com/legal/#terms
generated: '2026-08-29'
method: searched
accessModel:
pricing: freemium
onboarding: self-service
trial: true
try_now: true
public: true
label: Freemium — instant 24-hour key, no signup; $7/mo Supporter tier
confidence: high
source:
- https://gate.crawl4ai.com/ (#pricing)
- https://gate.crawl4ai.com/llms.txt
generated: '2026-08-29'
method: searched
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/crawl4ai.png
x-docs-platform: mkdocs
x-docs-platform-confidence: medium
x-docs-platform-checked: 2026-09-24
tags:
- AI Automation
- Web Crawling
- Web Scraping
- Data Extraction
- Search
- LLM Tooling
- Agents
- MCP
- Open Source
tags_raw:
- AI Automation
- Web Crawling
- Web Scraping
- Data Extraction
- Search
- LLM Tooling
- Agents
- MCP
- Open Source
- Open-Source
url: https://raw.githubusercontent.com/api-evangelist/crawl4ai/refs/heads/main/apis.yml
created: '2026-03-27'
modified: '2026-09-16'
specificationVersion: '0.23'
apis:
- aid: crawl4ai:crawl4ai
name: Crawl4AI Cloud API
description: 'The hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clean
Markdown or HTML, GET /search runs a browser-free multi-engine web search, GET /answer returns a direct answer with sources
(experimental), POST /extract pulls typed JSON out of a page with an instruction and/or JSON Schema, POST /scrape/batch
streams up to 50 URLs as NDJSON, and POST /scrape/jobs drains lists of up to 10,000 URLs in the background. Requests route
to the nearest of three regions (EU, Singapore, US). No OpenAPI is published for this surface.'
humanURL: https://gate.crawl4ai.com/docs
baseURL: https://gate.crawl4ai.com
tags:
- AI Automation
- Web Crawling
- Search
properties:
- type: Documentation
url: https://gate.crawl4ai.com/docs
- type: APIReference
url: https://gate.crawl4ai.com/docs
- type: LLMsTxt
url: llms/crawl4ai-llms.txt
- type: MCPServer
url: mcp/crawl4ai-mcp.yml
- type: ToolCrosswalk
url: mcp/crawl4ai-tool-crosswalk.yml
- aid: crawl4ai:crawl4ai-v1
name: Crawl4AI Cloud v1 API
description: The versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POST
/v1/markdown, /v1/screenshot (screenshot and PDF capture), /v1/extract (auto, LLM or reusable CSS schema), /v1/map (domain
URL discovery with BM25 relevance scoring), /v1/crawl/site (whole-site crawl up to 1,000 pages) and /v1/crawl (the full
40-field CrawlerRunConfig), each with an async variant, a job API and webhook callbacks. Authenticated with X-API-Key.
No OpenAPI is published; the reference lives in the provider's own agent skill.
humanURL: https://github.com/unclecode/crawl4ai-cloud-sdk/blob/main/python/claude-plugin/skills/crawl4ai/reference/endpoints.md
baseURL: https://api.crawl4ai.com
tags:
- AI Automation
- Web Crawling
- Data Extraction
properties:
- type: APIReference
url: skills/reference/crawl4ai-endpoints.md
- type: ErrorCatalog
url: skills/reference/crawl4ai-errors.md
- type: SourceCode
url: https://github.com/unclecode/crawl4ai-cloud-sdk
- aid: crawl4ai:crawl4ai-docker
name: Crawl4AI Self-Hosted API
description: 'The Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image,
serving on port 11235 on infrastructure the operator runs. Exposes /crawl, /crawl/stream, /crawl/job, /md, /llm, /screenshot,
/pdf, /html, /token, /health, /artifacts/{id}, /hooks/info, a monitoring API and an MCP bridge at /mcp/sse, /mcp/ws and
/mcp/schema. Since 0.9.0 it is secure-by-default: authentication on, loopback bind without a token, and a strict request
trust boundary.'
humanURL: https://docs.crawl4ai.com/core/self-hosting/
baseURL: http://{host}:11235
baseURLNote: Templated on purpose — this server runs on the customer's own host. Crawl4AI operates no instance of it.
tags:
- AI Automation
- Web Crawling
- Open Source
tags_raw:
- AI Automation
- Web Crawling
- Open Source
- Open-Source
properties:
- type: Documentation
url: https://docs.crawl4ai.com/core/self-hosting/
- type: SourceCode
url: https://github.com/unclecode/crawl4ai/tree/main/deploy/docker
- type: ContainerImage
url: https://hub.docker.com/r/unclecode/crawl4ai
- aid: crawl4ai:crawl4ai-crawl-api
name: Crawl4AI Crawl API
description: The Crawl API from Crawl4AI — 2 operation(s) for crawl.
humanURL: https://gate.crawl4ai.com/docs
baseURL: https://gate.crawl4ai.com
tags:
- Crawl
properties:
- type: OpenAPI
url: openapi/crawl4ai-crawl-api-openapi.yml
common:
- type: License
name: Apache-2.0
url: https://github.com/unclecode/crawl4ai-cloud-sdk/blob/main/LICENSE
- type: Website
url: https://crawl4ai.com
- type: DeveloperPortal
url: https://gate.crawl4ai.com/
- type: Documentation
url: https://docs.crawl4ai.com
- type: APIReference
url: https://gate.crawl4ai.com/docs
- type: GettingStarted
url: https://docs.crawl4ai.com/core/quickstart/
- type: Blog
url: https://docs.crawl4ai.com/blog/
- type: Support
url: https://discord.gg/jP8KfhDhyN
- type: GitHubOrganization
url: https://github.com/unclecode
- type: SourceCode
url: https://github.com/unclecode/crawl4ai
- type: Roadmap
url: https://github.com/unclecode/crawl4ai/blob/main/ROADMAP.md
- type: LinkedIn
url: https://www.linkedin.com/company/crawl4ai
- type: Integrations
url: https://docs.crawl4ai.com/marketplace/
- type: Pricing
url: https://gate.crawl4ai.com/#pricing
- type: SignUp
url: https://gate.crawl4ai.com/dashboard/
- type: TermsOfService
url: https://gate.crawl4ai.com/legal/#terms
- type: PrivacyPolicy
url: https://gate.crawl4ai.com/legal/#privacy
- type: StatusPage
url: https://unclecode.github.io/crawl4ai-status/
- type: Plans
url: plans/crawl4ai-plans-pricing.yml
- type: RateLimits
url: rate-limits/crawl4ai-rate-limits.yml
- type: FinOps
url: finops/crawl4ai-finops.yml
- type: OpenAPI
url: openapi/_original/crawl4ai-platform-gateway-openapi.json
- type: Overlay
url: overlays/crawl4ai-platform-gateway-overlay.yaml
- type: LLMsTxt
url: llms/crawl4ai-llms.txt
- type: MCPServer
url: mcp/crawl4ai-mcp.yml
- type: ToolCrosswalk
url: mcp/crawl4ai-tool-crosswalk.yml
- type: AgentSkill
url: skills/_index.yml
- type: Packages
url: packages/crawl4ai-packages.yml
- type: SDKs
url: packages/crawl4ai-packages.yml
- type: CLI
url: cli/crawl4ai-cli.yml
- type: Authentication
url: authentication/crawl4ai-authentication.yml
- type: ErrorCatalog
url: errors/crawl4ai-problem-types.yml
- type: Conventions
url: conventions/crawl4ai-conventions.yml
- type: Conformance
url: conformance/crawl4ai-conformance.yml
- type: Lifecycle
url: lifecycle/crawl4ai-lifecycle.yml
- type: Deprecation
url: lifecycle/crawl4ai-lifecycle.yml
- type: ChangeLog
url: changelog/crawl4ai-changelog.yml
- type: Webhooks
url: asyncapi/crawl4ai-webhooks.yml
- type: Sandbox
url: sandbox/crawl4ai-sandbox.yml
- type: DataModel
url: data-model/crawl4ai-data-model.yml
- type: DomainSecurity
url: security/crawl4ai-domain-security.yml
- type: VulnerabilityDisclosure
url: security/crawl4ai-vulnerability-disclosure.yml
- type: Security
url: security/crawl4ai-vulnerability-disclosure.yml
integrations:
- name: Crawl4AI
maintainers:
- FN: Kin Lane
email: kin@apievangelist.com
x-enrichment:
date: '2026-08-29'
status: enriched
artifacts_added: 27
pass: local-v1
Discovery needs no key. Ratings and market analysis are Pro.
Get an API key
Free tier, no form to fill in. Signing in shares your email address with us — we
store it to create your key and to recognise you if you sign in with another
provider. See our Privacy Policy and
Terms.
Kin Score
How this is scored →