Crawl4AI is an open-source, Apache-2.0 web crawler and scraper built to turn any URL into clean, LLM-ready data — Markdown, typed JSON, screenshots, PDFs, or a map of every URL on a domain. Operated by CONTEXT4AI PTE LTD of Singapore and created by Hossein Tohidi (@unclecode), the project pairs a 79,000-star Python library and self-hostable Docker API server with a hosted Cloud API at gate.crawl4ai.com that serves scrape, search, answer, extract and bulk-job endpoints from three regions, plus a live remote MCP server that exposes the same capabilities to agents as five native tools. Self-hosting is free and unmetered; the hosted tier is free at 5,000 searches and 1,000 scrapes a month.
Crawl4AI publishes 3 APIs on the APIs.io network. Tagged areas include AI Automation, Web Crawling, Web Scraping, Data Extraction, and Search.
The Crawl4AI catalog on APIs.io includes 1 event-driven AsyncAPI specification.
Crawl4AI’s developer surface includes documentation, API reference, getting-started guide, engineering blog, support, pricing, signup flow, and 35 more developer resources.
Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it
carries 10 points of the composite. It is scored from the published contracts
themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already
holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in
front of it, and the first time that check is skipped a duplicate record is created.
Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet,
not penalised by it.
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from github.com/api-evangelist/crawl4ai: open an issue to ask a question, or submit a pull request to add artifacts.
Submit an artifact on GitHub — free →Manage your own listing — the Influence plan, $499/mo →
The hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clean Markdown or HTML, GET /search runs a browser-free multi-engine web s...
The versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POST /v1/markdown, /v1/screenshot (screenshot and PDF capture), /v1/extra...
The Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image, serving on port 11235 on infrastructure the operator runs. Exposes /cra...
Crawl4AI ships a hosted, streamable-HTTP MCP server at https://gate.crawl4ai.com/mcp that exposes the Cloud API as five native agent tools. tools/list answers ANONYMOUSLY — the ...
aid: crawl4ai
name: Crawl4AI
description: Crawl4AI is an open-source, Apache-2.0 web crawler and scraper built to turn any URL into clean, LLM-ready data
— Markdown, typed JSON, screenshots, PDFs, or a map of every URL on a domain. Operated by CONTEXT4AI PTE LTD of Singapore
and created by Hossein Tohidi (@unclecode), the project pairs a 79,000-star Python library and self-hostable Docker API
server with a hosted Cloud API at gate.crawl4ai.com that serves scrape, search, answer, extract and bulk-job endpoints from
three regions, plus a live remote MCP server that exposes the same capabilities to agents as five native tools. Self-hosting
is free and unmetered; the hosted tier is free at 5,000 searches and 1,000 scrapes a month.
type: Index
deliveryModel:
model: open-source-and-hosted
open_source: true
commercial: true
callable_host: true
label: Apache-2.0 open source, plus a hosted commercial API
confidence: high
source:
- https://pypi.org/project/crawl4ai/
- https://gate.crawl4ai.com/
- https://gate.crawl4ai.com/legal/#terms
generated: '2026-08-29'
method: searched
accessModel:
pricing: freemium
onboarding: self-service
trial: true
try_now: true
public: true
label: Freemium — instant 24-hour key, no signup; $7/mo Supporter tier
confidence: high
source:
- https://gate.crawl4ai.com/ (#pricing)
- https://gate.crawl4ai.com/llms.txt
generated: '2026-08-29'
method: searched
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/crawl4ai.png
tags:
- AI Automation
- Web Crawling
- Web Scraping
- Data Extraction
- Search
- LLM Tooling
- Agents
- MCP
- Open-Source
tags_raw:
- AI Automation
- Web Crawling
- Web Scraping
- Data Extraction
- Search
- LLM Tooling
- Agents
- MCP
- Open Source
url: https://raw.githubusercontent.com/api-evangelist/crawl4ai/refs/heads/main/apis.yml
created: '2026-03-27'
modified: '2026-08-29'
specificationVersion: '0.23'
apis:
- aid: crawl4ai:crawl4ai
name: Crawl4AI Cloud API
description: 'The hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clean
Markdown or HTML, GET /search runs a browser-free multi-engine web search, GET /answer returns a direct answer with sources
(experimental), POST /extract pulls typed JSON out of a page with an instruction and/or JSON Schema, POST /scrape/batch
streams up to 50 URLs as NDJSON, and POST /scrape/jobs drains lists of up to 10,000 URLs in the background. Requests route
to the nearest of three regions (EU, Singapore, US). No OpenAPI is published for this surface.'
humanURL: https://gate.crawl4ai.com/docs
baseURL: https://gate.crawl4ai.com
tags:
- AI Automation
- Web Crawling
- Search
properties:
- type: Documentation
url: https://gate.crawl4ai.com/docs
- type: APIReference
url: https://gate.crawl4ai.com/docs
- type: LLMsTxt
url: llms/crawl4ai-llms.txt
- type: MCPServer
url: mcp/crawl4ai-mcp.yml
- type: ToolCrosswalk
url: mcp/crawl4ai-tool-crosswalk.yml
- aid: crawl4ai:crawl4ai-v1
name: Crawl4AI Cloud v1 API
description: The versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POST
/v1/markdown, /v1/screenshot (screenshot and PDF capture), /v1/extract (auto, LLM or reusable CSS schema), /v1/map (domain
URL discovery with BM25 relevance scoring), /v1/crawl/site (whole-site crawl up to 1,000 pages) and /v1/crawl (the full
40-field CrawlerRunConfig), each with an async variant, a job API and webhook callbacks. Authenticated with X-API-Key.
No OpenAPI is published; the reference lives in the provider's own agent skill.
humanURL: https://github.com/unclecode/crawl4ai-cloud-sdk/blob/main/python/claude-plugin/skills/crawl4ai/reference/endpoints.md
baseURL: https://api.crawl4ai.com
tags:
- AI Automation
- Web Crawling
- Data Extraction
properties:
- type: APIReference
url: skills/reference/crawl4ai-endpoints.md
- type: ErrorCatalog
url: skills/reference/crawl4ai-errors.md
- type: SourceCode
url: https://github.com/unclecode/crawl4ai-cloud-sdk
- aid: crawl4ai:crawl4ai-docker
name: Crawl4AI Self-Hosted API
description: 'The Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image,
serving on port 11235 on infrastructure the operator runs. Exposes /crawl, /crawl/stream, /crawl/job, /md, /llm, /screenshot,
/pdf, /html, /token, /health, /artifacts/{id}, /hooks/info, a monitoring API and an MCP bridge at /mcp/sse, /mcp/ws and
/mcp/schema. Since 0.9.0 it is secure-by-default: authentication on, loopback bind without a token, and a strict request
trust boundary.'
humanURL: https://docs.crawl4ai.com/core/self-hosting/
baseURL: http://{host}:11235
baseURLNote: Templated on purpose — this server runs on the customer's own host. Crawl4AI operates no instance of it.
tags:
- AI Automation
- Web Crawling
- Open-Source
tags_raw:
- AI Automation
- Web Crawling
- Open Source
properties:
- type: Documentation
url: https://docs.crawl4ai.com/core/self-hosting/
- type: SourceCode
url: https://github.com/unclecode/crawl4ai/tree/main/deploy/docker
- type: ContainerImage
url: https://hub.docker.com/r/unclecode/crawl4ai
common:
- type: License
name: Apache-2.0
url: https://github.com/unclecode/crawl4ai-cloud-sdk/blob/main/LICENSE
- type: Website
url: https://crawl4ai.com
- type: DeveloperPortal
url: https://gate.crawl4ai.com/
- type: Documentation
url: https://docs.crawl4ai.com
- type: APIReference
url: https://gate.crawl4ai.com/docs
- type: GettingStarted
url: https://docs.crawl4ai.com/core/quickstart/
- type: Blog
url: https://docs.crawl4ai.com/blog/
- type: Support
url: https://discord.gg/jP8KfhDhyN
- type: GitHubOrganization
url: https://github.com/unclecode
- type: SourceCode
url: https://github.com/unclecode/crawl4ai
- type: Roadmap
url: https://github.com/unclecode/crawl4ai/blob/main/ROADMAP.md
- type: LinkedIn
url: https://www.linkedin.com/company/crawl4ai
- type: Integrations
url: https://docs.crawl4ai.com/marketplace/
- type: Pricing
url: https://gate.crawl4ai.com/#pricing
- type: SignUp
url: https://gate.crawl4ai.com/dashboard/
- type: TermsOfService
url: https://gate.crawl4ai.com/legal/#terms
- type: PrivacyPolicy
url: https://gate.crawl4ai.com/legal/#privacy
- type: StatusPage
url: https://unclecode.github.io/crawl4ai-status/
- type: Plans
url: plans/crawl4ai-plans-pricing.yml
- type: RateLimits
url: rate-limits/crawl4ai-rate-limits.yml
- type: FinOps
url: finops/crawl4ai-finops.yml
- type: OpenAPI
url: openapi/crawl4ai-platform-gateway-openapi.json
- type: Overlay
url: overlays/crawl4ai-platform-gateway-overlay.yaml
- type: LLMsTxt
url: llms/crawl4ai-llms.txt
- type: MCPServer
url: mcp/crawl4ai-mcp.yml
- type: ToolCrosswalk
url: mcp/crawl4ai-tool-crosswalk.yml
- type: AgentSkill
url: skills/_index.yml
- type: Packages
url: packages/crawl4ai-packages.yml
- type: SDKs
url: packages/crawl4ai-packages.yml
- type: CLI
url: cli/crawl4ai-cli.yml
- type: Authentication
url: authentication/crawl4ai-authentication.yml
- type: ErrorCatalog
url: errors/crawl4ai-problem-types.yml
- type: Conventions
url: conventions/crawl4ai-conventions.yml
- type: Conformance
url: conformance/crawl4ai-conformance.yml
- type: Lifecycle
url: lifecycle/crawl4ai-lifecycle.yml
- type: Deprecation
url: lifecycle/crawl4ai-lifecycle.yml
- type: ChangeLog
url: changelog/crawl4ai-changelog.yml
- type: Webhooks
url: asyncapi/crawl4ai-webhooks.yml
- type: Sandbox
url: sandbox/crawl4ai-sandbox.yml
- type: DataModel
url: data-model/crawl4ai-data-model.yml
- type: DomainSecurity
url: security/crawl4ai-domain-security.yml
- type: VulnerabilityDisclosure
url: security/crawl4ai-vulnerability-disclosure.yml
- type: Security
url: security/crawl4ai-vulnerability-disclosure.yml
integrations:
- name: Crawl4AI
maintainers:
- FN: Kin Lane
email: kin@apievangelist.com
x-enrichment:
date: '2026-08-29'
status: enriched
artifacts_added: 27
pass: local-v1
Discovery needs no key. Ratings and market analysis are Pro.
Get an API key
Free tier, no form to fill in. Signing in shares your email address with us — we
store it to create your key and to recognise you if you sign in with another
provider. See our Privacy Policy and
Terms.
Kin Score
How this is scored →