Web Crawling
Providers using this tag (6)
Ranked by API Evangelist rating — Exemplar and Strong are expanded by default.
Exemplar 1 Complete, well-documented, and agent-ready
Strong 2 Solid coverage with minor gaps
Developing 2 Usable, with meaningful gaps to close
Thin 1 Limited public surface area
APIs with this tag (6)
Ranked by the provider's API Evangelist rating — the Kin Score is scored per provider, not per API, so every API of a provider shares its band. How the rating works →
Exemplar 4 Complete, well-documented, and agent-ready
Crawl4AI Cloud APIThe hosted Crawl4AI API. One key, plain JSON, one fast endpoint per job: POST /scrape turns a URL into clea...Crawl4AI Cloud v1 APIThe versioned Crawl4AI cloud surface the first-party Cloud SDKs and the Claude Code plugin call. Covers POS...Crawl4AI Self-Hosted APIThe Apache-2.0 API server that ships inside the crawl4ai project and the unclecode/crawl4ai Docker image, s...Wappalyzer Crawl APIDeep crawl API that indexes multiple pages of a website to build a comprehensive technology profile, suppor...
Developing 1 Usable, with meaningful gaps to close
Thin 1 Limited public surface area
Companies reaching this through an API (2)
These companies publish an API, specification or operation carrying “Web Crawling” but do not classify their business under it. Listed unranked and kept out of the count above, because one tagged operation is not a statement about what a company does.
Score breakdown
Frequency
30.2
log-scaled weighted occurrences
Breadth
0.2
spread across providers
Quality lift
65.3
mean composite of providers using it
Cohesion
0.4
strength of nearest seed neighbor
Related tags
Where this tag comes from
Provider tag6
Api tag7
Work with this as data
Every tag here is available over the APIs.io API and to AI agents over MCP.