emerging · 1.5 capabilitymarket actiondomain

Crawl

Score 22.9 / 100 2 providers +16 via APIs 18 APIs Search apis.io →
Business capabilities this tag reaches
Data & Reference Information Services 2

Providers using this tag (2)

Ranked by API Evangelist rating — Exemplar and Strong are expanded by default.

Strong 1 Solid coverage with minor gaps
BrowserlessBrowser AutomationBrowser InfrastructureAI AgentsWeb Scraping2 APIs55.0
Developing 1 Usable, with meaningful gaps to close
TavilySearchWeb SearchAI AgentsLLM1 API47.9

APIs with this tag (18)

Ranked by the provider's API Evangelist rating — the Kin Score is scored per provider, not per API, so every API of a provider shares its band. How the rating works →

Exemplar 1 Complete, well-documented, and agent-ready
Crawl4AI Crawl APIThe Crawl API from Crawl4AI — 2 operation(s) for crawl.70.3
Strong 5 Solid coverage with minor gaps
Nimbleway Crawl APIThe Crawl API from Nimbleway — 2 operation(s) for crawl.65.3Pagesnap Crawl APISite crawls and asynchronous jobs61.8Search1API Crawl APIThe Crawl API from Search1API — 5 operation(s) for crawl.59.5Irmu Crawl APIFetching a page, charging for it, and collecting it if it was slow.56.5SerpKite Crawl APIThe Crawl API from SerpKite — 2 operation(s) for crawl.56.3
Developing 11 Usable, with meaningful gaps to close
fastcrw-com Crawl APIAsync multi-page crawl jobs53.4Hyperbrowser Crawl APIRecursive crawl jobs across many pages with structured page-by-page results and status polling.53.2usewrit-app Crawl APIDragnet whole-site crawl — start/list/poll/cancel; pages aggregate under a synthetic workflow.50.9Tavily Crawl APIThe Crawl API from Tavily — 1 operation(s) for crawl.47.9Tavily Web APIThe Tavily Web API offers a unified set of endpoints for AI agents to search the web, extract page content,...47.9crawleo-dev Crawl APIThe Crawl API from crawleo-dev — 1 operation(s) for crawl.47.6Agent Exchange Crawl APIThe crawl API from Agent Exchange — 1 operation(s) for crawl.46.2gadgethumans-com Crawl APIThe Crawl API from gadgethumans-com — 1 operation(s) for crawl.45.7snoopscan-com Crawl APIThe crawl API from snoopscan-com — 8 operation(s) for crawl.43.6fastcrawl-net Crawl APIThe Crawl API from fastcrawl-net — 1 operation(s) for crawl.40.7Trieve Crawl APICreate and manage web crawls that ingest pages into a dataset for search and RAG.40.2
Thin 1 Limited public surface area
octocrawl-dev APIoctocrawl-dev API as documented publicly: 31 operations. Contract generated from the documentation by API E...35.9

Companies reaching this through an API (16)

These companies publish an API, specification or operation carrying “Crawl” but do not classify their business under it. Listed unranked and kept out of the count above, because one tagged operation is not a statement about what a company does.

Agent Exchange Crawl4AI crawleo-dev fastcrawl-net fastcrw-com gadgethumans-com Hyperbrowser Irmu Nimbleway octocrawl-dev Pagesnap Search1API SerpKite snoopscan-com Trieve usewrit-app

Score breakdown

Frequency
41.3
log-scaled weighted occurrences
Breadth
0.0
spread across providers
Quality lift
61.6
mean composite of providers using it
Cohesion
1.4
strength of nearest seed neighbor

Related tags

Extract 7 co-occurrences Web Scraping 11 co-occurrences Screenshots 5 co-occurrences Maps 6 co-occurrences Data Extraction 7 co-occurrences Monitors 5 co-occurrences Search 13 co-occurrences

Where this tag comes from

Api tag17
Provider tag2
Openapi tag14
Openapi op tag49

Work with this as data

Every tag here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for tags

7 MCP tools reach this
  • find_tagsBrowse and filter every tag in the catalog.
  • get_cohortThis tag as a scored cohort — every provider carrying it, with scores.
  • cohort_statsPRO — the distribution across this tag: mean, median, band split, adoption rates.
  • cohort_rankingsPRO — the leaderboard, on composite AND agent-readiness axes.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This tag 
curl "https://apis.io/api/v1/tags/crawl"
All tags 
curl "https://apis.io/api/v1/tags?limit=25"
As a scored cohort (paid plan)
The distribution (Pro)

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.