niche · -0.8 capabilitymarket actiondomainschema

Crawl

Score 17.4 / 100 1 providers +7 via APIs 9 APIs Search apis.io →

Providers using this tag (1)

Ranked by API Evangelist rating — Exemplar and Strong are expanded by default.

Developing 1 Usable, with meaningful gaps to close
TavilySearchWeb SearchAI AgentsLLM1 API39.5

APIs with this tag (9)

Ranked by the provider's API Evangelist rating — the Kin Score is scored per provider, not per API, so every API of a provider shares its band. How the rating works →

Exemplar 2 Complete, well-documented, and agent-ready
Search1API Crawl APIThe Crawl API from Search1API — 5 operation(s) for crawl.71.0Crawl4AI Crawl APIThe Crawl API from Crawl4AI — 2 operation(s) for crawl.67.1
Strong 2 Solid coverage with minor gaps
Pagesnap Crawl APISite crawls and asynchronous jobs58.6Nimbleway Crawl APIThe Crawl API from Nimbleway — 2 operation(s) for crawl.55.7
Developing 5 Usable, with meaningful gaps to close
Agent Exchange Crawl APIThe crawl API from Agent Exchange — 1 operation(s) for crawl.48.2Hyperbrowser Crawl APIRecursive crawl jobs across many pages with structured page-by-page results and status polling.46.3Trieve Crawl APICreate and manage web crawls that ingest pages into a dataset for search and RAG.40.2Tavily Crawl APIThe Crawl API from Tavily — 1 operation(s) for crawl.39.5Tavily Web APIThe Tavily Web API offers a unified set of endpoints for AI agents to search the web, extract page content,...39.5

Companies reaching this through an API (7)

These companies publish an API, specification or operation carrying “Crawl” but do not classify their business under it. Listed unranked and kept out of the count above, because one tagged operation is not a statement about what a company does.

Agent Exchange Crawl4AI Hyperbrowser Nimbleway Pagesnap Search1API Trieve

Score breakdown

Frequency
34.4
log-scaled weighted occurrences
Breadth
0.0
spread across providers
Quality lift
43.1
mean composite of providers using it
Cohesion
1.0
strength of nearest seed neighbor

Related tags

Web Scraping 5 co-occurrences Search 7 co-occurrences AI Agents 5 co-occurrences MCP 6 co-occurrences

Where this tag comes from

Api tag9
Provider tag2
Openapi tag6
Openapi op tag16

Work with this as data

Every tag here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for tags

7 MCP tools reach this
  • find_tagsBrowse and filter every tag in the catalog.
  • get_cohortThis tag as a scored cohort — every provider carrying it, with scores.
  • cohort_statsPRO — the distribution across this tag: mean, median, band split, adoption rates.
  • cohort_rankingsPRO — the leaderboard, on composite AND agent-readiness axes.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This tag 
curl "https://apis.io/api/v1/tags/crawl"
All tags 
curl "https://apis.io/api/v1/tags?limit=25"
As a scored cohort 
curl "https://apis.io/api/v1/cohorts/tag/crawl"
The distribution (Pro) 
curl "https://apis.io/api/v1/cohorts/tag/crawl/stats" \
  -H "X-API-Key: $APIS_IO_KEY"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.