Crawlbase website screenshot

Crawlbase

Crawlbase (formerly ProxyCrawl) is a web crawling and scraping platform that fetches any web page through a large rotating proxy network with optional headless-Chrome JavaScript rendering, returning raw HTML, Markdown, screenshots, or structured JSON. A single token-authenticated REST host (api.crawlbase.com) exposes the Crawling API, a Scraper API of ready-made site extractors, Cloud Storage for crawled pages, a Screenshots API, and a Leads API for domain email discovery.

Crawlbase publishes 5 API contracts indexed on the APIs.io network, including Crawling API, Leads API, Scraper API, and 2 more. Tagged areas include Web Scraping, Web Crawling, Web Intelligence, Data Extraction, and Proxy.

The Crawlbase catalog on APIs.io includes 1 Spectral governance ruleset.

Crawlbase’s developer surface includes authentication, documentation, engineering blog, and 9 more developer resources.

🌐 Visit website📡 Profile data on GitHub
27.7/100 thin ▬ flat Agent 28/100 agent aware Front door AI 6/6 dominant Heartbeat 98/100 alive saas Full breakdown ↓
scored 2026-11-13 · rubric v0.28.0 · Show this score on your site →
AccessFreemiumOpenFree trial⚡ Free to try
1 published contract 1 MCP Servers
Web ScrapingWeb CrawlingWeb IntelligenceData ExtractionProxyScraper APIData CollectionSERPWeb Data

What this lets a business do 1

Business capabilities this provider's published APIs can perform, derived from its own contracts. Browse all capabilities →

Data & Reference Information Services
AE-230
Leads API
GET /leads

Kin Score

Kin Score Kin Score How this is scored →
scored 2026-11-13 · rubric v0.28.0
Composite quality — 27.7/100 · thin
Contract Quality 9.1 / 19
Developer Ergonomics 4.8 / 15
Access Clarity 1.0 / 15
Operational Transparency 1.2 / 10
Contract Governance 1.2 / 9
Discoverability 5.1 / 8
Regulatory Posture 0.9 / 15
Create-or-Update Ergonomics 0.0 / 10
Agent readiness — 28/100 · agent aware
Machine-Readable Contract 18 / 18
Agentic Access Contract 10 / 10
Documented Reversibility 0 / 6
MCP Server 12 / 12
Machine-Readable Auth 10 / 10
Idempotency 0 / 9
Stable Error Semantics 0 / 8
Request/Response Examples 0 / 7
Rate-Limit Signaling 7 / 7
Typed Event Surface 0 / 6
Agent Skills 0 / 5
Well-Known Catalog 0 / 4
Consent & Bot Identity 0 / 3
A2A Agent Card 0 / 8
Dry-Run / Simulate Mode 0 / 4
Delegated User Identity 0 / 6
Protected Resource Metadata 0 / 5
Registration Without a Human 0 / 6
Agentic Commerce Well-Known Document 0 / 5
Pagination 5 / 5
Distinct Operations 3 / 3
MCP Tool Quality 6 / 6
Regulatory Posture applies to this provider. Its tags matched the Horizontal (data, software, accessibility, platform) regime, so Regulatory Posture carries 15 points of the composite. If this regime is wrong for your business, say so on your provider repo — the applicability map is public and we will correct it.
Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it carries 10 points of the composite. It is scored from the published contracts themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in front of it, and the first time that check is skipped a duplicate record is created. Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet, not penalised by it.
The six quality facets above are damped to 75 points between them, because the conditional facet above carries the other 25. That is why each facet's contribution is shown against a damped maximum: raising a quality facet moves the composite by 75% of its nominal weight, not 100%. The full arithmetic is at apis.io/rating/.
Show this score on your site. Kin Score badge preview for Crawlbase It updates itself when the score changes. Paste this where you link to your API docs: 
<a href="https://apis.io/providers/crawlbase/"><img src="https://apis.io/badge/crawlbase/card.svg" alt="Crawlbase on APIs.io — Kin Score by API Evangelist"></a>
Other styles, a README pill and a shields.io endpoint are on the badge page.
Raise your score.
Agent readiness — checks not met, largest first (exact, from this profile):
8 more unmet checks in the Agent Readiness table above.
Kin Score — facets with the most composite points still available (weight × headroom):
Each check links to what satisfies it. Publish the artifact, or tell us where it already lives, and the score moves on the next scoring run.
Not sure how this applies to your stack? Ask about any one item. Kin Lane reads every question, looks at your profile and your situation, and replies with what a fix would take — scoped to that one area, priced up front, yours whether you ship it or not. Ask a question →
Ask about:
Crawlbase · a person reads this and replies by email.
or email info@apis.io
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from the provider profile at artifacts.apis.io/crawlbase/apis.yml: open an issue on github.com/api-evangelist/crawlbase to ask a question or point us at an artifact you publish. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

Markets this provider is in provisional

The Kin Score Market layer reads a provider inside its market: its position, and how deep its integrations go. Depth is only comparable within a position. Provisional and unweighted: not part of the Kin Score or agent readiness.

MarketPositionDepthRegistered MCP serverDeclares compatibility
Agent Skills (SKILL.md packages) long-tail 5.0 (thin) - 1
Cisco DevNet Ecosystem (builders on Meraki, Catalyst, ISE, ThousandEyes, Splunk) long-tail 5.0 (thin) - 1

Registered MCP server and declares compatibility are report only — not weighted. Undeclared means we found nothing we can read, so there is no number: we never infer integrations.

Standards implemented 2

Standards this provider implements or carries. De facto ones are profiled and measured by the API Commons; formal ones come from the standards directory and are linked by the provider's own tags. The evidence column says how each claim was established — not that it was made.

Model Context Protocol (MCP)
measured — observed by APIs.io (verified)
OpenAPI
measured — observed by APIs.io (published)

APIs 5

Individual APIs this provider publishes, each with its own machine-readable definition.

Crawlbase Crawling API

Fetch any URL through the rotating proxy network, optionally rendered.

Crawlbase Leads API

Publicly visible email discovery for a domain (legacy).

Crawlbase Scraper API

Ready-made structured-data extractors for supported sites (legacy).

Crawlbase Screenshots API

Rendered page screenshots in headless Chrome (legacy).

Crawlbase Storage API

Retrieve, list, and delete previously stored crawls.

Open Collections 6

Open, tool-agnostic API collections (OpenAPI-derived and Bruno).

Crawlbase Crawling API API

OPEN COLLECTION

Crawlbase Crawling API Leads API API

OPEN COLLECTION

Crawlbase Crawling API Scraper API API

OPEN COLLECTION

Crawlbase Crawling API Screenshots API API

OPEN COLLECTION

Crawlbase Crawling API Storage API API

OPEN COLLECTION

Crawlbase API

OPEN COLLECTION

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

Crawlbase MCP Server

Remote MCP server at mcp.crawlbase.com over HTTP.

MCP SERVER

Crawlbase API pricing

Not yet checked against Crawlbase's own pricing page. Compiled by API Evangelist on 2026-07-12 from crawlbase.com/pricing; confirm before you rely on a figure. A free plan or tier is listed.

PlanPrice
Free Trial free —
Crawling API - Pay As You Go usage —
Smart / AI Proxy - Free free —
Smart / AI Proxy - Starter paid —
Smart / AI Proxy - Advanced paid —
Smart / AI Proxy - Premium paid —
Cloud Storage - Free free —
Cloud Storage - Developer paid —

All 10 plans, limits and sources →

Rate limits

Compiled by API Evangelist, not yet checked against Crawlbase's documentation. All 4 limits →

Pricing Plans 1

Published pricing tiers and plan structures.

Crawlbase Plans Pricing

10 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Crawlbase Rate Limits

4 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Crawlbase Finops

FINOPS

Spectral Rules 1

Spectral governance rulesets for linting and validating these APIs. Counts are the severity each rule declares, not results of running them.

Crawlbase API Rules

65 rules · 63 error-severity 1 warn-severity 1 info-severity

SPECTRAL

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Crawlbase Authentication

apiKey · 1 scheme

SECURITY

Crawlbase Domain Security

TLSv1.3 · HSTS · DMARC

SECURITY

Agentic Access (API Evangelist recommendations) 1

Recommended x-agentic-access execution contracts for AI agents.

Crawlbase Agentic Access

11 operations · 3 acting

11 operations · 3 acting

AGENTIC

Resources

Documentation 1

Reference material describing how the API behaves

Documentation
Documentation

Agent Surfaces 1

MCP servers, agent skills, and machine-readable catalogs

AgenticAccess
AgenticAccess

Build 1

SDKs, sample code, and the tooling you integrate with

GitHubOrganization
GitHubOrganization

Access & Security 2

Authentication, authorization, and security posture

DomainSecurity
DomainSecurity
 Authentication
Authentication

Operate 1

Status, limits, changes, and where to get help

RateLimits
RateLimits

Commercial 2

Pricing, plans, and the legal terms of use

Plans
Plans
 FinOps
FinOps

Company 3

The organization behind the API

LinkedIn
LinkedIn
 Website
Website
 Blog
Blog

Other 1

Properties that don't map to a standard resource type

CapabilityMap
CapabilityMap

Source (apis.yml)

apis.yml Raw ↑ 
aid: crawlbase
url: https://raw.githubusercontent.com/api-evangelist/crawlbase/refs/heads/main/apis.yml
name: Crawlbase
kind: company
description: Crawlbase (formerly ProxyCrawl) is a web crawling and scraping platform that fetches any web page through a large
  rotating proxy network with optional headless-Chrome JavaScript rendering, returning raw HTML, Markdown, screenshots, or
  structured JSON. A single token-authenticated REST host (api.crawlbase.com) exposes the Crawling API, a Scraper API of ready-made
  site extractors, Cloud Storage for crawled pages, a Screenshots API, and a Leads API for domain email discovery.
deliveryModel:
  model: saas
  open_source: false
  commercial: true
  callable_host: true
  label: Hosted service · you call their endpoint
  confidence: high
  source:
  - openapi
  - pricing
  generated: '2026-08-28'
  method: derived
accessModel:
  pricing: freemium
  onboarding: open
  trial: true
  try_now: true
  public: true
  gated: none
  gated_source:
  - onboarding
  label: Freemium (free trial) · Open access
  confidence: medium
  source:
  - plans
  - authentication
  - security
  generated: '2026-11-13'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/crawlbase.png
tags:
- Web Scraping
- Web Crawling
- Web Intelligence
- Data Extraction
- Proxy
- Scraper API
- Data Collection
- SERP
- Web Data
created: '2026-07-12'
modified: '2026-09-16'
specificationVersion: '0.23'
apis:
- aid: crawlbase:crawlbase-crawling-api-api
  name: Crawlbase Crawling API
  description: Fetch any URL through the rotating proxy network, optionally rendered.
  humanURL: https://crawlbase.com/docs/crawling-api/
  baseURL: https://api.crawlbase.com
  tags:
  - Crawling API
  properties:
  - type: OpenAPI
    url: openapi/crawlbase-crawling-api-api-openapi.yml
  - type: Documentation
    url: https://crawlbase.com/docs/crawling-api/
  - type: APIReference
    url: https://crawlbase.com/docs/crawling-api/parameters/
  - type: PostmanCollection
    url: collections/crawlbase.postman_collection.json
  - type: OpenCollection
    url: collections/crawlbase.opencollection.json
  - type: Documentation
    url: https://crawlbase.com/docs/scraper-api/
  - type: Documentation
    url: https://crawlbase.com/docs/storage-api/
  - type: Documentation
    url: https://crawlbase.com/docs/screenshots-api/
  - type: Documentation
    url: https://crawlbase.com/docs/leads-api/
- aid: crawlbase:crawlbase-leads-api-api
  name: Crawlbase Leads API
  description: Publicly visible email discovery for a domain (legacy).
  humanURL: https://crawlbase.com/docs/crawling-api/
  baseURL: https://api.crawlbase.com
  tags:
  - Leads API
  properties:
  - type: OpenAPI
    url: openapi/crawlbase-leads-api-api-openapi.yml
  - type: Documentation
    url: https://crawlbase.com/docs/crawling-api/
  - type: APIReference
    url: https://crawlbase.com/docs/crawling-api/parameters/
  - type: PostmanCollection
    url: collections/crawlbase.postman_collection.json
  - type: OpenCollection
    url: collections/crawlbase.opencollection.json
  - type: Documentation
    url: https://crawlbase.com/docs/scraper-api/
  - type: Documentation
    url: https://crawlbase.com/docs/storage-api/
  - type: Documentation
    url: https://crawlbase.com/docs/screenshots-api/
  - type: Documentation
    url: https://crawlbase.com/docs/leads-api/
- aid: crawlbase:crawlbase-scraper-api-api
  name: Crawlbase Scraper API
  description: Ready-made structured-data extractors for supported sites (legacy).
  humanURL: https://crawlbase.com/docs/crawling-api/
  baseURL: https://api.crawlbase.com
  tags:
  - Scraper API
  properties:
  - type: OpenAPI
    url: openapi/crawlbase-scraper-api-api-openapi.yml
  - type: Documentation
    url: https://crawlbase.com/docs/crawling-api/
  - type: APIReference
    url: https://crawlbase.com/docs/crawling-api/parameters/
  - type: PostmanCollection
    url: collections/crawlbase.postman_collection.json
  - type: OpenCollection
    url: collections/crawlbase.opencollection.json
  - type: Documentation
    url: https://crawlbase.com/docs/scraper-api/
  - type: Documentation
    url: https://crawlbase.com/docs/storage-api/
  - type: Documentation
    url: https://crawlbase.com/docs/screenshots-api/
  - type: Documentation
    url: https://crawlbase.com/docs/leads-api/
- aid: crawlbase:crawlbase-screenshots-api-api
  name: Crawlbase Screenshots API
  description: Rendered page screenshots in headless Chrome (legacy).
  humanURL: https://crawlbase.com/docs/crawling-api/
  baseURL: https://api.crawlbase.com
  tags:
  - Screenshots API
  properties:
  - type: OpenAPI
    url: openapi/crawlbase-screenshots-api-api-openapi.yml
  - type: Documentation
    url: https://crawlbase.com/docs/crawling-api/
  - type: APIReference
    url: https://crawlbase.com/docs/crawling-api/parameters/
  - type: PostmanCollection
    url: collections/crawlbase.postman_collection.json
  - type: OpenCollection
    url: collections/crawlbase.opencollection.json
  - type: Documentation
    url: https://crawlbase.com/docs/scraper-api/
  - type: Documentation
    url: https://crawlbase.com/docs/storage-api/
  - type: Documentation
    url: https://crawlbase.com/docs/screenshots-api/
  - type: Documentation
    url: https://crawlbase.com/docs/leads-api/
- aid: crawlbase:crawlbase-storage-api-api
  name: Crawlbase Storage API
  description: Retrieve, list, and delete previously stored crawls.
  humanURL: https://crawlbase.com/docs/crawling-api/
  baseURL: https://api.crawlbase.com
  tags:
  - Storage API
  properties:
  - type: OpenAPI
    url: openapi/crawlbase-storage-api-api-openapi.yml
  - type: Documentation
    url: https://crawlbase.com/docs/crawling-api/
  - type: APIReference
    url: https://crawlbase.com/docs/crawling-api/parameters/
  - type: PostmanCollection
    url: collections/crawlbase.postman_collection.json
  - type: OpenCollection
    url: collections/crawlbase.opencollection.json
  - type: Documentation
    url: https://crawlbase.com/docs/scraper-api/
  - type: Documentation
    url: https://crawlbase.com/docs/storage-api/
  - type: Documentation
    url: https://crawlbase.com/docs/screenshots-api/
  - type: Documentation
    url: https://crawlbase.com/docs/leads-api/
common:
- type: CapabilityMap
  url: capabilities/crawlbase-capability-edges.yml
  name: Crawlbase Business Capability Map
- type: AgenticAccess
  url: agentic-access/crawlbase-agentic-access.yml
- type: DomainSecurity
  url: security/crawlbase-domain-security.yml
- type: Authentication
  url: authentication/crawlbase-authentication.yml
- type: GitHubOrganization
  url: https://github.com/crawlbase
- type: LinkedIn
  url: https://www.linkedin.com/company/crawlbase
- type: Website
  url: https://crawlbase.com/
- type: Documentation
  url: https://crawlbase.com/docs
- type: Plans
  url: plans/crawlbase-plans-pricing.yml
- type: RateLimits
  url: rate-limits/crawlbase-rate-limits.yml
- type: FinOps
  url: finops/crawlbase-finops.yml
- url: https://crawlbase.com/blog
  type: Blog
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com

Crawlbase API — frequently asked questions

Does Crawlbase have an API?

Yes. APIs.io lists 5 APIs from Crawlbase, including Crawlbase Crawling API, Crawlbase Leads API and Crawlbase Scraper API, and 2 more.

Is there an OpenAPI specification for the Crawlbase API?

Yes. APIs.io indexes 5 OpenAPI specifications for Crawlbase, each linked for download from this page.

How do I authenticate with the Crawlbase API?

Crawlbase uses bearer-token authentication (apiKey · 1 scheme). The details are profiled on APIs.io as Crawlbase Authentication.

How much does the Crawlbase API cost?

APIs.io lists 10 pricing plans for Crawlbase, with each plan's price, limits and inclusions under Pricing on this page.

Does Crawlbase have an MCP server?

Yes. APIs.io lists 1 MCP server for Crawlbase: Remote MCP server at mcp.crawlbase.com over HTTP.

Is the Crawlbase API ready for AI agents?

Crawlbase has a Kin Score of 27.7 out of 100 (thin) and an Agent Readiness score of 28.1 (agent-aware), scored by API Evangelist on 2026-11-13.

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this
  • find_providersBrowse and filter every provider in the catalog.
  • get_provider_artifactsEvery artifact this provider publishes, grouped by type.
  • get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
  • get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
  • get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
  • get_provider_ratingPRO — composite, band, trend and facet scores.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This provider 
curl "https://apis.io/api/v1/providers/crawlbase"
All providers 
curl "https://apis.io/api/v1/providers?limit=25"
Every operation they expose 
curl "https://apis.io/api/v1/providers/crawlbase/operations?limit=25"
How their score was established 
curl "https://apis.io/api/v1/providers/crawlbase/evidence"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.