WebCrawlerAPI

WebCrawlerAPI is a web crawling and scraping API from 103 Labs (Netherlands) that turns websites into clean, LLM-ready markdown, cleaned text, HTML or link lists for AI agents, support bots and RAG pipelines. It offers asynchronous multi-page crawl jobs (POST /v1/crawl with polling or webhooks), single-page scraping (POST /v2/scrape, sync or async, with prompt-based structured outputs validated against a JSON Schema), scheduled change-detection Feeds delivered as Atom 1.0, JSON Feed 1.1 or webhooks, a prompt-driven autonomous crawling agent with a required per-run spend cap, and organization usage/balance endpoints. Requests authenticate with a Bearer API key; usage is metered per successfully crawled page on a $0/month pay-as-you-go tier or $29/$99/$499 monthly plans, with 7-day caching (cache hits free). Published surfaces: a Swagger 2.0 document behind a live Swagger UI, an Agent API OpenAPI 3.0.3, an A2A agent card and RFC 9727 API catalog at /.well-known/, llms.txt, SDKs for JavaScript, Python, PHP, .NET and Java, an npm MCP server (local stdio), the webcr CLI, a Claude Code Agent Skill, a public Postman collection and a dated changelog.

WebCrawlerAPI publishes 8 API contracts indexed on the APIs.io network, including Agent API, Feeds API, Jobs API, and 5 more. Tagged areas include Web Scraping, Web Crawling, Data Extraction, Markdown, and AI Agents.

The WebCrawlerAPI catalog on APIs.io includes 1 event-driven AsyncAPI specification.

WebCrawlerAPI’s developer surface includes documentation, API reference, getting-started guide, support, engineering blog, changelog, pricing, and 39 more developer resources.

🌐 Visit website 📡 Source on GitHub

64.5/100 strong ▬ flat Agent 40/100 agent ready Front door AI 5/6 prominent saas Full breakdown ↓
scored 2026-10-09 · rubric v0.23.0

3 published contracts 2 MCP Servers

Web Scraping Web Crawling Data Extraction Markdown AI Agents LLM RAG Web Feeds Change Detection Structured Data MCP A2A Agent-Native

Kin Score

Kin Score How this is scored →
scored 2026-10-09 · rubric v0.23.0

Composite quality — 64.5/100 · strong

Contract Quality 10.7 / 19

Developer Ergonomics 12.5 / 15

Access Clarity 11.4 / 15

Operational Transparency 4.4 / 10

Contract Governance 1.6 / 9

Discoverability 6.6 / 8

Regulatory Posture 4.5 / 15

Create-or-Update Ergonomics 0.0 / 10

Agent readiness — 40/100 · agent ready

Machine-Readable Contract 18 / 18

Agentic Access Contract 0 / 10

Documented Reversibility 6 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 0 / 9

Stable Error Semantics 8 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 7 / 7

Typed Event Surface 6 / 6

Agent Skills 5 / 5

Well-Known Catalog 4 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Delegated User Identity 0 / 6

Protected Resource Metadata 0 / 5

Registration Without a Human 0 / 6

Agentic Commerce Well-Known Document 0 / 5

Regulatory Posture applies to this provider. Its tags matched the Horizontal (data, software, accessibility, platform) regime, so Regulatory Posture carries 15 points of the composite. If this regime is wrong for your business, say so on your provider repo — the applicability map is public and we will correct it.

Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it carries 10 points of the composite. It is scored from the published contracts themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in front of it, and the first time that check is skipped a duplicate record is created. Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet, not penalised by it.

The six quality facets above are damped to 75 points between them, because the conditional facet above carries the other 25. That is why each facet's contribution is shown against a damped maximum: raising a quality facet moves the composite by 75% of its nominal weight, not 100%. The full arithmetic is at apis.io/rating/.

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from github.com/api-evangelist/webcrawlerapi-com: open an issue to ask a question, or submit a pull request to add artifacts. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

Standards implemented 1

Standards this provider implements or carries. De facto ones are profiled and measured by the API Commons; formal ones come from the standards directory and are linked by the provider's own tags. The evidence column says how each claim was established — not that it was made.

Markdown

tagged — the provider's own APIs.json carries the standard as a tag; implementation not verified

APIs 8

Individual APIs this provider publishes, each with its own machine-readable definition.

WebCrawlerAPI MCP Server

Official Model Context Protocol server distributed as the npm package webcrawler-mcp (npx -y webcrawler-mcp) and run locally over stdio - or self-hosted over streamable HTTP wit...

WebCrawlerAPI Agent Card

A2A-style agent card served from https://webcrawlerapi.com/.well-known/agent-card.json describing "WebCrawlerAPI Agent" (version 1.0.0) with three skills - scrape, crawl and cra...

WebCrawlerAPI Agent API

The Agent API from WebCrawlerAPI — 3 operation(s) for agent.

WebCrawlerAPI Feeds API

The Feeds API from WebCrawlerAPI — 9 operation(s) for feeds.

WebCrawlerAPI Jobs API

The Jobs API from WebCrawlerAPI — 6 operation(s) for jobs.

WebCrawlerAPI Organization API

The Organization API from WebCrawlerAPI — 1 operation(s) for organization.

WebCrawlerAPI Status API

The Status API from WebCrawlerAPI — 2 operation(s) for status.

WebCrawlerAPI V2 Scraping API

The V2 Scraping API from WebCrawlerAPI — 2 operation(s) for v2 scraping.

Scroll for all 8

MCP Servers 2

Model Context Protocol servers that expose these APIs to AI agents.

WebCrawlerAPI MCP Server

WebCrawlerAPI ships an official MCP server as an npm package (webcrawler-mcp) that a user runs locally over stdio, or self-hosts over HTTP with USE_HTTP=true. There is NO provid...

MCP SERVER

webcrawler-mcp on npm (local stdio package)

MCP SERVER

Pricing Plans 1

Published pricing tiers and plan structures.

Webcrawlerapi Com Plans Pricing

4 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Webcrawlerapi Com Rate Limits

0 limits

RATE LIMITS

Event Specifications 1

AsyncAPI definitions for this provider's event-driven and streaming APIs.

Webcrawlerapi Com Webhooks

ASYNCAPI

OpenAPI Overlays 1

Overlays applied on top of this provider's contracts. Each card says who wrote it: a document the provider publishes, or one API Evangelist derived or generated.

Webcrawlerapi Com Openapi Overlay

28 actions · documentation · webcrawlerapi-com-openapi.yml

GENERATED

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Webcrawlerapi Com Authentication

apiKey · 1 scheme

SECURITY

Webcrawlerapi Com Domain Security

TLSv1.3 · DMARC

SECURITY

Resources

Get Started 4

Portal, sign-up, and the first successful call

DeveloperPortal

DeveloperPortal

GettingStarted

GettingStarted

SignUp

SignUp

Login

Documentation 2

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Agent Surfaces 6

MCP servers, agent skills, and machine-readable catalogs

LLMsTxt

LLMsTxt

LLMsTxt

WellKnown

MCPServer

AgentSkill

Design & Contract 6

Pagination, idempotency, versioning, errors, and events

Conventions

ErrorCatalog

DataModel

Webhooks

Lifecycle

Conformance

Build 6

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

Postman

ToolCrosswalk

Packages

SDKs

CLI

Access & Security 2

Authentication, authorization, and security posture

Authentication

Authentication

DomainSecurity

DomainSecurity

Operate 5

Status, limits, changes, and where to get help

Support

ChangeLog

ChangeLog

StatusPage

RateLimits

Commercial 4

Pricing, plans, and the legal terms of use

Pricing

TermsOfService

PrivacyPolicy

Plans

Company 4

The organization behind the API

Website

Blog

BlogRSS

Twitter

Other 7

Properties that don't map to a standard resource type

RefundPolicy

Leadership

APICatalog

AgentCard

Overlay

Subprocessors

NoticeAndAction

Scroll for all 7

Source (apis.yml)

aid: webcrawlerapi-com
name: WebCrawlerAPI
description: 'WebCrawlerAPI is a web crawling and scraping API from 103 Labs (Netherlands) that turns websites into clean,
  LLM-ready markdown, cleaned text, HTML or link lists for AI agents, support bots and RAG pipelines. It offers asynchronous
  multi-page crawl jobs (POST /v1/crawl with polling or webhooks), single-page scraping (POST /v2/scrape, sync or async, with
  prompt-based structured outputs validated against a JSON Schema), scheduled change-detection Feeds delivered as Atom 1.0,
  JSON Feed 1.1 or webhooks, a prompt-driven autonomous crawling agent with a required per-run spend cap, and organization
  usage/balance endpoints. Requests authenticate with a Bearer API key; usage is metered per successfully crawled page on
  a $0/month pay-as-you-go tier or $29/$99/$499 monthly plans, with 7-day caching (cache hits free). Published surfaces: a
  Swagger 2.0 document behind a live Swagger UI, an Agent API OpenAPI 3.0.3, an A2A agent card and RFC 9727 API catalog at
  /.well-known/, llms.txt, SDKs for JavaScript, Python, PHP, .NET and Java, an npm MCP server (local stdio), the webcr CLI,
  a Claude Code Agent Skill, a public Postman collection and a dated changelog.'
deliveryModel:
  model: saas
  license_evidence:
    not_product:
    - webcrawlerapi/webcrawlerapi-mcp
  open_source: unknown
  commercial: true
  callable_host: true
  label: Hosted service · you call their endpoint
  confidence: low
  source:
  - openapi
  - pricing
  - repository-not-product
  generated: '2026-09-25'
  method: derived
image: https://webcrawlerapi.com/favicon.png
url: https://raw.githubusercontent.com/api-evangelist/webcrawlerapi-com/refs/heads/main/apis.yml
x-type: company
x-source: harvest:a2a-registry
x-tier: profiled
x-tier-reason: enrichment pass 2026-09-19 (local-v3; was a2a-registry harvest stub)
specificationVersion: '0.20'
created: '2026-09-19'
modified: '2026-09-19'
tags:
- Web Scraping
- Web Crawling
- Data Extraction
- Markdown
- AI Agents
- LLM
- RAG
- Web Feeds
- Change Detection
- Structured Data
- MCP
- A2A
- Agent-Native
tags_raw:
- Web Scraping
- Web Crawling
- Data Extraction
- Markdown
- AI Agents
- LLM
- RAG
- Web Feeds
- Change Detection
- Structured Data
- MCP
- A2A
- agent-native
apis:
- aid: webcrawlerapi-com:webcrawlerapi-mcp-server
  name: WebCrawlerAPI MCP Server
  description: 'Official Model Context Protocol server distributed as the npm package webcrawler-mcp (npx -y webcrawler-mcp)
    and run locally over stdio - or self-hosted over streamable HTTP with USE_HTTP=true - by the user; there is no provider-hosted
    remote endpoint. It exposes one tool, webcrawler-scrape (url, optional prompt, optional main_content_only), which POSTs
    to https://api.webcrawlerapi.com/v2/scrape with the API key from the WEBCRAWLER_API_KEY environment variable and returns
    markdown (or structured_data when a prompt is set). Source: github.com/WebCrawlerAPI/webcrawlerapi-mcp.'
  humanURL: https://webcrawlerapi.com/docs/sdk/mcp
  baseURL: https://api.webcrawlerapi.com
  tags:
  - MCP
  - AI Agents
  - Web Scraping
  - Agent-Native
  tags_raw:
  - MCP
  - AI Agents
  - Web Scraping
  - agent-native
  properties:
  - type: MCPServer
    url: mcp/webcrawlerapi-com-mcp.yml
  - type: MCPServer
    name: webcrawler-mcp on npm (local stdio package)
    url: https://www.npmjs.com/package/webcrawler-mcp
  - type: ToolCrosswalk
    url: mcp/webcrawlerapi-com-tool-crosswalk.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/sdk/mcp
  - type: SourceCode
    url: https://github.com/WebCrawlerAPI/webcrawlerapi-mcp
  image: https://webcrawlerapi.com/favicon.png
- aid: webcrawlerapi-com:webcrawlerapi-a2a-agent-card
  name: WebCrawlerAPI Agent Card
  description: 'A2A-style agent card served from https://webcrawlerapi.com/.well-known/agent-card.json describing "WebCrawlerAPI
    Agent" (version 1.0.0) with three skills - scrape, crawl and crawling-agent - Bearer securitySchemes, text/markdown/JSON
    input and output modes, a healthCheckUrl and an openapi_url pointing at the Agent API OpenAPI. Graded FLAVORED against
    A2A 1.0.0: protocolVersion is absent and the card''s url is the REST base path https://api.webcrawlerapi.com/v1, which
    does not answer A2A JSON-RPC (a tasks/get POST returns 404). It is a real, provider-served discovery document for the
    REST API rather than a working A2A endpoint.'
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com/v1
  tags:
  - A2A
  - AI Agents
  - Agent-Native
  tags_raw:
  - A2A
  - AI Agents
  - agent-native
  properties:
  - type: AgentCard
    url: a2a/webcrawlerapi-com-a2a.yml
  - type: AgentCard
    name: A2A Agent Card (provider-hosted)
    url: https://webcrawlerapi.com/.well-known/agent-card.json
  image: https://webcrawlerapi.com/favicon.png
- aid: webcrawlerapi-com:webcrawlerapi-com-agent-api
  name: WebCrawlerAPI Agent API
  description: The Agent API from WebCrawlerAPI — 3 operation(s) for agent.
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com
  tags:
  - Agents
  tags_raw:
  - Agent
  properties:
  - type: OpenAPI
    url: openapi/webcrawlerapi-com-agent-api-openapi.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/getting-started
  - type: APIReference
    url: https://api.webcrawlerapi.com/swagger/index.html
  - type: APIReference
    url: https://webcrawlerapi.com/docs/api/crawl
  - type: Authentication
    url: authentication/webcrawlerapi-com-authentication.yml
  - type: Conventions
    url: conventions/webcrawlerapi-com-conventions.yml
  - type: ErrorCatalog
    url: errors/webcrawlerapi-com-problem-types.yml
  - type: DataModel
    url: data-model/webcrawlerapi-com-data-model.yml
  - type: Webhooks
    url: asyncapi/webcrawlerapi-com-webhooks.yml
  - type: RateLimits
    url: rate-limits/webcrawlerapi-com-rate-limits.yml
  - type: Postman
    url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
- aid: webcrawlerapi-com:webcrawlerapi-com-feeds-api
  name: WebCrawlerAPI Feeds API
  description: The Feeds API from WebCrawlerAPI — 9 operation(s) for feeds.
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com
  tags:
  - Feed
  tags_raw:
  - Feeds
  properties:
  - type: OpenAPI
    url: openapi/webcrawlerapi-com-feeds-api-openapi.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/getting-started
  - type: APIReference
    url: https://api.webcrawlerapi.com/swagger/index.html
  - type: APIReference
    url: https://webcrawlerapi.com/docs/api/crawl
  - type: Authentication
    url: authentication/webcrawlerapi-com-authentication.yml
  - type: Conventions
    url: conventions/webcrawlerapi-com-conventions.yml
  - type: ErrorCatalog
    url: errors/webcrawlerapi-com-problem-types.yml
  - type: DataModel
    url: data-model/webcrawlerapi-com-data-model.yml
  - type: Webhooks
    url: asyncapi/webcrawlerapi-com-webhooks.yml
  - type: RateLimits
    url: rate-limits/webcrawlerapi-com-rate-limits.yml
  - type: Postman
    url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
- aid: webcrawlerapi-com:webcrawlerapi-com-jobs-api
  name: WebCrawlerAPI Jobs API
  description: The Jobs API from WebCrawlerAPI — 6 operation(s) for jobs.
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com
  tags:
  - Job
  tags_raw:
  - Jobs
  properties:
  - type: OpenAPI
    url: openapi/webcrawlerapi-com-jobs-api-openapi.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/getting-started
  - type: APIReference
    url: https://api.webcrawlerapi.com/swagger/index.html
  - type: APIReference
    url: https://webcrawlerapi.com/docs/api/crawl
  - type: Authentication
    url: authentication/webcrawlerapi-com-authentication.yml
  - type: Conventions
    url: conventions/webcrawlerapi-com-conventions.yml
  - type: ErrorCatalog
    url: errors/webcrawlerapi-com-problem-types.yml
  - type: DataModel
    url: data-model/webcrawlerapi-com-data-model.yml
  - type: Webhooks
    url: asyncapi/webcrawlerapi-com-webhooks.yml
  - type: RateLimits
    url: rate-limits/webcrawlerapi-com-rate-limits.yml
  - type: Postman
    url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
- aid: webcrawlerapi-com:webcrawlerapi-com-organization-api
  name: WebCrawlerAPI Organization API
  description: The Organization API from WebCrawlerAPI — 1 operation(s) for organization.
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com
  tags:
  - Organization
  properties:
  - type: OpenAPI
    url: openapi/webcrawlerapi-com-organization-api-openapi.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/getting-started
  - type: APIReference
    url: https://api.webcrawlerapi.com/swagger/index.html
  - type: APIReference
    url: https://webcrawlerapi.com/docs/api/crawl
  - type: Authentication
    url: authentication/webcrawlerapi-com-authentication.yml
  - type: Conventions
    url: conventions/webcrawlerapi-com-conventions.yml
  - type: ErrorCatalog
    url: errors/webcrawlerapi-com-problem-types.yml
  - type: DataModel
    url: data-model/webcrawlerapi-com-data-model.yml
  - type: Webhooks
    url: asyncapi/webcrawlerapi-com-webhooks.yml
  - type: RateLimits
    url: rate-limits/webcrawlerapi-com-rate-limits.yml
  - type: Postman
    url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
- aid: webcrawlerapi-com:webcrawlerapi-com-status-api
  name: WebCrawlerAPI Status API
  description: The Status API from WebCrawlerAPI — 2 operation(s) for status.
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com
  tags:
  - Status
  properties:
  - type: OpenAPI
    url: openapi/webcrawlerapi-com-status-api-openapi.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/getting-started
  - type: APIReference
    url: https://api.webcrawlerapi.com/swagger/index.html
  - type: APIReference
    url: https://webcrawlerapi.com/docs/api/crawl
  - type: Authentication
    url: authentication/webcrawlerapi-com-authentication.yml
  - type: Conventions
    url: conventions/webcrawlerapi-com-conventions.yml
  - type: ErrorCatalog
    url: errors/webcrawlerapi-com-problem-types.yml
  - type: DataModel
    url: data-model/webcrawlerapi-com-data-model.yml
  - type: Webhooks
    url: asyncapi/webcrawlerapi-com-webhooks.yml
  - type: RateLimits
    url: rate-limits/webcrawlerapi-com-rate-limits.yml
  - type: Postman
    url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
- aid: webcrawlerapi-com:webcrawlerapi-com-v2-scraping-api
  name: WebCrawlerAPI V2 Scraping API
  description: The V2 Scraping API from WebCrawlerAPI — 2 operation(s) for v2 scraping.
  humanURL: https://webcrawlerapi.com/docs/getting-started
  baseURL: https://api.webcrawlerapi.com
  tags:
  - V2 Scraping
  properties:
  - type: OpenAPI
    url: openapi/webcrawlerapi-com-v2-scraping-api-openapi.yml
  - type: Documentation
    url: https://webcrawlerapi.com/docs/getting-started
  - type: APIReference
    url: https://api.webcrawlerapi.com/swagger/index.html
  - type: APIReference
    url: https://webcrawlerapi.com/docs/api/crawl
  - type: Authentication
    url: authentication/webcrawlerapi-com-authentication.yml
  - type: Conventions
    url: conventions/webcrawlerapi-com-conventions.yml
  - type: ErrorCatalog
    url: errors/webcrawlerapi-com-problem-types.yml
  - type: DataModel
    url: data-model/webcrawlerapi-com-data-model.yml
  - type: Webhooks
    url: asyncapi/webcrawlerapi-com-webhooks.yml
  - type: RateLimits
    url: rate-limits/webcrawlerapi-com-rate-limits.yml
  - type: Postman
    url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com
- FN: APIs.json
  email: info@apis.io
common:
- type: Website
  url: https://webcrawlerapi.com/
- type: DeveloperPortal
  url: https://webcrawlerapi.com/docs/getting-started
- type: Documentation
  url: https://webcrawlerapi.com/docs/getting-started
- type: APIReference
  url: https://api.webcrawlerapi.com/swagger/index.html
- type: GettingStarted
  url: https://webcrawlerapi.com/docs/getting-started
- type: Support
  url: mailto:support@webcrawlerapi.com
- type: Blog
  url: https://webcrawlerapi.com/blog
- type: BlogRSS
  url: https://webcrawlerapi.com/rss.xml
- type: ChangeLog
  url: https://webcrawlerapi.com/changelog
- type: ChangeLog
  url: changelog/webcrawlerapi-com-changelog.yml
- type: StatusPage
  url: https://status.webcrawlerapi.com
- type: GitHubOrganization
  url: https://github.com/WebCrawlerAPI
- type: Twitter
  url: https://x.com/Webcrawlerapihq
- type: Pricing
  url: https://webcrawlerapi.com/docs/pricing
- type: SignUp
  url: https://dash.webcrawlerapi.com/sign-up
- type: Login
  url: https://dash.webcrawlerapi.com/sign-in
- type: TermsOfService
  url: https://webcrawlerapi.com/tos
- type: PrivacyPolicy
  url: https://webcrawlerapi.com/privacy
- type: RefundPolicy
  url: https://webcrawlerapi.com/refund
- type: Leadership
  url: https://webcrawlerapi.com/about
- type: Postman
  url: https://www.postman.com/webcrawlerapi/webcrawlerapi-public-workspace/collection/xz2fs5u/webcrawlerapi
- type: LLMsTxt
  url: llms/webcrawlerapi-com-llms.txt
- type: LLMsTxt
  name: llms.txt (provider-hosted)
  url: https://webcrawlerapi.com/llms.txt
- type: LLMsTxt
  name: docs llms.txt (provider-hosted)
  url: https://webcrawlerapi.com/docs/llms.txt
- type: WellKnown
  url: well-known/webcrawlerapi-com-well-known.yml
- type: APICatalog
  url: https://webcrawlerapi.com/.well-known/api-catalog
- type: AgentCard
  url: a2a/webcrawlerapi-com-a2a.yml
- type: MCPServer
  url: mcp/webcrawlerapi-com-mcp.yml
- type: ToolCrosswalk
  url: mcp/webcrawlerapi-com-tool-crosswalk.yml
- type: AgentSkill
  url: skills/_index.yml
- type: Packages
  url: packages/webcrawlerapi-com-packages.yml
- type: SDKs
  url: packages/webcrawlerapi-com-packages.yml
- type: CLI
  url: cli/webcrawlerapi-com-cli.yml
- type: Authentication
  url: authentication/webcrawlerapi-com-authentication.yml
- type: Conventions
  url: conventions/webcrawlerapi-com-conventions.yml
- type: ErrorCatalog
  url: errors/webcrawlerapi-com-problem-types.yml
- type: DataModel
  url: data-model/webcrawlerapi-com-data-model.yml
- type: Webhooks
  url: asyncapi/webcrawlerapi-com-webhooks.yml
- type: RateLimits
  url: rate-limits/webcrawlerapi-com-rate-limits.yml
- type: Plans
  url: plans/webcrawlerapi-com-plans-pricing.yml
- type: Lifecycle
  url: lifecycle/webcrawlerapi-com-lifecycle.yml
- type: Conformance
  url: conformance/webcrawlerapi-com-conformance.yml
- type: Overlay
  url: overlays/webcrawlerapi-com-openapi-overlay.yaml
- type: DomainSecurity
  url: security/webcrawlerapi-com-domain-security.yml
- type: Subprocessors
  url: https://webcrawlerapi.com/legal/subprocessor-list
- type: NoticeAndAction
  url: https://webcrawlerapi.com/tos
x-enrichment:
  date: '2026-09-19'
  status: enriched
  artifacts_added: 30
  pass: local-v3

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/webcrawlerapi-com"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/webcrawlerapi-com/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/webcrawlerapi-com/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.