# WebCrawlerAPI

**Canonical:** https://apis.io/providers/webcrawlerapi-com/  
**Website:** https://webcrawlerapi.com/  
**APIs profiled:** 3

WebCrawlerAPI is a web crawling and scraping API from 103 Labs (Netherlands) that turns websites into clean, LLM-ready markdown, cleaned text, HTML or link lists for AI agents, support bots and RAG pipelines. It offers asynchronous multi-page crawl jobs (POST /v1/crawl with polling or webhooks), single-page scraping (POST /v2/scrape, sync or async, with prompt-based structured outputs validated against a JSON Schema), scheduled change-detection Feeds delivered as Atom 1.0, JSON Feed 1.1 or webhooks, a prompt-driven autonomous crawling agent with a required per-run spend cap, and organization usage/balance endpoints. Requests authenticate with a Bearer API key; usage is metered per successfully crawled page on a $0/month pay-as-you-go tier or $29/$99/$499 monthly plans, with 7-day caching (cache hits free). Published surfaces: a Swagger 2.0 document behind a live Swagger UI, an Agent API OpenAPI 3.0.3, an A2A agent card and RFC 9727 API catalog at /.well-known/, llms.txt, SDKs for JavaScript, Python, PHP, .NET and Java, an npm MCP server (local stdio), the webcr CLI, a Claude Code Agent Skill, a public Postman collection and a dated changelog.

## Kin Score — 64.5 / 100 (strong)

Scored 2026-10-09 under rubric 0.23.0. Trend: flat (+0.0 from 64.5).

| Facet | Score |
|---|---|
| Discoverability | 88.3 |
| Contract Quality | 57.2 |
| Contract Governance | 18.2 |
| Operational Transparency | 44.7 |
| Developer Ergonomics | 83.3 |
| Access Clarity | 76.3 |

Regulatory layer — **Horizontal (data, software, accessibility, platform)**: 30.0 (matched via fallback).

## Agent readiness — 39.7 (agent-ready)

| Dimension | Value |
|---|---|
| Spec Presence | yes |
| Agentic Access | no |
| Reversibility Documented | documented |
| MCP Server | documented |
| Auth Clarity | bearer |
| Idempotency | no |
| Error Semantics | verified |
| OpenAPI Examples | no |
| Rate Limit Signal | documented |
| Event Surface Described | yes |
| Agent Skills | yes |
| Well Known Catalog | yes |
| Consent Identity | no |
| Agent Card | no |
| Dry Run Mode | no |
| Delegated Identity | no |
| Protected Resource Metadata | no |
| Dynamic Client Registration | no |
| Agentic Commerce | no |

## APIs (8)

- **WebCrawlerAPI MCP Server** — Official Model Context Protocol server distributed as the npm package webcrawler-mcp (npx -y webcrawler-mcp) and run locally over stdio - or self-hosted over streamable HTTP wit...
- **WebCrawlerAPI Agent Card** — A2A-style agent card served from https://webcrawlerapi.com/.well-known/agent-card.json describing "WebCrawlerAPI Agent" (version 1.0.0) with three skills - scrape, crawl and cra...
- **WebCrawlerAPI Agent API** — The Agent API from WebCrawlerAPI — 3 operation(s) for agent.
- **WebCrawlerAPI Feeds API** — The Feeds API from WebCrawlerAPI — 9 operation(s) for feeds.
- **WebCrawlerAPI Jobs API** — The Jobs API from WebCrawlerAPI — 6 operation(s) for jobs.
- **WebCrawlerAPI Organization API** — The Organization API from WebCrawlerAPI — 1 operation(s) for organization.
- **WebCrawlerAPI Status API** — The Status API from WebCrawlerAPI — 2 operation(s) for status.
- **WebCrawlerAPI V2 Scraping API** — The V2 Scraping API from WebCrawlerAPI — 2 operation(s) for v2 scraping.

## MCP servers (2)

- **WebCrawlerAPI MCP Server** — WebCrawlerAPI ships an official MCP server as an npm package (webcrawler-mcp) that a user runs locally over stdio, or self-hosts over HTTP with USE_HTTP=true. There is NO provid...
- **webcrawler-mcp on npm (local stdio package)**

## Security (2)

- **Webcrawlerapi Com Authentication** — apiKey · 1 scheme
- **Webcrawlerapi Com Domain Security** — TLSv1.3 · DMARC

## Plans (1)

- **Webcrawlerapi Com Plans Pricing**

## Tags

Web Scraping, Web Crawling, Data Extraction, Markdown, AI Agents, LLM, RAG, Web Feeds, Change Detection, Structured Data, MCP, A2A, Agent-Native

---

Profiled by [API Evangelist](https://apievangelist.com) and published on [APIs.io](https://apis.io/providers/webcrawlerapi-com/). Scores are computed from the provider's own public artifacts under a published rubric.