Spider

Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown, structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies, headless browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.

Spider publishes 9 APIs on the APIs.io network, including Crawling API, Data API, Fetch API, and 6 more. Tagged areas include Crawling, Scraping, Data Extraction, URLs, and Artificial Intelligence.

Spider’s developer surface includes authentication, documentation, API reference, pricing, support, engineering blog, and 16 more developer resources.

🌐 Visit website 📡 Source on GitHub

48.3/100 developing ▬ flat Agent 23/100 agent aware Full breakdown ↓
scored 2026-08-25 · rubric v0.14.0

AccessFreemiumSelf serve⚡ Free to try

10 APIs 1 MCP Servers

Crawling Scraping Data Extraction URLs Artificial Intelligence Markdown MCP Rust Headless Browser Proxies

Kin Score

Kin Score How this is scored →
scored 2026-08-25 · rubric v0.14.0

Composite quality — 48.3/100 · developing

Contract Quality 14.0 / 25

Developer Ergonomics 10.5 / 20

Access Clarity 10.0 / 20

Operational Transparency 6.5 / 13

Contract Governance 0.0 / 12

Discoverability 7.4 / 10

Agent readiness — 23/100 · agent aware

Machine-Readable Contract 18 / 18

Agentic Access Contract 10 / 10

Documented Reversibility 0 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 0 / 9

Stable Error Semantics 0 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 7 / 7

Typed Event Surface 0 / 6

Agent Skills 0 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Delegated User Identity 0 / 6

Protected Resource Metadata 0 / 5

Registration Without a Human 0 / 6

Agentic Commerce Surface 0 / 5

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. This rating is computed from github.com/api-evangelist/spider-cloud: open an issue to ask a question, or submit a pull request to add artifacts. Want it done for you? Prioritized profiling — $2,500 →

APIs 10

Individual APIs this provider publishes, each with its own machine-readable definition.

Open Collections 11

Open, tool-agnostic API collections (OpenAPI-derived and Bruno).

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

Rate Limits 1

Documented rate limits and quota policies.

Spider Cloud Rate Limits

4 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Documentation 3

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Guides

Guides

Agent Surfaces 2

MCP servers, agent skills, and machine-readable catalogs

AgenticAccess

AgenticAccess

MCPServer

MCPServer

Build 6

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

SourceCode

SDKs

SDKs

SDKs

SDKs

Access & Security 2

Authentication, authorization, and security posture

DomainSecurity

DomainSecurity

Authentication

Authentication

Operate 3

Status, limits, changes, and where to get help

StatusPage

StatusPage

Support

Support

RateLimits

RateLimits

Commercial 4

Pricing, plans, and the legal terms of use

License

Pricing

Plans

FinOps

Company 2

The organization behind the API

Website

Website

Blog

Blog

Source (apis.yml)

aid: spider-cloud
name: Spider
description: Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown,
  structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies, headless
  browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.
kind: contract
accessModel:
  pricing: freemium
  onboarding: self-serve
  trial: false
  try_now: true
  public: false
  label: Freemium · Self-serve signup
  confidence: high
  source:
  - plans
  - authentication
  generated: '2026-07-22'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/spider-cloud.png
access: 3rd-Party
created: '2026-05-25'
modified: '2026-05-25'
position: Consuming
tags:
- Crawling
- Scraping
- Data Extraction
- URLs
- Artificial Intelligence
- Markdown
- MCP
- Rust
- Headless Browser
- Proxies
tags_raw:
- Crawling
- Scraping
- Data Extraction
- URLs
- AI
- Markdown
- MCP
- Rust
- Headless Browser
- Proxies
url: https://raw.githubusercontent.com/api-evangelist/spider-cloud/refs/heads/main/apis.yml
specificationVersion: '0.23'
apis:
- aid: spider-cloud:spider-mcp
  name: Spider Cloud MCP Server
  description: Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations (crawl, scrape, search,
    links, screenshot, unblocker, transform, get_credits), five AI-routed variants, and nine browser-automation tools (open,
    navigate, click, fill, screenshot, content, evaluate, wait_for, close) — to any MCP-compatible client. Authenticated with
    a Bearer API key. Also runnable locally via `npx spider-cloud-mcp`.
  humanURL: https://spider.cloud/docs/integrations/mcp
  baseURL: https://mcp.spider.cloud/mcp
  tags:
  - MCP
  - Artificial Intelligence
  - Agents
  - Crawling
  - Scraping
  - Browser Automation
  tags_raw:
  - MCP
  - AI
  - Agents
  - Crawling
  - Scraping
  - Browser Automation
  properties:
  - type: Documentation
    url: https://spider.cloud/docs/integrations/mcp
  - type: SourceCode
    url: https://github.com/spider-rs/spider-cloud-mcp-v2
  - type: Authentication
    url: https://spider.cloud/docs/integrations/mcp
- aid: spider-cloud:spider-cloud-crawling-api
  name: Spider Crawling API
  description: Recursively crawl entire websites and collect every page.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Crawling
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-crawling-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-data-api
  name: Spider Data API
  description: Account data — scraper directory, crawl logs, credits balance.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Data
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-data-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-fetch-api
  name: Spider Fetch API
  description: Per-website APIs with AI-discovered configurations.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Fetch
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-fetch-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-links-api
  name: Spider Links API
  description: Collect all links from a website.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Links
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-links-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-scraping-api
  name: Spider Scraping API
  description: Extract content from individual web pages.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Scraping
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-scraping-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-screenshot-api
  name: Spider Screenshot API
  description: Capture full-page or viewport screenshots.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Screenshot
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-screenshot-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-search-api
  name: Spider Search API
  description: Search the web and crawl results.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Search
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-search-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-transform-api
  name: Spider Transform API
  description: Convert raw HTML or PDF into clean output (markdown, JSON, text).
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Transforms
  tags_raw:
  - Transform
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-transform-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-unblocker-api
  name: Spider Unblocker API
  description: Access content behind anti-bot protections.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Unblocker
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-unblocker-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
common:
- type: License
  name: MIT
  url: https://github.com/spider-rs/spider-cloud-mcp-v2/blob/main/LICENSE
- type: AgenticAccess
  url: agentic-access/spider-cloud-agentic-access.yml
- type: DomainSecurity
  url: security/spider-cloud-domain-security.yml
- type: Authentication
  url: authentication/spider-cloud-authentication.yml
- type: Website
  url: https://spider.cloud
- type: Documentation
  url: https://spider.cloud/docs/overview
- type: APIReference
  url: https://spider.cloud/docs/api
- type: Guides
  url: https://spider.cloud/guides
- type: Pricing
  url: https://spider.cloud/pricing
- type: StatusPage
  url: https://spidercloud.statuspage.io
- type: GitHubOrganization
  url: https://github.com/spider-rs
- type: SourceCode
  url: https://github.com/spider-rs/spider
- type: SDKs
  url: https://github.com/spider-rs/spider-clients
- type: SDKs
  url: https://github.com/spider-rs/spider-py
- type: SDKs
  url: https://github.com/spider-rs/spider-nodejs
- type: SDKs
  url: https://docs.rs/spider
- type: MCPServer
  url: https://mcp.spider.cloud/mcp
- type: Integrations
  url: https://spider.cloud/docs/integrations/mcp
- type: Integrations
  url: https://spider.cloud/docs/integrations/langchain
- type: Integrations
  url: https://spider.cloud/docs/integrations/llamaindex
- type: Integrations
  url: https://spider.cloud/docs/integrations/crewai
- type: Integrations
  url: https://spider.cloud/docs/integrations/flowiseai
- type: Integrations
  url: https://spider.cloud/docs/integrations/agno
- type: Integrations
  url: https://spider.cloud/docs/integrations/zapier
- type: Integrations
  url: https://spider.cloud/docs/integrations/x402
- type: Support
  url: mailto:support@spider.cloud
- type: Plans
  url: plans/spider-cloud-plans-pricing.yml
- type: RateLimits
  url: rate-limits/spider-cloud-rate-limits.yml
- type: FinOps
  url: finops/spider-cloud-finops.yml
- url: https://spider.cloud/blog
  type: Blog
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/spider-cloud"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/spider-cloud/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/spider-cloud/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no email required.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

Spider

APIs 10

Open Collections 11

MCP Servers 1

Pricing Plans 1

Rate Limits 1

FinOps 1

Security Posture 2

Agentic Access 1

Documentation 3

Agent Surfaces 2

Build 6

Access & Security 2

Operate 3

Commercial 4

Company 2

Source (apis.yml)

MCP server

Tools for providers

Call it yourself

Get an API key