Spider website screenshot

Spider

Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown, structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies, headless browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.

Spider publishes 9 APIs on the APIs.io network, including Crawling API, Data API, Fetch API, and 6 more. Tagged areas include Crawling, Scraping, Data Extraction, URLs, and AI.

Spider’s developer surface includes authentication, documentation, API reference, pricing, support, engineering blog, and 15 more developer resources.

🌐 Visit website 📡 Source on GitHub
49.7/100 developing ▬ flat Agent 31/100 agent aware Full breakdown ↓
scored 2026-08-05 · rubric v0.9.1
AccessFreemiumSelf serve⚡ Free to try
10 APIs
CrawlingScrapingData ExtractionURLsAIMarkdownMCPRustHeadless BrowserProxies

Kin Score

Kin Score Kin Score How this is scored →
scored 2026-08-05 · rubric v0.9.1
Composite quality — 49.7/100 · developing
Contract Quality 15.9 / 25
Developer Ergonomics 9.6 / 20
Commercial Clarity 10.0 / 20
Operational Transparency 6.8 / 13
Governance 0.0 / 12
Discoverability 7.4 / 10
Agent readiness — 31/100 · agent aware
Machine-Readable Contract 18 / 18
Agentic Access Contract 10 / 10
MCP Server 0 / 12
Machine-Readable Auth 10 / 10
Idempotency 0 / 9
Stable Error Semantics 0 / 8
Request/Response Examples 0 / 7
Rate-Limit Signaling 7 / 7
Typed Event Surface 0 / 6
Agent Skills 0 / 5
Well-Known Catalog 0 / 4
Consent & Bot Identity 0 / 3
A2A Agent Card 0 / 8
Dry-Run / Simulate Mode 0 / 4
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. This rating is computed from github.com/api-evangelist/spider-cloud: open an issue to ask a question, or submit a pull request to add artifacts. Want it done for you? Prioritized profiling — $2,500 →

APIs 10

Individual APIs this provider publishes, each with its own machine-readable definition.

Spider Cloud MCP Server

Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations (crawl, scrape, search, links, screenshot, unblocker, transform, get_credits), five AI-rou...

Spider Crawling API

Recursively crawl entire websites and collect every page.

Spider Data API

Account data — scraper directory, crawl logs, credits balance.

Spider Fetch API

Per-website APIs with AI-discovered configurations.

Spider Links API

Collect all links from a website.

Spider Scraping API

Extract content from individual web pages.

Spider Screenshot API

Capture full-page or viewport screenshots.

Spider Search API

Search the web and crawl results.

Spider Transform API

Convert raw HTML or PDF into clean output (markdown, JSON, text).

Spider Unblocker API

Access content behind anti-bot protections.

Scroll for all 10

Open Collections 1

Open, tool-agnostic API collections (OpenAPI-derived and Bruno).

Spider Cloud API

OPEN COLLECTION

Pricing Plans 1

Published pricing tiers and plan structures.

Spider Cloud Plans Pricing

4 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Spider Cloud Rate Limits

4 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Spider Cloud Finops

FINOPS

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Spider Cloud Authentication

http · 1 scheme

SECURITY

Spider Cloud Domain Security

TLSv1.3 · HSTS

SECURITY

Agentic Access 1

Recommended x-agentic-access execution contracts for AI agents.

Spider Cloud Agentic Access

11 operations · 8 acting

11 operations · 8 acting

AGENTIC

Resources

Documentation 3

Reference material describing how the API behaves

Documentation
Documentation
 APIReference
APIReference
 Guides
Guides

Agent Surfaces 2

MCP servers, agent skills, and machine-readable catalogs

AgenticAccess
AgenticAccess
 MCP
MCP

Build 6

SDKs, sample code, and the tooling you integrate with

GitHubOrganization
GitHubOrganization
 SourceCode
SourceCode
 SDKs
SDKs
 SDKs
SDKs
 SDKs
SDKs
 SDKs
SDKs

Access & Security 2

Authentication, authorization, and security posture

DomainSecurity
DomainSecurity
 Authentication
Authentication

Operate 3

Status, limits, changes, and where to get help

StatusPage
StatusPage
 Support
Support
 RateLimits
RateLimits

Commercial 3

Pricing, plans, and the legal terms of use

Pricing
Pricing
 Plans
Plans
 FinOps
FinOps

Company 2

The organization behind the API

Website
Website
 Blog
Blog

Source (apis.yml)

apis.yml Raw ↑ 
aid: spider-cloud
name: Spider
description: Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown,
  structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies, headless
  browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.
kind: contract
accessModel:
  pricing: freemium
  onboarding: self-serve
  trial: false
  try_now: true
  public: false
  label: Freemium · Self-serve signup
  confidence: high
  source:
  - plans
  - authentication
  generated: '2026-07-22'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/spider-cloud.png
access: 3rd-Party
created: '2026-05-25'
modified: '2026-05-25'
position: Consuming
tags:
- Crawling
- Scraping
- Data Extraction
- URLs
- AI
- Markdown
- MCP
- Rust
- Headless Browser
- Proxies
url: https://raw.githubusercontent.com/api-evangelist/spider-cloud/refs/heads/main/apis.yml
specificationVersion: '0.19'
apis:
- aid: spider-cloud:spider-mcp
  name: Spider Cloud MCP Server
  description: Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations (crawl, scrape, search,
    links, screenshot, unblocker, transform, get_credits), five AI-routed variants, and nine browser-automation tools (open,
    navigate, click, fill, screenshot, content, evaluate, wait_for, close) — to any MCP-compatible client. Authenticated with
    a Bearer API key. Also runnable locally via `npx spider-cloud-mcp`.
  humanURL: https://spider.cloud/docs/integrations/mcp
  baseURL: https://mcp.spider.cloud/mcp
  tags:
  - MCP
  - AI
  - Agents
  - Crawling
  - Scraping
  - Browser Automation
  properties:
  - type: Documentation
    url: https://spider.cloud/docs/integrations/mcp
  - type: SourceCode
    url: https://github.com/spider-rs/spider-cloud-mcp-v2
  - type: Authentication
    url: https://spider.cloud/docs/integrations/mcp
- aid: spider-cloud:spider-cloud-crawling-api
  name: Spider Crawling API
  description: Recursively crawl entire websites and collect every page.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Crawling
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-crawling-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-data-api
  name: Spider Data API
  description: Account data — scraper directory, crawl logs, credits balance.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Data
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-data-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-fetch-api
  name: Spider Fetch API
  description: Per-website APIs with AI-discovered configurations.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Fetch
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-fetch-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-links-api
  name: Spider Links API
  description: Collect all links from a website.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Links
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-links-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-scraping-api
  name: Spider Scraping API
  description: Extract content from individual web pages.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Scraping
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-scraping-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-screenshot-api
  name: Spider Screenshot API
  description: Capture full-page or viewport screenshots.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Screenshot
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-screenshot-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-search-api
  name: Spider Search API
  description: Search the web and crawl results.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Search
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-search-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-transform-api
  name: Spider Transform API
  description: Convert raw HTML or PDF into clean output (markdown, JSON, text).
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Transform
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-transform-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-unblocker-api
  name: Spider Unblocker API
  description: Access content behind anti-bot protections.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Unblocker
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-unblocker-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
common:
- type: AgenticAccess
  url: agentic-access/spider-cloud-agentic-access.yml
- type: DomainSecurity
  url: security/spider-cloud-domain-security.yml
- type: Authentication
  url: authentication/spider-cloud-authentication.yml
- type: Website
  url: https://spider.cloud
- type: Documentation
  url: https://spider.cloud/docs/overview
- type: APIReference
  url: https://spider.cloud/docs/api
- type: Guides
  url: https://spider.cloud/guides
- type: Pricing
  url: https://spider.cloud/pricing
- type: StatusPage
  url: https://spidercloud.statuspage.io
- type: GitHubOrganization
  url: https://github.com/spider-rs
- type: SourceCode
  url: https://github.com/spider-rs/spider
- type: SDKs
  url: https://github.com/spider-rs/spider-clients
- type: SDKs
  url: https://github.com/spider-rs/spider-py
- type: SDKs
  url: https://github.com/spider-rs/spider-nodejs
- type: SDKs
  url: https://docs.rs/spider
- type: MCP
  url: https://mcp.spider.cloud/mcp
- type: Integrations
  url: https://spider.cloud/docs/integrations/mcp
- type: Integrations
  url: https://spider.cloud/docs/integrations/langchain
- type: Integrations
  url: https://spider.cloud/docs/integrations/llamaindex
- type: Integrations
  url: https://spider.cloud/docs/integrations/crewai
- type: Integrations
  url: https://spider.cloud/docs/integrations/flowiseai
- type: Integrations
  url: https://spider.cloud/docs/integrations/agno
- type: Integrations
  url: https://spider.cloud/docs/integrations/zapier
- type: Integrations
  url: https://spider.cloud/docs/integrations/x402
- type: Support
  url: mailto:support@spider.cloud
- type: Plans
  url: plans/spider-cloud-plans-pricing.yml
- type: RateLimits
  url: rate-limits/spider-cloud-rate-limits.yml
- type: FinOps
  url: finops/spider-cloud-finops.yml
- url: https://spider.cloud/blog
  type: Blog
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com