Spider

aid: spider-cloud
name: Spider
description: >-
  Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown,
  structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies,
  headless browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.
kind: contract
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/apis-json-logo.jpg
access: 3rd-Party
created: '2026-05-25'
modified: '2026-05-25'
position: Consuming
tags:
  - Crawling
  - Scraping
  - Data Extraction
  - URLs
  - AI
  - Markdown
  - MCP
  - Rust
  - Headless Browser
  - Proxies
url: https://raw.githubusercontent.com/api-evangelist/spider-cloud/refs/heads/main/apis.yml
specificationVersion: '0.19'
apis:
  - aid: spider-cloud:spider-api
    name: Spider API
    description: >-
      The Spider Cloud REST API exposes core /crawl, /scrape, /search, /links, /screenshot, /unblocker, and /transform
      endpoints, plus AI-discovered per-domain /fetch endpoints, residential/ISP/mobile proxy routing via
      proxy.spider.cloud, CDP/WebDriver BiDi cloud browsers, scraper directory lookups, crawl logs, and a credits
      endpoint. All endpoints accept and return JSON (also XML, CSV, JSONL via content-type) and authenticate with
      Bearer API keys. Up to 10,000 RPM per account.
    humanURL: https://spider.cloud/docs/api
    baseURL: https://api.spider.cloud
    tags:
      - Crawling
      - Scraping
      - Data Extraction
      - URLs
      - AI
      - Markdown
      - Search
      - Screenshot
      - Headless Browser
      - Proxies
    properties:
      - type: Documentation
        url: https://spider.cloud/docs/api
      - type: Documentation
        url: https://spider.cloud/docs/overview
      - type: OpenAPI
        url: openapi/spider-cloud-openapi.yml
      - type: Authentication
        url: https://spider.cloud/docs/api
      - type: RateLimits
        url: https://spider.cloud/docs/api
  - aid: spider-cloud:spider-mcp
    name: Spider Cloud MCP Server
    description: >-
      Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations (crawl, scrape, search,
      links, screenshot, unblocker, transform, get_credits), five AI-routed variants, and nine browser-automation tools
      (open, navigate, click, fill, screenshot, content, evaluate, wait_for, close) — to any MCP-compatible client.
      Authenticated with a Bearer API key. Also runnable locally via `npx spider-cloud-mcp`.
    humanURL: https://spider.cloud/docs/integrations/mcp
    baseURL: https://mcp.spider.cloud/mcp
    tags:
      - MCP
      - AI
      - Agents
      - Crawling
      - Scraping
      - Browser Automation
    properties:
      - type: Documentation
        url: https://spider.cloud/docs/integrations/mcp
      - type: SourceCode
        url: https://github.com/spider-rs/spider-cloud-mcp-v2
      - type: Authentication
        url: https://spider.cloud/docs/integrations/mcp
common:
  - type: Website
    url: https://spider.cloud
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: APIReference
    url: https://spider.cloud/docs/api
  - type: Guides
    url: https://spider.cloud/guides
  - type: Pricing
    url: https://spider.cloud/pricing
  - type: StatusPage
    url: https://spidercloud.statuspage.io
  - type: GitHubOrganization
    url: https://github.com/spider-rs
  - type: SourceCode
    url: https://github.com/spider-rs/spider
  - type: SDKs
    url: https://github.com/spider-rs/spider-clients
  - type: SDK
    url: https://github.com/spider-rs/spider-py
  - type: SDK
    url: https://github.com/spider-rs/spider-nodejs
  - type: SDK
    url: https://docs.rs/spider
  - type: MCP
    url: https://mcp.spider.cloud/mcp
  - type: Integrations
    url: https://spider.cloud/docs/integrations/mcp
  - type: Integration
    url: https://spider.cloud/docs/integrations/langchain
  - type: Integration
    url: https://spider.cloud/docs/integrations/llamaindex
  - type: Integration
    url: https://spider.cloud/docs/integrations/crewai
  - type: Integration
    url: https://spider.cloud/docs/integrations/flowiseai
  - type: Integration
    url: https://spider.cloud/docs/integrations/agno
  - type: Integration
    url: https://spider.cloud/docs/integrations/zapier
  - type: Integration
    url: https://spider.cloud/docs/integrations/x402
  - type: Support
    url: mailto:support@spider.cloud
  - type: Plans
    url: plans/spider-cloud-plans-pricing.yml
  - type: RateLimits
    url: rate-limits/spider-cloud-rate-limits.yml
  - type: FinOps
    url: finops/spider-cloud-finops.yml
maintainers:
  - FN: Kin Lane
    email: kin@apievangelist.com