Spider

Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown, structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies, headless browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.

Spider publishes 10 API contracts indexed on the APIs.io network, including Crawling API, Data API, Fetch API, and 7 more. Tagged areas include Crawling, Web Scraping, Data Extraction, URLs, and Artificial Intelligence.

The Spider catalog on APIs.io includes 1 Spectral governance ruleset.

Spider’s developer surface includes authentication, documentation, API reference, pricing, support, engineering blog, and 18 more developer resources.

🌐 Visit website 📡 Profile data on GitHub

32.3/100 thin ▬ flat Agent 44/100 agent ready Front door AI 5/6 prominent Heartbeat 100/100 alive open core · MIT Full breakdown ↓
scored 2026-11-13 · rubric v0.28.0 · Show this score on your site →

AccessFreemiumSelf serve⚡ Free to try

1 published contract 1 MCP Servers

Crawling Web Scraping Data Extraction URLs Artificial Intelligence Markdown MCP Rust Browser Automation Proxies

Kin Score

Kin Score How this is scored →
scored 2026-11-13 · rubric v0.28.0

Composite quality — 32.3/100 · thin

Contract Quality 8.2 / 16

Developer Ergonomics 6.8 / 13

Access Clarity 2.2 / 13

Operational Transparency 2.2 / 8

Contract Governance 1.1 / 8

Discoverability 4.4 / 7

Regulatory Posture 0.8 / 15

Open Source Surface 0.0 / 10

Create-or-Update Ergonomics 0.0 / 10

Agent readiness — 44/100 · agent ready

Machine-Readable Contract 18 / 18

Agentic Access Contract 10 / 10

Documented Reversibility 0 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 0 / 9

Stable Error Semantics 0 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 7 / 7

Typed Event Surface 0 / 6

Agent Skills 0 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Delegated User Identity 6 / 6

Protected Resource Metadata 5 / 5

Registration Without a Human 6 / 6

Agentic Commerce Well-Known Document 0 / 5

Pagination 0 / 5

Distinct Operations 3 / 3

MCP Tool Quality 6 / 6

Regulatory Posture applies to this provider. Its tags matched the Horizontal (data, software, accessibility, platform) regime, so Regulatory Posture carries 15 points of the composite. If this regime is wrong for your business, say so on your provider repo — the applicability map is public and we will correct it.

Open Source Surface applies to this provider. This product is open source and we read its repository directly, so Open Source Surface carries 10 points of the composite. It is scored from what the repository actually publishes — a security policy, a contribution guide, a release history, a code of conduct — read live from the provider rather than inferred from our own catalog pointers. This facet adds; nothing was taken away to make room for it. An open-source project is not excused from the commercial facets, because exemption would strip it of the points it does earn. If we have the wrong repository, or this product is not open source, say so on your provider repo and we will drop the facet rather than have you publish against it.

Create-or-Update Ergonomics applies to this provider. This API accepts writes, so it carries 10 points of the composite. It is scored from the published contracts themselves: whether a caller can create-or-update in one call, whether the write accepts a key the caller already holds, and whether the response says which branch ran. Without that, every write needs a search-and-branch in front of it, and the first time that check is skipped a duplicate record is created. Scored against the observed mean rather than raw — a provider at the catalog average is unchanged by this facet, not penalised by it.

The six quality facets above are damped to 65 points between them, because both conditional facets apply and carry 35 points together. That is why each facet's contribution is shown against a damped maximum: raising a quality facet moves the composite by 65% of its nominal weight, not 100%. The full arithmetic is at apis.io/rating/.

Show this score on your site.

It updates itself when the score changes. Paste this where you link to your API docs:

<a href="https://apis.io/providers/spider-cloud/"><img src="https://apis.io/badge/spider-cloud/card.svg" alt="Spider on APIs.io — Kin Score by API Evangelist"></a>

Other styles, a README pill and a shields.io endpoint are on the badge page.

Raise your score.

Agent readiness — checks not met, largest first (exact, from this profile):

Idempotency +9 pts — OpenAPI mutating operations declare an idempotency key — a header, a request-body property, or a named de-duplication token whose own description says so (verified); else common[].type matches /idempoten/i (documented) Ask about this →
Stable Error Semantics +8 pts — OpenAPI $refs a shared error schema across 4xx/5xx responses (verified), else common[].type includes ErrorCatalog / ErrorCodes / DeclineCodes Ask about this →
A2A Agent Card +8 pts — a2a/-a2a.yml manifest, graded on conformance.grade against A2A 1.0.0 Ask about this →
Request/Response Examples +7 pts — share of operations carrying request/response examples; else an Examples / Code Samples link Ask about this →
Typed Event Surface +6 pts — AsyncAPI document, OR webhooks/callbacks objects in the as-published OpenAPI, OR common[].type includes Webhooks Ask about this →
Documented Reversibility +6 pts — a reversal operation in the contract, graded up when a window is documented too Ask about this →

6 more unmet checks in the Agent Readiness table above.

Kin Score — facets with the most composite points still available (weight × headroom):

Access Clarity scores 17.0 — up to +16.6 composite points Ask about this →
Contract Quality scores 50.2 — up to +12.5 composite points Ask about this →
Contract Governance scores 13.6 — up to +10.4 composite points Ask about this →

Each check links to what satisfies it. Publish the artifact, or tell us where it already lives, and the score moves on the next scoring run.

Not sure how this applies to your stack? Ask about any one item. Kin Lane reads every question, looks at your profile and your situation, and replies with what a fix would take — scoped to that one area, priced up front, yours whether you ship it or not. Ask a question →

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from the provider profile at artifacts.apis.io/spider-cloud/apis.yml: open an issue on github.com/api-evangelist/spider-cloud to ask a question or point us at an artifact you publish. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

Standards implemented 5

Standards this provider implements or carries. De facto ones are profiled and measured by the API Commons; formal ones come from the standards directory and are linked by the provider's own tags. The evidence column says how each claim was established — not that it was made.

Markdown

tagged — the provider's own APIs.json carries the standard as a tag; implementation not verified

Model Context Protocol (MCP)

measured — observed by APIs.io (verified)

OAuth 2.0 Protected Resource Metadata

measured — observed by APIs.io (verified)

Dynamic Client Registration

measured — observed by APIs.io (published)

OpenAPI

measured — observed by APIs.io (published)

APIs 10

Individual APIs this provider publishes, each with its own machine-readable definition.

Spider Cloud MCP Server

Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations (crawl, scrape, search, links, screenshot, unblocker, transform, get_credits), five AI-rou...

Spider Crawling API

Recursively crawl entire websites and collect every page.

Spider Data API

Account data — scraper directory, crawl logs, credits balance.

Spider Fetch API

Per-website APIs with AI-discovered configurations.

Spider Links API

Collect all links from a website.

Spider Scraping API

Extract content from individual web pages.

Spider Screenshot API

Capture full-page or viewport screenshots.

Spider Search API

Search the web and crawl results.

Spider Transform API

Convert raw HTML or PDF into clean output (markdown, JSON, text).

Spider Unblocker API

Access content behind anti-bot protections.

Scroll for all 10

Open Collections 10

Open, tool-agnostic API collections (OpenAPI-derived and Bruno).

Spider Cloud Crawling API

OPEN COLLECTION

Spider Cloud Crawling Data API

OPEN COLLECTION

Spider Cloud Crawling Fetch API

OPEN COLLECTION

Spider Cloud Crawling Links API

OPEN COLLECTION

Spider Cloud Crawling Scraping API

OPEN COLLECTION

Spider Cloud Crawling Screenshot API

OPEN COLLECTION

Spider Cloud Crawling Search API

OPEN COLLECTION

Spider Cloud Crawling Transform API

OPEN COLLECTION

Spider Cloud Crawling Unblocker API

OPEN COLLECTION

Spider Cloud API

OPEN COLLECTION

Scroll for all 10

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

Spider MCP Server

MCP SERVER

Spider API pricing

Not yet checked against Spider's own pricing page. Compiled by API Evangelist on 2026-05-25; confirm before you rely on a figure. A free plan or tier is listed.

Plan	Price
Free Credits freemium	Free
Pay As You Go paid	$1.00 per bandwidth · $0.001 per compute
AI Tier subscription	—
Enterprise / Custom enterprise	—

Plan details, limits and sources →

Rate limits

Default Core API RPM: 10000 requests per minute
Browser Session Concurrency: 5 concurrent sessions
Browser Session Idle Timeout: 5 idle minutes

Compiled by API Evangelist, not yet checked against Spider's documentation. All 4 limits →

Pricing Plans 1

Published pricing tiers and plan structures.

Spider Plans Pricing

4 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Spider Rate Limits

4 limits

RATE LIMITS

Data License 1

How its data and outputs may be reused, quoted from its own terms.

Spider

2 quoted statements, read from Spider's own page on 2026-11-10.

LICENSE

FinOps 1

Cost, billing, and metering signals for API financial operations.

Spider Finops

FINOPS

Spectral Rules 1

Spectral governance rulesets for linting and validating these APIs. Counts are the severity each rule declares, not results of running them.

Spider API Rules

69 rules · 67 error-severity 1 warn-severity 1 info-severity

SPECTRAL

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Spider Authentication

http · 1 scheme

SECURITY

Spider Domain Security

TLSv1.3 · HSTS

SECURITY

Agentic Access (API Evangelist recommendations) 1

Recommended x-agentic-access execution contracts for AI agents.

Spider Agentic Access

11 operations · 6 acting

AGENTIC

Resources

Documentation 3

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Guides

Guides

Agent Surfaces 2

MCP servers, agent skills, and machine-readable catalogs

AgenticAccess

AgenticAccess

MCPServer

MCPServer

Build 6

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

SourceCode

SDKs

SDKs

SDKs

SDKs

Access & Security 2

Authentication, authorization, and security posture

DomainSecurity

DomainSecurity

Authentication

Authentication

Operate 4

Status, limits, changes, and where to get help

IssueTracker

StatusPage

Support

RateLimits

Commercial 5

Pricing, plans, and the legal terms of use

License

Pricing

Plans

FinOps

DataLicense

Company 2

The organization behind the API

Website

Website

Blog

Blog

Source (apis.yml)

aid: spider-cloud
name: Spider
description: Spider is a Rust-based, AI-friendly web scraping and crawling cloud. Point it at a URL and get back clean markdown,
  structured JSON, screenshots, or links — at up to 100K pages per second — with anti-bot bypass, residential proxies, headless
  browsers, and native MCP, LangChain, LlamaIndex, CrewAI, and AutoGen integrations.
kind: contract
deliveryModel:
  model: open-core
  license: MIT
  open_source: true
  commercial: true
  callable_host: true
  label: Open core · an OSS project plus a commercial hosted product
  confidence: high
  source:
  - license
  - openapi
  - pricing
  generated: '2026-08-28'
  method: derived
accessModel:
  pricing: freemium
  onboarding: self-serve
  trial: false
  try_now: true
  public: false
  gated: none
  gated_source:
  - onboarding
  label: Freemium · Self-serve signup
  confidence: medium
  source:
  - plans
  - authentication
  - security
  generated: '2026-11-13'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/spider-cloud.png
access: 3rd-Party
created: '2026-05-25'
modified: '2026-05-25'
position: Consuming
tags:
- Crawling
- Web Scraping
- Data Extraction
- URLs
- Artificial Intelligence
- Markdown
- MCP
- Rust
- Browser Automation
- Proxies
tags_raw:
- Crawling
- Scraping
- Data Extraction
- URLs
- AI
- Markdown
- MCP
- Rust
- Headless Browser
- Proxies
url: https://raw.githubusercontent.com/api-evangelist/spider-cloud/refs/heads/main/apis.yml
specificationVersion: '0.23'
apis:
- aid: spider-cloud:spider-mcp
  name: Spider Cloud MCP Server
  description: Spider's hosted Model Context Protocol server exposes 22 tools — eight core operations (crawl, scrape, search,
    links, screenshot, unblocker, transform, get_credits), five AI-routed variants, and nine browser-automation tools (open,
    navigate, click, fill, screenshot, content, evaluate, wait_for, close) — to any MCP-compatible client. Authenticated with
    a Bearer API key. Also runnable locally via `npx spider-cloud-mcp`.
  humanURL: https://spider.cloud/docs/integrations/mcp
  baseURL: https://mcp.spider.cloud/mcp
  tags:
  - MCP
  - Artificial Intelligence
  - Agents
  - Crawling
  - Web Scraping
  - Browser Automation
  tags_raw:
  - MCP
  - AI
  - Agents
  - Crawling
  - Scraping
  - Browser Automation
  properties:
  - type: Documentation
    url: https://spider.cloud/docs/integrations/mcp
  - type: SourceCode
    url: https://github.com/spider-rs/spider-cloud-mcp-v2
  - type: Authentication
    url: https://spider.cloud/docs/integrations/mcp
- aid: spider-cloud:spider-cloud-crawling-api
  name: Spider Crawling API
  description: Recursively crawl entire websites and collect every page.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Crawling
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-crawling-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-data-api
  name: Spider Data API
  description: Account data — scraper directory, crawl logs, credits balance.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Data
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-data-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-fetch-api
  name: Spider Fetch API
  description: Per-website APIs with AI-discovered configurations.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Fetch
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-fetch-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-links-api
  name: Spider Links API
  description: Collect all links from a website.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Links
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-links-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-scraping-api
  name: Spider Scraping API
  description: Extract content from individual web pages.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Web Scraping
  tags_raw:
  - Scraping
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-scraping-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-screenshot-api
  name: Spider Screenshot API
  description: Capture full-page or viewport screenshots.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Screenshots
  tags_raw:
  - Screenshot
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-screenshot-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-search-api
  name: Spider Search API
  description: Search the web and crawl results.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Search
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-search-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-transform-api
  name: Spider Transform API
  description: Convert raw HTML or PDF into clean output (markdown, JSON, text).
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Transforms
  tags_raw:
  - Transform
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-transform-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
- aid: spider-cloud:spider-cloud-unblocker-api
  name: Spider Unblocker API
  description: Access content behind anti-bot protections.
  humanURL: https://spider.cloud/docs/api
  baseURL: https://api.spider.cloud
  tags:
  - Unblocker
  properties:
  - type: OpenAPI
    url: openapi/spider-cloud-unblocker-api-openapi.yml
  - type: Documentation
    url: https://spider.cloud/docs/api
  - type: Documentation
    url: https://spider.cloud/docs/overview
  - type: Authentication
    url: https://spider.cloud/docs/api
  - type: RateLimits
    url: https://spider.cloud/docs/api
common:
- type: IssueTracker
  url: https://github.com/spider-rs/spider-cloud-mcp-v2/issues
- type: License
  name: MIT
  url: https://github.com/spider-rs/spider-cloud-mcp-v2/blob/main/LICENSE
- type: AgenticAccess
  url: agentic-access/spider-cloud-agentic-access.yml
- type: DomainSecurity
  url: security/spider-cloud-domain-security.yml
- type: Authentication
  url: authentication/spider-cloud-authentication.yml
- type: Website
  url: https://spider.cloud
- type: Documentation
  url: https://spider.cloud/docs/overview
- type: APIReference
  url: https://spider.cloud/docs/api
- type: Guides
  url: https://spider.cloud/guides
- type: Pricing
  url: https://spider.cloud/pricing
- type: StatusPage
  url: https://spidercloud.statuspage.io
- type: GitHubOrganization
  url: https://github.com/spider-rs
- type: SourceCode
  url: https://github.com/spider-rs/spider
- type: SDKs
  url: https://github.com/spider-rs/spider-clients
- type: SDKs
  url: https://github.com/spider-rs/spider-py
- type: SDKs
  url: https://github.com/spider-rs/spider-nodejs
- type: SDKs
  url: https://docs.rs/spider
- type: MCPServer
  url: https://mcp.spider.cloud/mcp
- type: Integrations
  url: https://spider.cloud/docs/integrations/mcp
- type: Integrations
  url: https://spider.cloud/docs/integrations/langchain
- type: Integrations
  url: https://spider.cloud/docs/integrations/llamaindex
- type: Integrations
  url: https://spider.cloud/docs/integrations/crewai
- type: Integrations
  url: https://spider.cloud/docs/integrations/flowiseai
- type: Integrations
  url: https://spider.cloud/docs/integrations/agno
- type: Integrations
  url: https://spider.cloud/docs/integrations/zapier
- type: Integrations
  url: https://spider.cloud/docs/integrations/x402
- type: Support
  url: mailto:support@spider.cloud
- type: Plans
  url: plans/spider-cloud-plans-pricing.yml
- type: RateLimits
  url: rate-limits/spider-cloud-rate-limits.yml
- type: FinOps
  url: finops/spider-cloud-finops.yml
- url: https://spider.cloud/blog
  type: Blog
- type: DataLicense
  url: data-license/spider-cloud-data-license.yml
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com

Spider API — frequently asked questions

Does Spider have an API?

Yes. APIs.io lists 10 APIs from Spider, including Spider Cloud MCP Server, Spider Crawling API and Spider Data API, and 7 more.

Is there an OpenAPI specification for the Spider API?

Yes. APIs.io indexes 9 OpenAPI specifications for Spider, each linked for download from this page.

How do I authenticate with the Spider API?

Spider documents its authentication (http · 1 scheme). The details are profiled on APIs.io as Spider Authentication.

How much does the Spider API cost?

APIs.io lists 4 pricing plans for Spider, with each plan's price, limits and inclusions under Pricing on this page.

Does Spider have an MCP server?

Yes. APIs.io lists 1 MCP server for Spider.

Is the Spider API ready for AI agents?

Spider has a Kin Score of 32.3 out of 100 (thin) and an Agent Readiness score of 44.2 (agent-ready), scored by API Evangelist on 2026-11-13.

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/spider-cloud"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/spider-cloud/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/spider-cloud/evidence"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.