Scrapy website screenshot

Scrapy

Scrapy is an open-source Python web crawling framework for extracting structured data from websites using spiders and built-in data pipelines.

Scrapy publishes 1 API on the APIs.io network. Tagged areas include Crawling, Data Extraction, and Scraping.

Scrapy’s developer surface includes documentation and 4 more developer resources.

🌐 Visit website 📡 Source on GitHub
18.5/100 emerging ▬ flat Agent 3/100 human only Full breakdown ↓
scored 2026-08-05 · rubric v0.9.1
AccessFreemium
1 APIs
CrawlingData ExtractionScraping

Kin Score

Kin Score Kin Score How this is scored →
scored 2026-08-05 · rubric v0.9.1
Composite quality — 18.5/100 · emerging
Contract Quality 0.0 / 25
Developer Ergonomics 1.7 / 20
Commercial Clarity 7.9 / 20
Operational Transparency 4.8 / 13
Governance 0.0 / 12
Discoverability 4.1 / 10
Agent readiness — 3/100 · human only
Machine-Readable Contract 0 / 18
Agentic Access Contract 0 / 10
MCP Server 0 / 12
Machine-Readable Auth 0 / 10
Idempotency 0 / 9
Stable Error Semantics 0 / 8
Request/Response Examples 0 / 7
Rate-Limit Signaling 7 / 7
Typed Event Surface 0 / 6
Agent Skills 0 / 5
Well-Known Catalog 0 / 4
Consent & Bot Identity 0 / 3
A2A Agent Card 0 / 8
Dry-Run / Simulate Mode 0 / 4
Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. This rating is computed from github.com/api-evangelist/scrapy: open an issue to ask a question, or submit a pull request to add artifacts. Want it done for you? Prioritized profiling — $2,500 →

APIs 1

Individual APIs this provider publishes, each with its own machine-readable definition.

Scrapy

Scrapy is an open-source Python web crawling framework for extracting structured data from websites using spiders and built-in data pipelines.

Pricing Plans 1

Published pricing tiers and plan structures.

Scrapy Plans Pricing

3 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Scrapy Rate Limits

5 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Scrapy Finops

FINOPS

Security Posture 1

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Scrapy Domain Security

TLSv1.3 · HSTS

SECURITY

Resources

Documentation 1

Reference material describing how the API behaves

Documentation
Documentation

Agent Surfaces 1

MCP servers, agent skills, and machine-readable catalogs

LlmsText
LlmsText

Build 1

SDKs, sample code, and the tooling you integrate with

GitHubOrganization
GitHubOrganization

Access & Security 1

Authentication, authorization, and security posture

DomainSecurity
DomainSecurity

Company 1

The organization behind the API

Website
Website

Source (apis.yml)

apis.yml Raw ↑ 
aid: scrapy
name: Scrapy
description: Scrapy is an open-source Python web crawling framework for extracting structured data from websites using spiders
  and built-in data pipelines.
type: Index
accessModel:
  pricing: freemium
  onboarding: unknown
  trial: false
  try_now: false
  public: false
  label: Freemium
  confidence: medium
  source:
  - plans
  generated: '2026-07-22'
  method: derived
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/scrapy.png
tags:
- Crawling
- Data Extraction
- Scraping
url: https://raw.githubusercontent.com/api-evangelist/scrapy/refs/heads/main/apis.yml
created: '2026-03-29'
modified: '2026-03-29'
specificationVersion: '0.19'
apis:
- aid: scrapy:scrapy
  name: Scrapy
  description: Scrapy is an open-source Python web crawling framework for extracting structured data from websites using spiders
    and built-in data pipelines.
  humanURL: https://scrapy.org/
  tags:
  - Crawling
  - Data Extraction
  - Scraping
  properties:
  - type: Documentation
    url: https://docs.scrapy.org/
  - type: GettingStarted
    url: https://docs.scrapy.org/en/latest/intro/tutorial.html
common:
- type: DomainSecurity
  url: security/scrapy-domain-security.yml
- type: Website
  url: https://scrapy.org/
- type: Documentation
  url: https://docs.scrapy.org/
- type: GitHubOrganization
  url: https://github.com/scrapy
- type: LlmsText
  url: https://docs.scrapy.org/llms.txt
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com