Screaming Frog SEO Spider
The Screaming Frog SEO Spider is a desktop website crawler for Windows, macOS, and Ubuntu that performs comprehensive technical SEO audits. It crawls websites to find broken links, analyze page titles and meta data, discover duplicate content, generate XML sitemaps, and integrate with third-party APIs including Google Analytics, Google Search Console, PageSpeed Insights, Ahrefs, Majestic, Moz, and AI services (OpenAI, Gemini, Ollama, Anthropic). The Spider acts as an API consumer, pulling external metrics and data into crawl reports.
Documentation
Documentation
https://www.screamingfrog.co.uk/seo-spider/user-guide/
APIReference
https://www.screamingfrog.co.uk/guides/mcp-server/
GettingStarted
https://www.screamingfrog.co.uk/seo-spider/tutorials/quick-start-guide/
Schemas & Data
JSONSchema
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/json-schema/screaming-frog-crawl-result-schema.json
JSONStructure
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/json-structure/screaming-frog-crawl-result-structure.json
Other Resources
Pricing
https://www.screamingfrog.co.uk/seo-spider/pricing/
ReleaseHistory
https://www.screamingfrog.co.uk/seo-spider/release-history/
JSONLDContext
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/json-ld/screaming-frog-context.jsonld
Vocabulary
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/vocabulary/screaming-frog-vocabulary.yml
Examples
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/examples/screaming-frog-crawl-result-example.json
MCPServer
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/mcp/screaming-frog-mcp.yml
CLI
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/cli/screaming-frog-cli.yml
AgentSkill
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/skills/_index.yml
DataModel
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/data-model/screaming-frog-data-model.yml
Support
https://www.screamingfrog.co.uk/seo-spider/support/
APIsJSON
https://raw.githubusercontent.com/api-evangelist/screaming-frog/refs/heads/main/apis.yml