superagnt-com Web: Scrape API
Extract the content of a single web page.
Extract the content of a single web page.
Every API here is available over the APIs.io API and to AI agents over MCP.
One button, every client — Claude, Cursor, VS Code and the rest.
https://apis.io/mcp
find_apisBrowse and filter every API in the catalog.get_api_artifactsOne API's artifacts, grouped by type.get_openapiThe primary OpenAPI for this API.find_similar_apisAPIs that look like this one.apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.resolveTurn a domain, URL or GitHub org into the provider it belongs to.find_cohortsEvery scored population of providers in the catalog.curl "https://apis.io/api/v1/apis/superagnt-com:superagnt-com-web-scrape-api"
curl "https://apis.io/api/v1/apis?limit=25"
Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.
Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.
A second provider on the same verified email joins the account you already have.
openapi: 3.2.0
info:
title: 'superagnt Web: Scrape API'
description: Unified API for social media data. Access LinkedIn, YouTube, TikTok, X, Instagram, Reddit, and Facebook through a single API key.
version: 1.0.0
contact:
name: superagnt
url: https://superagnt.com
servers:
- url: https://api.superagnt.com
description: superagnt API
security:
- ApiKeyAuth: []
tags:
- name: 'Web: Scrape'
description: Extract the content of a single web page.
paths:
/v1/data/web/scrape:
post:
operationId: scrape
summary: Scrape a web page
description: 'Fetches a single URL and returns its content. Default is clean main-content `markdown` — cheapest and right for almost everything; just read the markdown. Reach for the pricier options only when the task truly needs them: the `json` format with a `schema` runs an LLM to extract structured fields (costs several credits per page — prefer scraping markdown and parsing it in code), and `proxy: enhanced` forces heavy anti-bot handling. Scrape only the specific pages you will actually use. Handles JavaScript-rendered pages.'
tags:
- 'Web: Scrape'
requestBody:
required: true
content:
application/json:
schema:
type: object
required:
- url
properties:
url:
type: string
format: uri
description: The URL to scrape (must be a publicly reachable http(s) URL).
formats:
type: array
description: 'Output formats to return. Each entry is either a string preset or an object form. String presets: ''markdown'' (default), ''summary'', ''html'', ''rawHtml'', ''links'', ''images''. Object forms include { type: ''json'', schema, prompt } for structured extraction, { type: ''screenshot'', fullPage, quality } for an image, and { type: ''changeTracking'', modes } for diffs. Defaults to [''markdown''] when omitted.'
items:
oneOf:
- type: string
enum:
- markdown
- summary
- html
- rawHtml
- links
- images
- type: object
required:
- type
properties:
type:
type: string
enum:
- json
- screenshot
- changeTracking
schema:
type: object
description: JSON schema describing the structured data to extract (for type 'json').
prompt:
type: string
description: Natural-language extraction prompt (for type 'json').
fullPage:
type: boolean
description: Capture the full scrollable page (for type 'screenshot').
modes:
type: array
items:
type: string
enum:
- git-diff
- json
description: Change-tracking modes (for type 'changeTracking').
onlyMainContent:
type: boolean
description: Return only the main content of the page, excluding navs, headers, and footers.
default: true
includeTags:
type: array
items:
type: string
description: Only include content within these HTML tags / CSS selectors.
excludeTags:
type: array
items:
type: string
description: Exclude content within these HTML tags / CSS selectors.
maxAge:
type: integer
description: Return cached content up to this age in milliseconds (faster, cheaper). Default 172800000 (2 days).
minimum: 0
waitFor:
type: integer
description: Milliseconds to wait for the page to render before scraping.
minimum: 0
default: 0
mobile:
type: boolean
description: Emulate a mobile device.
default: false
timeout:
type: integer
description: Request timeout in milliseconds (1000-300000). Default 60000.
minimum: 1000
maximum: 300000
parsers:
type: array
items: {}
description: PDF parsing configuration.
proxy:
type: string
enum:
- basic
- enhanced
- auto
description: Proxy / anti-bot mode. 'auto' (default) escalates only on failure; 'enhanced' forces advanced anti-bot handling and costs more.
default: auto
location:
type: object
description: Geographic proxy location and language settings.
properties:
country:
type: string
description: ISO 3166-1 alpha-2 country code.
languages:
type: array
items:
type: string
example:
url: https://example.com
formats:
- markdown
responses:
'200':
description: The scraped page content in the requested formats.
content:
application/json:
schema:
type: object
'400':
description: Bad request — missing `url` or invalid parameter.
'401':
description: Unauthorized.
'402':
description: Insufficient balance.
'429':
description: Too many requests.
5XX:
description: Upstream error.
security:
- ApiKeyAuth: []
components:
securitySchemes:
ApiKeyAuth:
type: apiKey
in: header
name: Authorization
description: 'Bearer token. Pass your agntdata API key as: Authorization: Bearer agnt_live_...'