WebCrawlerAPI · AsyncAPI Specification

Webcrawlerapi Com Webhooks

Version

View Spec View on GitHub Web ScrapingWeb CrawlingData ExtractionMarkdownAI AgentsLLMRAGWeb FeedsChange DetectionStructured DataMCPA2AAgent-NativeAsyncAPIEvents

AsyncAPI Specification

Raw ↑
generated: '2026-09-19'
method: searched
source: https://webcrawlerapi.com/docs/async-requests + https://webcrawlerapi.com/docs/api/job + https://webcrawlerapi.com/docs/feeds
  + POST /v1/job/{id}/webhook/resend and /v2/feed/{id}/webhook/resend in openapi/_original/webcrawlerapi-com-swagger.json
  + changelog 2025-09-25 (webhook status tracking / resend)
api: WebCrawler API
asyncapi_published: false
summary: Two webhook producers, both configured per resource with a webhook_url field on the create request. Crawl
  jobs POST the job object once the job completes; feeds POST a feed-run payload when changes are detected. Delivery
  state is exposed on the job (webhook_status, webhook_error) and in the dashboard, and both can be re-sent through
  dedicated resend endpoints. No signing secret, signature header, retry schedule or event-type catalogue is documented
  - the payload is identified by the resource id, not by an event name.
delivery:
  transport: https-post
  content_type: application/json
  configured_via: webhook_url on POST /v1/crawl and POST /v2/feed
  signature: null
  signature_note: No HMAC / signature header is documented. A receiver cannot authenticate the sender except by
    URL secrecy and by re-fetching GET /v1/job/{id} with its own key.
  retries: not documented; manual resend via API or dashboard "Resend" button
  dedup_field: id (job id) / feed run id
  delivery_status_fields:
  - webhook_status
  - webhook_error
  resend:
  - operation: POST /v1/job/{id}/webhook/resend
    response: WebhookResendResponse {success, status_code, webhook_error, webhook_url, job_id, message}
  - operation: POST /v2/feed/{id}/webhook/resend
    note: Resends the webhook for the latest completed feed run.
events:
- name: crawl_job.completed
  trigger: A crawl job reaches a terminal status (done or error).
  payload: 'The job object as returned by GET /v1/job/{id}: {id, status, url, output_formats, job_items[...], ...}'
  source: https://webcrawlerapi.com/docs/async-requests
  note: 'The docs show the payload as {"id": ..., ...} - i.e. the job body. There is no event-type field.'
- name: feed.changes_detected
  trigger: A scheduled or forced feed run detects new / changed / unavailable (and, with include_errors, error)
    pages.
  payload: Feed run details with change information; each changed/new page carries a content_url in the feed's output_format.
  source: https://webcrawlerapi.com/docs/feeds
polling_alternative:
  crawl: GET /v1/job/{id}; the job carries recommended_pull_delay_ms
  scrape: GET /v2/scrape/{id} after POST /v2/scrape?async=true
  agent: GET /v1/agent/job/{id} (agent runs have no webhook_url)
  feeds: GET /v2/feed/{id}/rss (Atom 1.0) or /json (JSON Feed 1.1)
gap: Publish an AsyncAPI (or at least a signed-payload scheme with a documented header and retry policy) for the
  two webhook events; today a receiver has no way to verify origin.

Work with this as data

Every AsyncAPI spec here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for asyncapi

4 MCP tools reach this
  • find_asyncapisBrowse and filter every AsyncAPI spec in the catalog.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This AsyncAPI spec
curl "https://apis.io/api/v1/asyncapis/webcrawlerapi-com-webhooks"
All asyncapi
curl "https://apis.io/api/v1/asyncapis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

A second provider on the same verified email joins the account you already have.