hydrafetch-com Extract API

openapi: 3.2.0
info:
  title: Hydrafetch Extract API
  description: 'Hydrafetch is a web data API for developers and agents. Send a URL and get back clean Markdown, the page''s own structured data, schema-shaped JSON, links, or a summary, with the navigation, banners and boilerplate stripped out. Scrape one page, crawl a whole site, run a search, or extract to a schema, all through one API with one response shape. Every call costs one credit a page whatever it took to fetch, and failures are never billed.


    Point us at a whole site and get every page. Ask a question and get answers with per-field

    confidence and the passage each value came from. You describe the outcome you want — the

    pipeline decides how to get it.


    ## Authentication


    Every request is authenticated with your API key in the `X-API-Key` header. Keys are scoped to a

    workspace and carry its credit balance.


    ## Credits


    Calls are billed in credits and charged only on success. A standard scrape is one credit; richer

    formats and the extraction tier cost more. Each response reports what it consumed.


    ## Conventions


    All timestamps are UTC ISO 8601. Long-running jobs (crawl, batch) return a job id you poll, or a

    webhook you register.


    ## Errors


    Every failure returns the same shape, whatever the status:


    ```json

    { "success": false, "error": { "code": "INSUFFICIENT_CREDITS", "message": "Insufficient credits" },

    "meta": { "requestId": "019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c" } }

    ```


    Branch on `error.code`, which is stable. `error.message` is written for a person and may be

    reworded without notice. Validation failures add `error.details`. Quote `meta.requestId` when

    asking us about a specific failure. Every operation documents the codes it can return.


    ## Rate limits


    Limits are per workspace, not per key, over a 60 second window, and the ceiling comes from your

    plan. Every response carries the IETF RateLimit header fields so you can self-throttle rather

    than discovering the limit by hitting it:


    ```http

    RateLimit-Policy: "workspace";q=600;w=60

    RateLimit: "workspace";r=599;t=42

    ```


    `q` is the quota, `w` the window in seconds, `r` the requests remaining and `t` the seconds until

    the window resets. A 429 also carries `Retry-After` in seconds; wait that long rather than

    retrying immediately. The older `X-RateLimit-Limit`, `X-RateLimit-Remaining` and

    `X-RateLimit-Reset` headers are still sent and mean the same thing.


    ## Versioning and deprecation


    The version is in the path: every endpoint on this API lives under `/v1/`. Within a version we

    only make additive changes — new endpoints, new optional parameters, new fields in a response.

    Adding a field to a response is not a breaking change, so parse defensively and ignore what you

    do not recognise.


    Anything that would break an existing integration ships under a new version path instead. When an

    endpoint or a version is retired, the responses say so before it stops working:


    ```http

    Deprecation: @1780272000

    Sunset: Sat, 01 May 2027 00:00:00 GMT

    Link: ; rel="deprecation"

    ```


    `Deprecation` (RFC 9745) marks when the endpoint became deprecated, `Sunset` (RFC 8594) when it

    stops responding, and the `deprecation` link relation points at what to move to. Nothing on this

    API is deprecated today, so you will not see these headers yet. Watch for them rather than for an

    announcement: the headers are the notice, and the linked changelog carries what to move to.'
  version: '1.0'
  contact:
    name: Hydrafetch
    url: https://hydrafetch.com
    email: support@hydrafetch.com
servers:
- url: https://api.hydrafetch.com
  description: Production
security:
- apiKey: []
tags:
- name: Extract
  description: Pull schema-shaped JSON out of one or many pages, with optional per-field confidence and the source passage behind each value.
paths:
  /v1/web/extract:
    post:
      description: Pull schema-shaped JSON out of one or many pages in a single call. An LLM maps each page onto your schema and/or prompt. Point at a single page, a list, or a crawl scope with a trailing `/*` wildcard; optionally let web search find extra source pages, return per-field confidence with the source passage behind each value, and merge everything into one deduplicated collection of entities. Charged per page that returns data.
      operationId: run
      parameters: []
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ExtractRequestDto'
      responses:
        '200':
          description: The extracted data.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ExtractResponseDto'
        '400':
          description: The request body or query failed validation.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: VALIDATION_ERROR
                  message: url must be a valid URL
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '401':
          description: The X-API-Key header is missing, malformed, or names a key that no longer exists.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: UNAUTHORIZED
                  message: 'Missing X-API-Key header. Agents: https://hydrafetch.com/auth.md'
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '402':
          description: The workspace has no credits left for this request. Nothing was charged and nothing was fetched.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: INSUFFICIENT_CREDITS
                  message: Insufficient credits
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '403':
          description: The key is valid but the workspace may not make this request.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: WORKSPACE_BANNED
                  message: This workspace has been suspended and cannot make API requests.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '404':
          description: No such job, or the id belongs to another workspace.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: NOT_FOUND
                  message: Job not found
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '422':
          description: The target site did not respond, or we could not read that page. This is a fact about that URL rather than a fault on our side, and retrying it will not change the answer. When details.retryAfterSeconds is present, asking again before then returns this same answer. Nothing was charged.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: UPSTREAM_UNREACHABLE
                  message: The origin did not respond to any attempt we made.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '429':
          description: The workspace exceeded its per-minute request limit. Read the RateLimit headers, or Retry-After on this response, and retry after the window resets.
          headers:
            Retry-After:
              description: Seconds to wait before retrying. Prefer this over a fixed backoff.
              schema:
                type: integer
                example: 17
            RateLimit:
              description: Requests remaining (r) and seconds until the window resets (t).
              schema:
                type: string
                example: '"workspace";r=0;t=17'
            RateLimit-Policy:
              description: The quota (q) for your plan and the window it applies over (w), in seconds.
              schema:
                type: string
                example: '"workspace";q=600;w=60'
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: TOO_MANY_REQUESTS
                  message: Rate limit exceeded for this workspace and plan.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '500':
          description: Unexpected server error. Nothing was charged.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: INTERNAL_SERVER_ERROR
                  message: Internal server error
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '503':
          description: A dependency needed for the requested formats is unavailable. Safe to retry.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: SERVICE_UNAVAILABLE
                  message: The summary and json formats are not available.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
      summary: Extract structured data from URLs
      tags:
      - Extract
components:
  schemas:
    WebErrorBody:
      type: object
      required:
      - code
      - message
      properties:
        code:
          type: string
          enum:
          - VALIDATION_ERROR
          - INVALID_INPUT
          - UNAUTHORIZED
          - INSUFFICIENT_CREDITS
          - FORBIDDEN
          - WORKSPACE_BANNED
          - NOT_FOUND
          - TOO_MANY_REQUESTS
          - UPSTREAM_UNREACHABLE
          - SERVICE_UNAVAILABLE
          - INTERNAL_SERVER_ERROR
          description: Stable, machine-readable reason. Branch on this rather than on the message, which may be reworded.
          example: INSUFFICIENT_CREDITS
        message:
          type: string
          description: Human-readable explanation. Safe to show a user, not to parse.
          example: Insufficient credits
        details:
          type: object
          additionalProperties: true
          description: Optional machine-readable context, present on validation failures.
    ExtractItemResultDto:
      type: object
      properties:
        url:
          type: string
          example: https://example.com/products/widget
          description: The page this result came from.
        data:
          type:
          - object
          - 'null'
          additionalProperties: true
          description: The extracted data, shaped by your schema and/or prompt. Null when nothing matched.
        fields:
          type: object
          additionalProperties:
            $ref: '#/components/schemas/FieldProvenanceDto'
          description: Per-field confidence and source passage, keyed by field name. Present only when `showConfidence` is set.
        error:
          type:
          - object
          - 'null'
          example: null
          description: Set when this page could not be extracted; null on success.
      required:
      - url
      - data
      - error
    FieldProvenanceDto:
      type: object
      properties:
        confidence:
          type: number
          minimum: 0
          maximum: 1
          example: 0.92
          description: How certain the value is correct given the page, from 0 to 1.
        evidence:
          type: string
          example: Priced at $49.00 with free shipping.
          description: The exact short passage the value was drawn from, or an empty string if none.
      required:
      - confidence
      - evidence
    ExtractResultDto:
      type: object
      properties:
        results:
          description: One result per extracted page.
          type: array
          items:
            $ref: '#/components/schemas/ExtractItemResultDto'
        sources:
          description: The concrete URLs actually extracted, after wildcard and web-search expansion. Present only when `showSources` is set.
          type: array
          items:
            type: string
        collection:
          description: The deduplicated collection, one row per entity. Present only when `mergeEntities` is set.
          type: array
          items:
            $ref: '#/components/schemas/MergedEntityDto'
      required:
      - results
    LocationDto:
      type: object
      properties:
        country:
          type: string
          example: us
          description: ISO 3166 alpha-2 country to fetch the page as if from.
        languages:
          example:
          - en-US
          - en
          description: Preferred content languages, most-preferred first.
          type: array
          items:
            type: string
    ExtractResponseDto:
      type: object
      properties:
        data:
          $ref: '#/components/schemas/ExtractResultDto'
      required:
      - data
    MergedEntityDto:
      type: object
      properties:
        data:
          type: object
          additionalProperties: true
          description: The unioned data for one entity, filled across every contributing page.
        sources:
          description: Every source URL that contributed to this entity.
          type: array
          items:
            type: string
      required:
      - data
      - sources
    WebErrorMeta:
      type: object
      properties:
        requestId:
          type: string
          description: Quote this when contacting support about a specific failure.
          example: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
    WebError:
      type: object
      required:
      - success
      - error
      - meta
      description: Every failure on this API returns this shape, whatever the status. Successful responses never carry it.
      properties:
        success:
          type: boolean
          enum:
          - false
          example: false
        error:
          $ref: '#/components/schemas/WebErrorBody'
        meta:
          $ref: '#/components/schemas/WebErrorMeta'
    ExtractRequestDto:
      type: object
      properties:
        urls:
          maxItems: 10
          example:
          - https://example.com/products/widget
          - https://example.com/products/*
          description: 'The pages to extract from. Each must be an http(s) URL. A trailing `/*` marks a crawl scope: every page discovered under that path is extracted and merged into the result.'
          type: array
          items:
            type: string
        schema:
          type: object
          additionalProperties: true
          description: JSON Schema describing the shape you want back. Optional if `prompt` is given; when both are present the schema fixes the field names and types while the prompt guides what to pull.
        prompt:
          type: string
          maxLength: 2000
          example: Pull the product name, price in USD, and whether it is in stock.
          description: Natural-language instruction for what to extract. Use with or instead of a schema.
        preferStructure:
          type: boolean
          description: Preserve document structure (headings, lists, tables) over prose density when reading the page — good for listing and catalog pages. Default off.
        enableWebSearch:
          type: boolean
          description: Pull in extra source pages by web-searching your prompt, to fill fields your URLs do not cover. Requires a `prompt`.
        showSources:
          type: boolean
          description: Return the concrete list of URLs that were actually extracted, after any wildcard and web-search expansion. Default off.
        showConfidence:
          type: boolean
          description: For each field, return a confidence score and the exact source passage the value was drawn from. Default off.
        mergeEntities:
          type: boolean
          description: Merge the per-page results into one deduplicated collection — one row per entity, with its contributing source URLs — instead of a separate result per page. Default off.
        maxAge:
          type: number
          minimum: 0
          maximum: 604800000
          description: Reuse a recent capture of each page if it is younger than this many milliseconds. Omit or 0 to always fetch fresh. Capped at 7 days.
        location:
          description: Fetch each page as if from this country and with these preferred languages, for sites that change prices or content by region.
          allOf:
          - $ref: '#/components/schemas/LocationDto'
        headers:
          type: object
          additionalProperties:
            type: string
          example:
            Cookie: store_id=1234
          description: Extra request headers to send when fetching each page, such as a cookie that picks the store or delivery area a site prices by.
      required:
      - urls
  securitySchemes:
    apiKey:
      type: apiKey
      in: header
      name: X-API-Key