hydrafetch-com Search API

openapi: 3.2.0
info:
  title: Hydrafetch Search API
  description: 'Hydrafetch is a web data API for developers and agents. Send a URL and get back clean Markdown, the page''s own structured data, schema-shaped JSON, links, or a summary, with the navigation, banners and boilerplate stripped out. Scrape one page, crawl a whole site, run a search, or extract to a schema, all through one API with one response shape. Every call costs one credit a page whatever it took to fetch, and failures are never billed.


    Point us at a whole site and get every page. Ask a question and get answers with per-field

    confidence and the passage each value came from. You describe the outcome you want — the

    pipeline decides how to get it.


    ## Authentication


    Every request is authenticated with your API key in the `X-API-Key` header. Keys are scoped to a

    workspace and carry its credit balance.


    ## Credits


    Calls are billed in credits and charged only on success. A standard scrape is one credit; richer

    formats and the extraction tier cost more. Each response reports what it consumed.


    ## Conventions


    All timestamps are UTC ISO 8601. Long-running jobs (crawl, batch) return a job id you poll, or a

    webhook you register.


    ## Errors


    Every failure returns the same shape, whatever the status:


    ```json

    { "success": false, "error": { "code": "INSUFFICIENT_CREDITS", "message": "Insufficient credits" },

    "meta": { "requestId": "019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c" } }

    ```


    Branch on `error.code`, which is stable. `error.message` is written for a person and may be

    reworded without notice. Validation failures add `error.details`. Quote `meta.requestId` when

    asking us about a specific failure. Every operation documents the codes it can return.


    ## Rate limits


    Limits are per workspace, not per key, over a 60 second window, and the ceiling comes from your

    plan. Every response carries the IETF RateLimit header fields so you can self-throttle rather

    than discovering the limit by hitting it:


    ```http

    RateLimit-Policy: "workspace";q=600;w=60

    RateLimit: "workspace";r=599;t=42

    ```


    `q` is the quota, `w` the window in seconds, `r` the requests remaining and `t` the seconds until

    the window resets. A 429 also carries `Retry-After` in seconds; wait that long rather than

    retrying immediately. The older `X-RateLimit-Limit`, `X-RateLimit-Remaining` and

    `X-RateLimit-Reset` headers are still sent and mean the same thing.


    ## Versioning and deprecation


    The version is in the path: every endpoint on this API lives under `/v1/`. Within a version we

    only make additive changes — new endpoints, new optional parameters, new fields in a response.

    Adding a field to a response is not a breaking change, so parse defensively and ignore what you

    do not recognise.


    Anything that would break an existing integration ships under a new version path instead. When an

    endpoint or a version is retired, the responses say so before it stops working:


    ```http

    Deprecation: @1780272000

    Sunset: Sat, 01 May 2027 00:00:00 GMT

    Link: ; rel="deprecation"

    ```


    `Deprecation` (RFC 9745) marks when the endpoint became deprecated, `Sunset` (RFC 8594) when it

    stops responding, and the `deprecation` link relation points at what to move to. Nothing on this

    API is deprecated today, so you will not see these headers yet. Watch for them rather than for an

    announcement: the headers are the notice, and the linked changelog carries what to move to.'
  version: '1.0'
  contact:
    name: Hydrafetch
    url: https://hydrafetch.com
    email: support@hydrafetch.com
servers:
- url: https://api.hydrafetch.com
  description: Production
security:
- apiKey: []
tags:
- name: Search
  description: Search the web and get the ranked results back already scraped, so an agent has content to cite rather than links to fetch.
paths:
  /v1/web/search:
    post:
      description: 'Search the web for a query and get back ranked results: title, url and snippet. Costs 1 credit. Set `scrapeResults` to true to also fetch each result as clean data, which costs 1 additional credit per page scraped.'
      operationId: run
      parameters: []
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/SearchRequestDto'
      responses:
        '200':
          description: The ranked search results.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/SearchResponseDto'
        '400':
          description: The request body or query failed validation.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: VALIDATION_ERROR
                  message: url must be a valid URL
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '401':
          description: The X-API-Key header is missing, malformed, or names a key that no longer exists.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: UNAUTHORIZED
                  message: 'Missing X-API-Key header. Agents: https://hydrafetch.com/auth.md'
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '402':
          description: The workspace has no credits left for this request. Nothing was charged and nothing was fetched.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: INSUFFICIENT_CREDITS
                  message: Insufficient credits
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '403':
          description: The key is valid but the workspace may not make this request.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: WORKSPACE_BANNED
                  message: This workspace has been suspended and cannot make API requests.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '404':
          description: No such job, or the id belongs to another workspace.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: NOT_FOUND
                  message: Job not found
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '422':
          description: The target site did not respond, or we could not read that page. This is a fact about that URL rather than a fault on our side, and retrying it will not change the answer. When details.retryAfterSeconds is present, asking again before then returns this same answer. Nothing was charged.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: UPSTREAM_UNREACHABLE
                  message: The origin did not respond to any attempt we made.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '429':
          description: The workspace exceeded its per-minute request limit. Read the RateLimit headers, or Retry-After on this response, and retry after the window resets.
          headers:
            Retry-After:
              description: Seconds to wait before retrying. Prefer this over a fixed backoff.
              schema:
                type: integer
                example: 17
            RateLimit:
              description: Requests remaining (r) and seconds until the window resets (t).
              schema:
                type: string
                example: '"workspace";r=0;t=17'
            RateLimit-Policy:
              description: The quota (q) for your plan and the window it applies over (w), in seconds.
              schema:
                type: string
                example: '"workspace";q=600;w=60'
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: TOO_MANY_REQUESTS
                  message: Rate limit exceeded for this workspace and plan.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '500':
          description: Unexpected server error. Nothing was charged.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: INTERNAL_SERVER_ERROR
                  message: Internal server error
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
        '503':
          description: A dependency needed for the requested formats is unavailable. Safe to retry.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/WebError'
              example:
                success: false
                error:
                  code: SERVICE_UNAVAILABLE
                  message: The summary and json formats are not available.
                meta:
                  requestId: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
      summary: Search the web
      tags:
      - Search
components:
  schemas:
    OptLocationDto:
      type: object
      properties:
        country:
          type: string
          example: us
          description: ISO 3166 alpha-2 country to fetch as if from.
        languages:
          example:
          - en-US
          - en
          description: Preferred content languages, most-preferred first.
          type: array
          items:
            type: string
    WebErrorBody:
      type: object
      required:
      - code
      - message
      properties:
        code:
          type: string
          enum:
          - VALIDATION_ERROR
          - INVALID_INPUT
          - UNAUTHORIZED
          - INSUFFICIENT_CREDITS
          - FORBIDDEN
          - WORKSPACE_BANNED
          - NOT_FOUND
          - TOO_MANY_REQUESTS
          - UPSTREAM_UNREACHABLE
          - SERVICE_UNAVAILABLE
          - INTERNAL_SERVER_ERROR
          description: Stable, machine-readable reason. Branch on this rather than on the message, which may be reworded.
          example: INSUFFICIENT_CREDITS
        message:
          type: string
          description: Human-readable explanation. Safe to show a user, not to parse.
          example: Insufficient credits
        details:
          type: object
          additionalProperties: true
          description: Optional machine-readable context, present on validation failures.
    WebStructuredDataDto:
      type: object
      properties:
        entities:
          description: The page's own structured data, normalised into one deduplicated list of typed entities.
          type: array
          items:
            $ref: '#/components/schemas/WebStructuredEntityDto'
        jsonLd:
          description: Raw JSON-LD blocks, as found on the page.
          type: array
          items:
            type: object
        microdata:
          description: Raw microdata items.
          type: array
          items:
            type: object
        opengraph:
          description: Raw OpenGraph/Twitter-card tags.
          type: array
          items:
            type: object
        rdfa:
          description: Raw RDFa items.
          type: array
          items:
            type: object
        appState:
          description: Names of embedded framework app-state blocks detected on the page.
          type: array
          items:
            type: string
      required:
      - entities
      - jsonLd
      - microdata
      - opengraph
      - rdfa
      - appState
    WebLinksDto:
      type: object
      properties:
        internal:
          description: Links pointing to the same site.
          type: array
          items:
            type: string
        external:
          description: Links pointing to other sites.
          type: array
          items:
            type: string
      required:
      - internal
      - external
    WebScrapeMetadataDto:
      type: object
      properties:
        title:
          type:
          - object
          - 'null'
          example: Example Domain
          description: The page title.
        structure:
          type: string
          enum:
          - markdown
          - plain
          example: markdown
          description: 'Whether the content carries markdown structure (headings, lists, emphasis) or is plain prose. `plain` is not an error — the default extractor optimises for capturing every word and wins on listing and forum pages. If you need the structure, re-request with `preferStructure: true`.'
        pageType:
          type: string
          example: article
          description: Coarse page classification (e.g. article, listing, forum, docs).
        wordCount:
          type: number
          example: 214
          description: Word count of the extracted main content.
        description:
          type:
          - object
          - 'null'
          example: A short summary of the page, as published by the page itself.
          description: The page's own description (meta description / og:description).
        language:
          type:
          - object
          - 'null'
          example: en
          description: The language the page declares.
        author:
          type:
          - object
          - 'null'
          example: Jane Doe
          description: The declared author.
        siteName:
          type:
          - object
          - 'null'
          example: Example Blog
          description: The declared site name.
        publishedTime:
          type:
          - object
          - 'null'
          example: '2026-01-05'
          description: When the page says it was published (ISO 8601).
        modifiedTime:
          type:
          - object
          - 'null'
          example: '2026-02-11'
          description: When the page says it was last updated (ISO 8601).
        image:
          type:
          - object
          - 'null'
          example: https://example.com/cover.png
          description: The page's lead image (og:image).
      required:
      - title
      - structure
      - pageType
      - wordCount
      - description
      - language
      - author
      - siteName
      - publishedTime
      - modifiedTime
      - image
    WebScrapeDataDto:
      type: object
      properties:
        url:
          type: string
          example: https://example.com
          description: The URL you requested.
        finalUrl:
          type: string
          example: https://example.com/
          description: The final URL after any redirects.
        redirected:
          type: boolean
          example: false
          description: 'Whether the origin redirected: true when `finalUrl` differs from the URL you requested. Explicit so a page with no redirect is distinguishable from one whose redirect was not tracked.'
        status:
          type: number
          example: 200
          description: HTTP status of the fetched page.
        cached:
          type: boolean
          example: false
          description: Whether this result was served from cache.
        warning:
          type: string
          description: Set when the page was returned with a caveat (e.g. partial content).
        metadata:
          $ref: '#/components/schemas/WebScrapeMetadataDto'
        usage:
          $ref: '#/components/schemas/WebUsageDto'
        quality:
          description: Per-page extraction quality signals.
          allOf:
          - $ref: '#/components/schemas/WebQualityDto'
        markdown:
          type: string
          description: Clean Markdown of the main content. Returned when `markdown` is requested.
        html:
          type: string
          description: Cleaned main-content HTML. Returned when `html` is requested.
        rawHtml:
          type: string
          description: The unmodified page HTML. Returned when `rawHtml` is requested.
        links:
          description: Returned when `links` is requested.
          allOf:
          - $ref: '#/components/schemas/WebLinksDto'
        structured:
          description: Returned when `structured` is requested.
          allOf:
          - $ref: '#/components/schemas/WebStructuredDataDto'
        summary:
          type: string
          description: A concise factual summary. Returned when `summary` is requested (LLM-backed).
        json:
          type: object
          additionalProperties: true
          description: Schema-shaped JSON. Returned when `json` is requested (LLM-backed).
      required:
      - url
      - finalUrl
      - redirected
      - status
      - cached
      - metadata
    WebUsageDto:
      type: object
      properties:
        creditsUsed:
          type: number
          example: 1
          description: Credits this call consumed. Charged only on success.
        creditsRemaining:
          type: number
          example: 4999
          description: Credits left in your workspace's balance after this call.
        freshness:
          type: string
          enum:
          - cache
          - fresh
          example: fresh
          description: Whether the result was served from cache or freshly fetched.
      required:
      - creditsUsed
      - creditsRemaining
      - freshness
    WebStructuredEntityDto:
      type: object
      properties:
        type:
          type: string
          example: Product
          description: The schema.org type of the entity.
        source:
          type: string
          enum:
          - json-ld
          - microdata
          - rdfa
          - opengraph
          description: Which structured-data syntax the entity came from.
        properties:
          type: object
          additionalProperties: true
          description: The entity's properties, as published.
      required:
      - type
      - source
      - properties
    SearchResponseDto:
      type: object
      properties:
        data:
          $ref: '#/components/schemas/SearchResultDto'
      required:
      - data
    SearchResultItemDto:
      type: object
      properties:
        title:
          type: string
          example: Best open source vector databases
          description: The result title.
        url:
          type: string
          example: https://example.com/vector-databases
          description: The result URL.
        snippet:
          type: string
          example: A rundown of the leading open source vector databases and how they compare.
          description: A short snippet describing the result.
        rank:
          type: number
          example: 1
          description: The result's position in the ranked list.
        data:
          description: The scraped page for this result, or null when scraping is off or unavailable.
          type:
          - object
          - 'null'
          allOf:
          - $ref: '#/components/schemas/WebScrapeDataDto'
      required:
      - title
      - url
      - snippet
      - rank
      - data
    SearchRequestDto:
      type: object
      properties:
        query:
          type: string
          maxLength: 500
          example: best open source vector databases
          description: What to search the web for.
        limit:
          type: number
          minimum: 1
          maximum: 15
          example: 5
          description: How many ranked results to return. Omit for the default.
        scrapeResults:
          type: boolean
          example: false
          description: When true, each result also comes back with its page fetched and returned as clean data, at 1 additional credit per page scraped. Defaults to false, which returns just the ranked title, url and snippet for the flat search credit.
        scrapeOptions:
          $ref: '#/components/schemas/ScrapeOptionsDto'
        timeRange:
          type: string
          enum:
          - day
          - week
          - month
          - year
          example: week
          description: Restrict results to a recency window relative to now.
        country:
          type: string
          example: us
          description: ISO 3166 alpha-2 country code to bias results toward a region.
        includeDomains:
          maxItems: 15
          example:
          - github.com
          - arxiv.org
          description: Only return results from these domains.
          type: array
          items:
            type: string
        excludeDomains:
          maxItems: 15
          example:
          - pinterest.com
          description: Drop results from these domains.
          type: array
          items:
            type: string
      required:
      - query
    SearchResultDto:
      type: object
      properties:
        query:
          type: string
          example: best open source vector databases
          description: The query you searched.
        status:
          type: string
          enum:
          - ok
          - no_results
          example: ok
          description: 'Whether the search found anything: `ok` when it returned results, `no_results` when the web had no match. An empty `results` with `no_results` is a real miss — if the search cannot be completed at all, the request fails with 503 rather than returning an empty list.'
        results:
          description: The ranked results.
          type: array
          items:
            $ref: '#/components/schemas/SearchResultItemDto'
      required:
      - query
      - status
      - results
    WebErrorMeta:
      type: object
      properties:
        requestId:
          type: string
          description: Quote this when contacting support about a specific failure.
          example: 019e8a3c-9f0b-7c12-88ab-1d2e3f4a5b6c
    WebQualityDto:
      type: object
      properties:
        confidence:
          type: number
          example: 0.94
          minimum: 0
          maximum: 1
          description: How trustworthy the extraction is, from 0 to 1. High when independent checks corroborate substantial content; near zero when the page yielded almost nothing.
        complete:
          type: boolean
          example: true
          description: Whether the result captured the bulk of the content available on the page. False when the result looks thin or truncated.
        blocked:
          type: boolean
          example: false
          description: Whether the page appeared to be behind a challenge or bot wall.
      required:
      - confidence
      - complete
      - blocked
    WebError:
      type: object
      required:
      - success
      - error
      - meta
      description: Every failure on this API returns this shape, whatever the status. Successful responses never carry it.
      properties:
        success:
          type: boolean
          enum:
          - false
          example: false
        error:
          $ref: '#/components/schemas/WebErrorBody'
        meta:
          $ref: '#/components/schemas/WebErrorMeta'
    ScrapeOptionsDto:
      type: object
      properties:
        formats:
          type: array
          example:
          - markdown
          description: What to return per page. Omit for Markdown only.
          items:
            type: string
            enum:
            - markdown
            - html
            - rawHtml
            - links
            - structured
        onlyMainContent:
          type: boolean
          description: Return only the main content, dropping boilerplate. Default true.
        includeTags:
          maxItems: 50
          description: CSS selectors to keep.
          type: array
          items:
            type: string
        excludeTags:
          maxItems: 50
          description: CSS selectors to strip before extraction.
          type: array
          items:
            type: string
        removeBase64Images:
          type: boolean
          description: Strip inline base64 images. Default true.
        blockAds:
          type: boolean
          description: Remove common ad and tracking elements. Default true.
        includeLinks:
          type: boolean
          description: Keep inline links in the markdown. By default links are kept wherever the page text carries them. Set true to keep them on every page, or false for the densest prose.
        waitFor:
          type: number
          minimum: 0
          maximum: 30000
          description: Extra milliseconds to let the page settle before capture.
        timeout:
          type: number
          minimum: 1000
          maximum: 100000
          description: Per-page time budget, in milliseconds.
        location:
          $ref: '#/components/schemas/OptLocationDto'
        headers:
          type: object
          additionalProperties:
            type: string
          description: Extra request headers to send when fetching each page.
        preferStructure:
          type: boolean
          description: Preserve document structure over prose density. Default off.
        maxAge:
          type: number
          minimum: 0
          maximum: 604800000
          description: Serve a page from cache if younger than this many milliseconds.
  securitySchemes:
    apiKey:
      type: apiKey
      in: header
      name: X-API-Key