SerpKite Crawl API

openapi: 3.2.0
info:
  title: SerpKite SERP Crawl API
  version: 1.12.4
  description: 'Public search API (api.serpkite.com). Google results as clean JSON or Markdown

    for AI agents, plus a Google Custom Search–compatible endpoint. Official SDKs:

    `serpkite` on npm and PyPI, `github.com/serpkite/serpkite-go`. **Auth:** `Authorization: Bearer skt_live_…`. A bare `Authorization: skt_live_…`

    (no scheme) is also accepted, for MCP hosts that can''t add one. GET requests

    may pass `?api_key=` instead. `/customsearch/v1` also accepts Google''s `?key=`. **Bodies:** every vertical accepts `POST` with a JSON object or `GET` with the

    same parameters in the query string (only `GET /v1/search` is spelled out

    here). Unknown parameters are rejected with 400, and so is a parameter the

    endpoint would ignore (e.g. `time` on `/v1/scholar`, `ll` on `/v1/search`). To run many queries at once,

    use `POST /v1/batches`. **Formats:** `format=json` (default) returns the schemas below. `format=markdown` returns `text/markdown` for every vertical. `format=compact` and `fields=` return a reduced JSON object (same top-level

    keys, fewer item fields, `meta` always kept), so required item fields may be

    absent there. **Responses** share one envelope: `request` (the normalised request),

    `results` (the vertical''s primary list), vertical-specific extras and `meta`. All keys are snake_case. **Credits** are decimal numbers (e.g. `0.5`). Failed, empty and blocked

    responses are refunded. Every billed response carries `X-Credits-Used`,

    `X-Credits-Remaining`, `X-Cost-USD`, `X-Cache` (`HIT`|`MISS`), `X-Latency-Ms`,

    `X-Tokens-Estimate` and `X-Request-Id`. **Errors** share one body: `{"error":{"code","message","request_id"}}`. Upstream failures (`upstream_error`, `upstream_blocked`, `upstream_timeout`)

    are `503` with `Retry-After`, never billed and safe to retry. The API never

    returns `502` or `504`. `429` also carries `Retry-After`: `rate_limited` is

    the per-second limit; `daily_limit_reached` is the free daily limit of an

    account that never bought a pack, which resets at 00:00 UTC (the message

    names the limit and the billing page; buying any pack removes it). `403 account_restricted` means the account''s free use is paused because it

    appears to be one of several accounts run by the same person (same device

    or network, or requests in step with other accounts) or uses a disposable

    email address; buying any pack lifts it. **CORS:** `Access-Control-Allow-Origin: *` without credentials; the `X-*`

    headers above are exposed, so a browser playground can call the API with a key. The keyless `/playground/search` is the exception: it only answers serpkite.com

    and app.serpkite.com (other `Origin`s get 403). Credit table: search/news/images/videos/maps/places/shopping/scholar/patents/

    webpage/customsearch = 1 per page; place details = 1; `num=100` depth bundle = 7; reviews = 1 per 10;

    autocomplete = 0.5; `include_content` = +1 per fetched page;

    batches ×0.5 (from 2026-11-23 a batch depth request is 0.5 per page with

    results, without the bundle cap: `num=100` = 5); cache hit via `max_age` ×0.5. `engine=consensus` (search only) =

    the sum of one page per provider that returned results. **AI** (Claude writes, OpenAI''s Decisions API runs typed checks; a failed

    model step is never billed): an answer (`include_answer` on `/v1/search` and

    `/v1/news`, the model part of `/v1/answer`) = +2 over the result page, +4

    when it read pages (`include_content`, `/v1/answer` depth=standard);

    `/v1/extract` `summary` = +4 and `output_schema` = +4 per page;

    `/v1/research` = lite 30, standard 75, deep 200. Responses that used a

    model carry `meta.llm`. **AI assistants** (`/v1/chatgpt`, `/v1/gemini`, `/v1/copilot`,

    `/v1/ai_mode`, `/v1/brave_ai`): a `prompt` sent to the public assistant

    as a logged-out user; the response is its answer

    (`text`, `markdown`) with the `sources` it cited or listed,

    `search_queries`, `products` and `places` where the assistant shows them,

    and `mentions` for the brands and domains in `track`.'
servers:
- url: https://api.serpkite.com
- url: http://localhost:8080
security:
- bearer: []
tags:
- name: Crawl
paths:
  /v1/crawl:
    post:
      summary: Start an async crawl of one site (1 credit per page read; `limit` credits…
      operationId: createCrawl
      description: 'From `url` and (unless `sitemap: skip`) the site''s sitemaps, same host (optionally

        subdomains), up to `limit` pages and `max_depth` link hops, filtered by

        `include_paths` / `exclude_paths` (regular expressions matched against the URL

        path). Breadth first, or best first with `query` (pages whose URL and link text

        match it are read first). Each host''s robots.txt is honoured (RFC 9309: rules,

        `Crawl-delay`; a robots.txt that answers 5xx stops the crawl at the start page).

        URLs are canonicalised (fragments and tracking parameters dropped, `www.`,

        trailing slashes and redirects folded) so each page is read once. Each page comes

        back as Markdown (or text) with its metadata. A crawl runs for at most 30 minutes

        and returns up to 16 MB of pages; `result.stats.stopped` says why it ended.


        Returns `202` with a task; poll `GET /v1/crawl/{id}` or wait for the signed

        `crawl.completed` webhook. Pages read cost 1 credit each (0.5 from cache); a crawl

        that read nothing fails (`no_pages`) and is refunded. At most 20 crawls per account

        may be queued or running at once (`429`); a crawl interrupted by a deploy restarts

        from the beginning.'
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CrawlRequest'
            example:
              url: https://docs.example.com/
              limit: 50
              max_depth: 2
              include_paths:
              - ^/guides/
      responses:
        '202':
          $ref: '#/components/responses/TaskAccepted'
        '400':
          $ref: '#/components/responses/Error'
        '401':
          $ref: '#/components/responses/Error'
        '402':
          $ref: '#/components/responses/Error'
        '403':
          $ref: '#/components/responses/Error'
        '429':
          $ref: '#/components/responses/Error'
        '503':
          $ref: '#/components/responses/Unavailable'
      tags:
      - Crawl
  /v1/crawl/{id}:
    parameters:
    - name: id
      in: path
      required: true
      schema:
        type: string
        format: uuid
    get:
      summary: Poll a crawl (kept for 24 hours)
      operationId: getCrawl
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CrawlTask'
        '401':
          $ref: '#/components/responses/Error'
        '404':
          $ref: '#/components/responses/Error'
        '429':
          $ref: '#/components/responses/Error'
      tags:
      - Crawl
    delete:
      summary: Cancel a crawl.
      description: '`404` when there is no such crawl, `409` when it already finished.'
      operationId: cancelCrawl
      responses:
        '200':
          $ref: '#/components/responses/TaskCanceled'
        '401':
          $ref: '#/components/responses/Error'
        '404':
          $ref: '#/components/responses/Error'
        '409':
          $ref: '#/components/responses/Error'
        '429':
          $ref: '#/components/responses/Error'
      tags:
      - Crawl
components:
  responses:
    Unavailable:
      description: 'Temporarily unavailable, not billed, safe to retry after `Retry-After`.

        Codes: `upstream_error` (the upstream page couldn''t be fetched or parsed),

        `upstream_blocked` (the upstream is rate limiting us or refused the query), `upstream_timeout`

        (the search took too long), `unavailable` (endpoint or dependency down).

        The API never sends 502 or 504.

        '
      headers:
        Retry-After:
          $ref: '#/components/headers/RetryAfter'
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
    TaskAccepted:
      description: Queued. Poll `poll_url` or wait for the webhook
      headers:
        X-Credits-Reserved:
          description: The maximum price, reserved until the task ends
          schema:
            type: number
        X-Credits-Remaining:
          $ref: '#/components/headers/CreditsRemaining'
        Location:
          description: Path of the task (`/v1/{kind}/{id}`)
          schema:
            type: string
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/TaskCreated'
          example:
            id: 01926a3e-7b2c-7c4e-9f1a-2b3c4d5e6f70
            kind: crawl
            status: queued
            created_at: '2026-10-03T12:00:00Z'
            poll_url: https://api.serpkite.com/v1/crawl/01926a3e-7b2c-7c4e-9f1a-2b3c4d5e6f70
            credits_reserved: 50
            webhook_url: null
    TaskCanceled:
      description: '`canceled` (it was queued: refunded at once) or `canceling` (it is running and stops at its next checkpoint)'
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/TaskCancelResponse'
    Error:
      description: Error
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
  schemas:
    CrawlRequest:
      type: object
      required:
      - url
      properties:
        url:
          type: string
          format: uri
          description: 'Start page (always read: path filters and robots.txt apply to the pages found from it)'
        limit:
          type: integer
          minimum: 1
          maximum: 1000
          default: 25
          description: Maximum pages read (and credits reserved)
        max_depth:
          type: integer
          minimum: 0
          maximum: 10
          default: 2
          description: Link hops from url (sitemap pages count as 1 hop; 0 reads only url)
        include_paths:
          type: array
          items:
            type: string
          maxItems: 20
          description: Regular expressions matched against the URL path; follow only paths matching one
        exclude_paths:
          type: array
          items:
            type: string
          maxItems: 20
          description: Regular expressions matched against the URL path; never follow paths matching one
        include_subdomains:
          type: boolean
          default: false
        sitemap:
          type: string
          enum:
          - include
          - only
          - skip
          default: include
          description: '`include` also seeds the crawl with the site''s sitemap URLs (after the start page''s links), `only` reads the start page and sitemap URLs without following links, `skip` follows links only'
        query:
          type: string
          maxLength: 512
          description: Read the pages most relevant to these words first (best-first crawl)
        ignore_query_parameters:
          type: boolean
          default: false
          description: Treat URLs that differ only in their query string as one page
        format:
          type: string
          enum:
          - markdown
          - text
          default: markdown
        include_links:
          type: boolean
          default: false
          description: Return each page's outbound links
        max_tokens:
          type: integer
          minimum: 100
          maximum: 100000
          description: Trim each page to this many tokens
        max_age:
          type: integer
          minimum: 0
          maximum: 2592000
          description: Accept cached pages up to this many seconds old (0.5 credit each)
        webhook_url:
          type: string
          format: uri
          description: Receives the signed `crawl.completed` event. Defaults to the account webhook
    CrawlResult:
      type: object
      required:
      - url
      - pages
      - failed
      - stats
      properties:
        url:
          type: string
        pages:
          type: array
          items:
            $ref: '#/components/schemas/CrawlPage'
        failed:
          type: array
          items:
            $ref: '#/components/schemas/ExtractFailure'
        stats:
          type: object
          properties:
            pages:
              type: integer
            failed:
              type: integer
            seconds:
              type: integer
            discovered:
              type: integer
              description: Unique URLs found that passed the filters
            queued:
              type: integer
              description: Found but not read when the crawl ended
            duplicates:
              type: integer
              description: Pages that redirected to a page already read (not charged)
            sitemap_urls:
              type: integer
              description: URLs taken from sitemaps
            robots:
              type: string
              enum:
              - found
              - missing
              - unreachable
              - none
              description: The start host's robots.txt
            robots_disallowed:
              type: integer
              description: Disallow rules in force on the start host
            robots_blocked:
              type: integer
              description: URLs skipped because robots.txt disallows them
            crawl_delay_ms:
              type: integer
              description: The start host's Crawl-delay, honoured
            stopped:
              type: string
              enum:
              - done
              - limit
              - time_limit
              - size_limit
              - too_many_failures
              - canceled
              description: Why the crawl ended
    PageLink:
      type: object
      required:
      - url
      properties:
        url:
          type: string
          description: Absolute URL
        text:
          type: string
          description: Anchor text
    CrawlPage:
      type: object
      required:
      - url
      - depth
      properties:
        url:
          type: string
        depth:
          type: integer
          description: Link hops from the start page
        title:
          type: string
        markdown:
          type: string
        text:
          type: string
        published_at:
          type: string
        metadata:
          $ref: '#/components/schemas/PageMetadata'
        links:
          type: array
          items:
            $ref: '#/components/schemas/PageLink'
    PageMetadata:
      type: object
      properties:
        title:
          type: string
        description:
          type: string
        language:
          type: string
        canonical:
          type: string
        site_name:
          type: string
        image:
          type: string
        published_time:
          type: string
        author:
          type: string
        content_type:
          type: string
          description: The page's media type
          examples:
          - text/html
          - application/pdf
        pages:
          type: integer
          description: Page count of a PDF
    CrawlTask:
      type: object
      required:
      - id
      - kind
      - status
      - created_at
      - credits_reserved
      - credits_used
      properties:
        id:
          type: string
          format: uuid
        kind:
          type: string
          const: crawl
        status:
          $ref: '#/components/schemas/TaskStatus'
        created_at:
          type: string
          format: date-time
        started_at:
          type:
          - string
          - 'null'
          format: date-time
        completed_at:
          type:
          - string
          - 'null'
          format: date-time
        credits_reserved:
          type: number
        credits_used:
          type: number
        webhook_status:
          type:
          - string
          - 'null'
          enum:
          - pending
          - delivered
          - failed
          - null
        progress:
          type:
          - object
          - 'null'
          additionalProperties: true
          properties:
            pages_done:
              type: integer
            pages_failed:
              type: integer
            pages_queued:
              type: integer
            pages_discovered:
              type: integer
            limit:
              type: integer
        result:
          description: Set once the crawl ends (also on cancel, with the pages read so far)
          anyOf:
          - $ref: '#/components/schemas/CrawlResult'
          - type: 'null'
        error:
          $ref: '#/components/schemas/TaskError'
    ExtractFailure:
      type: object
      required:
      - url
      - error
      properties:
        url:
          type: string
        error:
          type: object
          required:
          - code
          - message
          properties:
            code:
              type: string
              examples:
              - invalid_request
              - upstream_error
              - upstream_timeout
              - empty_page
            message:
              type: string
    TaskError:
      type:
      - object
      - 'null'
      description: Set when the task failed or was canceled
      properties:
        code:
          type: string
          examples:
          - no_pages
          - canceled
          - timeout
          - internal
        message:
          type: string
    TaskCancelResponse:
      type: object
      required:
      - id
      - status
      properties:
        id:
          type: string
          format: uuid
        status:
          type: string
          enum:
          - canceled
          - canceling
    TaskCreated:
      type: object
      required:
      - id
      - kind
      - status
      - created_at
      - poll_url
      - credits_reserved
      properties:
        id:
          type: string
          format: uuid
        kind:
          type: string
          enum:
          - crawl
          - research
        status:
          $ref: '#/components/schemas/TaskStatus'
        created_at:
          type: string
          format: date-time
        poll_url:
          type: string
        credits_reserved:
          type: number
        webhook_url:
          type:
          - string
          - 'null'
    TaskStatus:
      type: string
      enum:
      - queued
      - running
      - completed
      - failed
      - canceled
    Error:
      type: object
      required:
      - error
      properties:
        error:
          type: object
          required:
          - code
          - message
          properties:
            code:
              type: string
              description: '403: spend_cap_reached, key_limit_reached, account_restricted, forbidden. 409/422: idempotency_key_reused. 422: target_unavailable (the URL you asked for failed: its site answered 404, its domain doesn''t resolve; resending won''t help), no_pages. 429: rate_limited, daily_limit_reached. 503: upstream_error, upstream_blocked, upstream_timeout, unavailable (the API never sends 502 or 504)'
              examples:
              - unauthorized
              - invalid_request
              - insufficient_credits
              - rate_limited
              - daily_limit_reached
              - spend_cap_reached
              - key_limit_reached
              - account_restricted
              - idempotency_key_reused
              - forbidden
              - not_found
              - upstream_error
              - upstream_blocked
              - upstream_timeout
              - unavailable
              - internal
            message:
              type: string
            request_id:
              type: string
  headers:
    RetryAfter:
      description: Seconds to wait before retrying
      schema:
        type: integer
    CreditsRemaining:
      schema:
        type: number
  securitySchemes:
    bearer:
      type: http
      scheme: bearer
    oauth:
      type: oauth2
      description: OAuth 2.1 for the remote MCP server only (authorization code + PKCE S256; app-api.serpkite.com is the authorization server).
      flows:
        authorizationCode:
          authorizationUrl: https://app-api.serpkite.com/oauth/authorize
          tokenUrl: https://app-api.serpkite.com/oauth/token
          refreshUrl: https://app-api.serpkite.com/oauth/token
          scopes:
            mcp: Use the SerpKite MCP tools, billed to your account
            offline_access: Stay connected with refresh tokens