Algolia

Algolia Crawler API

Manages Algolia's hosted web crawler that extracts content from websites and pushes it into indices on a schedule. The only Algolia API that authenticates with HTTP Basic rather than the x-algolia-* header pair.

GitHub OpenAPI

Operations 20

GET /1/crawlers List crawlers #
POST /1/crawlers Create a crawler #
GET /1/crawlers/{id} Retrieve crawler details #
PATCH /1/crawlers/{id} Replace crawler configuration #
DELETE /1/crawlers/{id} Delete a crawler #
POST /1/crawlers/{id}/run Unpause a crawler #
POST /1/crawlers/{id}/pause Pause a crawler #
POST /1/crawlers/{id}/reindex Start a crawl #
POST /1/crawlers/{id}/test Test crawl a URL #
POST /1/crawlers/{id}/urls/crawl Crawl URLs #
GET /1/crawlers/{id}/stats/urls Retrieve crawler stats #
GET /1/crawlers/{id}/crawl_runs List crawler runs #
GET /1/crawlers/{id}/{logId}/download Crawler run file #
POST /1/crawlers/{id}/delete_runs Crawler run record deletion #
PATCH /1/crawlers/{id}/config Update crawler configuration #
GET /1/crawlers/{id}/config/versions List configuration versions #
GET /1/crawlers/{id}/config/versions/{version} Retrieve a configuration version #
GET /1/crawlers/{id}/tasks/{taskID} Retrieve task status #
POST /1/crawlers/{id}/tasks/{taskID}/cancel Cancel a blocking task #
GET /1/domains List registered domains #

Documentation

📖
Documentation
https://www.algolia.com/doc/tools/crawler/getting-started/overview/

Specifications

⚙
OpenAPI
https://raw.githubusercontent.com/api-evangelist/algolia/refs/heads/main/openapi/algolia-crawler-api-openapi.yml

Other Resources

🔗
Overlay
https://raw.githubusercontent.com/api-evangelist/algolia/refs/heads/main/overlays/algolia-crawler-api-overlay.yaml

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/algolia-crawler-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

algolia-crawler-api-openapi.yml Raw ↑ 
openapi: 3.2.0
info:
  title: Crawler API
  summary: The Crawler API lets you manage and run your crawlers
  description: '## Base URL


    The base URL for making requests to the Crawler API is:


    - `https://crawler.algolia.com/api`


    **All requests must use HTTPS.**


    ## Availability and authentication


    To authenticate your API requests, use the **basic authentication** header:


    - `Authorization: Basic `


    Where `` is a base64-encoded string `:`.'
  version: 1.0.0
servers:
- url: https://crawler.algolia.com/api
  description: The URL of the Crawler API.
security:
- BasicAuth: []
tags:


# --- truncated at 32 KB (121 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/algolia/refs/heads/main/openapi/algolia-crawler-api-openapi.yml