crawleo-dev

crawleo-dev Crawl API

The Crawl API from crawleo-dev — 1 operation(s) for crawl.

Documentation GitHub OpenAPI

Operations 1

POST /crawl Crawl a list of URLs #

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this

find_apisBrowse and filter every API in the catalog.
get_api_artifactsOne API's artifacts, grouped by type.
get_openapiThe primary OpenAPI for this API.
find_similar_apisAPIs that look like this one.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This API

curl "https://apis.io/api/v1/apis/crawleo-dev:crawleo-dev-crawl-api"

All apis

curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

openapi: 3.2.0
info:
  title: Crawleo Crawl API
  description: AI-powered web scraping and search API. Extract data from any website with ease.
  version: 1.0.0
  contact:
    name: Crawleo Support
    email: support@crawleo.dev
    url: https://www.crawleo.dev
  license:
    name: MIT
servers:
- url: https://api.crawleo.dev
  description: Production
security:
- bearerAuth: []
- apiKeyHeader: []
tags:
- name: Crawl
paths:
  /crawl:
    post:
      summary: Crawl a list of URLs
      description: Crawls the provided URLs using a headful browser and returns extracted data.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                urls:
                  type: array
                  items:
                    type: string
                    format: uri
                  description: List of URLs to crawl
                options:
                  type: object
                  properties:
                    render:
                      type: boolean
                      default: true
                    extract:
                      type: boolean
                      default: true
              required:
              - urls
      responses:
        '200':
          description: Crawl results
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CrawlResponse'
        '400':
          description: Bad request
        '401':
          description: Unauthorized
      tags:
      - Crawl
      operationId: postCrawl
      x-operation-id-source: derived
components:
  schemas:
    CrawlResponse:
      type: object
      properties:
        results:
          type: array
          description: Array of crawl result objects
          items:
            type: object
            properties:
              url:
                type: string
                description: The URL that was crawled
              status:
                type: integer
                description: HTTP status code of the crawled page
              raw_html:
                type: string
                description: Raw HTML content (if requested)
              enhanced_html:
                type: string
                description: Cleaned HTML with ads and scripts removed
              page_text:
                type: string
                description: Plain text extraction of the page
              markdown:
                type: string
                description: Markdown conversion of the page content
              screenshot:
                type: string
                description: Base64 encoded screenshot (if requested)
              error:
                type: string
                description: Error message if the crawl failed for this URL
        credits:
          type: integer
          description: Total credits used for this request
        successful_pages:
          type: integer
          description: Number of URLs successfully crawled
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: 'Bearer token authentication using your API key. Format: Bearer YOUR_API_KEY'
    apiKeyHeader:
      type: apiKey
      in: header
      name: x-api-key
      description: API key for authentication