octocrawl-dev

Octocrawl provides a web scraping API that extracts web pages as Markdown and structured data via REST, SDKs or MCP. It offers hosted scrape, map, crawl and batch jobs, or can run locally. The service delivers clean Markdown and structured data from public pages, each with a receipt, supporting JSON-LD, microdata, meta tags, tables and PDF label-value extraction.

octocrawl-dev is listed with 2 APIs on the APIs.io network. Tagged areas include Company, API, WebScraping, DataExtraction, and SaaS.

The octocrawl-dev catalog on APIs.io includes 1 JSON-LD context and 1 Spectral governance ruleset.

octocrawl-dev’s developer surface includes CLI, changelog, authentication, documentation, engineering blog, API reference, and 24 more developer resources.

🌐 Visit website 📡 Profile data on GitHub

57.5/100 strong Agent 47/100 agent native Full breakdown ↓
scored 2026-10-31 · rubric v0.25.0 · Show this score on your site →

2 APIs 1 MCP Servers

CompanyAPIWebScrapingDataExtractionSaaSAutomation

Kin Score

Kin Score How this is scored →
scored 2026-10-31 · rubric v0.25.0

Composite quality — 57.5/100 · strong

Contract Quality 11.7 / 21

Developer Ergonomics 10.4 / 17

Access Clarity 5.8 / 17

Operational Transparency 4.9 / 11

Contract Governance 8.1 / 10

Discoverability 5.5 / 9

Regulatory Posture 5.1 / 15

Agent readiness — 47/100 · agent native

Machine-Readable Contract 18 / 18

Agentic Access Contract 0 / 10

Documented Reversibility 0 / 6

MCP Server 12 / 12

Machine-Readable Auth 10 / 10

Idempotency 9 / 9

Stable Error Semantics 8 / 8

Request/Response Examples 0 / 7

Rate-Limit Signaling 7 / 7

Typed Event Surface 0 / 6

Agent Skills 5 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Delegated User Identity 6 / 6

Protected Resource Metadata 0 / 5

Registration Without a Human 6 / 6

Agentic Commerce Well-Known Document 0 / 5

Pagination 5 / 5

Distinct Operations 3 / 3

Regulatory Posture applies to this provider. Its tags matched the Horizontal (data, software, accessibility, platform) regime, so Regulatory Posture carries 15 points of the composite. If this regime is wrong for your business, say so on your provider repo — the applicability map is public and we will correct it.

Create-or-Update Ergonomics could not be measured. We hold no machine-readable contract for this provider to read, so there is nothing to measure a write surface against. Excluded rather than scored zero: never-measured and measured-empty are different facts. Publishing an OpenAPI is what makes this facet — and several others — scorable at all.

The six quality facets above are damped to 85 points between them, because the conditional facet above carries the other 15. That is why each facet's contribution is shown against a damped maximum: raising a quality facet moves the composite by 85% of its nominal weight, not 100%. The full arithmetic is at apis.io/rating/.

Show this score on your site.

It updates itself when the score changes. Paste this where you link to your API docs:

<a href="https://apis.io/providers/octocrawl-dev/"><img src="https://apis.io/badge/octocrawl-dev/card.svg" alt="octocrawl-dev on APIs.io — Kin Score by API Evangelist"></a>

Other styles, a README pill and a shields.io endpoint are on the badge page.

Raise your score.

Agent readiness — checks not met, largest first (exact, from this profile):

Agentic Access Contract +10 pts — agentic_access artifact present (x-agentic-access), graded by author
A2A Agent Card +8 pts — a2a/-a2a.yml manifest, graded on conformance.grade against A2A 1.0.0
Request/Response Examples +7 pts — share of operations carrying request/response examples; else an Examples / Code Samples link
Typed Event Surface +6 pts — AsyncAPI document, OR webhooks/callbacks objects in the as-published OpenAPI, OR common[].type includes Webhooks
Documented Reversibility +6 pts — a reversal operation in the contract, graded up when a window is documented too
Agentic Commerce Well-Known Document +5 pts — a parsing /.well-known/ucp.json or acp.json declaring a version and at least one service — `self` when its endpoints are on the provider's own domain, `platform` when they are on a vendor's

4 more unmet checks in the Agent Readiness table above.

Kin Score — facets with the most composite points still available (weight × headroom):

Access Clarity scores 34.2 — up to +13.2 composite points
Contract Quality scores 55.2 — up to +11.2 composite points
Developer Ergonomics scores 61.2 — up to +7.8 composite points

Each check links to what satisfies it. Publish the artifact, or tell us where it already lives, and the score moves on the next scoring run.

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. Every facet and dimension name above is a link: it opens that measurement's own page — what it means, the exact checks that feed it, how the whole catalog distributes on it, and the providers at the top of it. This rating is computed from the provider profile at artifacts.apis.io/octocrawl-dev/apis.yml: open an issue on github.com/api-evangelist/octocrawl-dev to ask a question or point us at an artifact you publish. Submit an artifact on GitHub — free → Manage your own listing — the Influence plan, $499/mo →

APIs 2

Individual APIs this provider publishes, each with its own machine-readable definition.

Octocrawl API

Octocrawl REST API for scraping, mapping, crawling and batch jobs.

octocrawl-dev API

octocrawl-dev API as documented publicly: 31 operations. Contract generated from the documentation by API Evangelist (2026-10-30); not the provider's own document.

MCP Servers 1

Model Context Protocol servers that expose these APIs to AI agents.

octocrawl-dev MCP Server

Remote MCP server at mcp.octocrawl.dev.

MCP SERVER

Pricing Plans 1

Published pricing tiers and plan structures.

Octocrawl Dev Plans Pricing

0 plans

PLANS

Rate Limits 1

Documented rate limits and quota policies.

Octocrawl Dev Rate Limits

3 limits

RATE LIMITS

Semantic Vocabularies 1

JSON-LD contexts and semantic vocabularies used across these APIs.

Octocrawl Dev Context

29 classes · 40 properties

JSON-LD

Spectral Rules 1

Spectral governance rulesets for linting and validating these APIs.

octocrawl-dev API Rules

12 rules · 10 errors 1 warnings 1 info

SPECTRAL

JSON Schema 6

Standalone JSON Schema definitions for this provider's data models.

GetApiCapabilityResponse

2 properties

JSON SCHEMA

GetApiQuotaResponse

2 properties

JSON SCHEMA

PostCrawlRequest

3 properties

JSON SCHEMA

PostV1LoginsImportRequest

2 properties

JSON SCHEMA

PostV1LoginsImportResponse

6 properties

JSON SCHEMA

PostV1ScrapeRequest

6 properties

JSON SCHEMA

Security Posture 2

Authentication, domain security, vulnerability disclosure, and trust-center signals.

Octocrawl Dev Authentication

1 scheme

SECURITY

Octocrawl Dev Domain Security

TLSv1.3 · HSTS

SECURITY

Scopes 1

OAuth scopes governing access to this provider's APIs.

Octocrawl Dev Scopes

OAuth 2.0 · no documented scopes

0 scopes

SCOPES

Resources

Get Started 2

Portal, sign-up, and the first successful call

Login

DeveloperPortal

DeveloperPortal

Documentation 2

Reference material describing how the API behaves

Documentation

Documentation

APIReference

APIReference

Agent Surfaces 4

MCP servers, agent skills, and machine-readable catalogs

AgentSkill

LLMsTxt

MCPServer

WellKnown

Design & Contract 8

Pagination, idempotency, versioning, errors, and events

Spectral

JSONLD

Vocabulary

DataModel

Idempotency

Conventions

ErrorCatalog

Conformance

Scroll for all 8

Build 3

SDKs, sample code, and the tooling you integrate with

CLI

CLI

Packages

Packages

SDKs

SDKs

Access & Security 2

Authentication, authorization, and security posture

Authentication

Authentication

DomainSecurity

DomainSecurity

Operate 3

Status, limits, changes, and where to get help

RateLimits

RateLimits

ChangeLog

ChangeLog

ChangeLog

ChangeLog

Commercial 2

Pricing, plans, and the legal terms of use

TermsOfService

TermsOfService

PrivacyPolicy

PrivacyPolicy

Company 2

The organization behind the API

Blog

Blog

Website

Website

Other 2

Properties that don't map to a standard resource type

Hosts

Hosts

Vendors

Vendors

Source (apis.yml)

aid: octocrawl-dev
name: octocrawl-dev
description: Octocrawl provides a web scraping API that extracts web pages as Markdown and structured data via REST, SDKs
  or MCP. It offers hosted scrape, map, crawl and batch jobs, or can run locally. The service delivers clean Markdown and
  structured data from public pages, each with a receipt, supporting JSON-LD, microdata, meta tags, tables and PDF label-value
  extraction.
url: https://artifacts.apis.io/octocrawl-dev/apis.yml
image: https://octocrawl.dev/assets/octopus-ghost.webp
x-type: company
x-source: harvest:mcp-registry
x-tier: profiled
x-tier-reason: enrichment pass 2026-10-30 (modular-v1; was harvest)
specificationVersion: '0.20'
created: '2026-10-30'
modified: '2026-10-30'
tags:
- Company
- API
- WebScraping
- DataExtraction
- SaaS
- Automation
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com
- FN: APIs.json
  email: info@apis.io
common:
- type: RateLimits
  url: rate-limits/octocrawl-dev-rate-limits.yml
- type: Spectral
  url: rules/octocrawl-dev-rules.yml
- type: JSONLD
  url: json-ld/octocrawl-dev-context.jsonld
- type: Vocabulary
  url: vocabulary/octocrawl-dev-vocabulary.yml
- type: AgentSkill
  url: skills/_index.yml
- type: DataModel
  url: data-model/octocrawl-dev-data-model.yml
- type: CLI
  url: cli/octocrawl-dev-cli.yml
- type: ChangeLog
  url: changelog/octocrawl-dev-changelog.yml
- type: Idempotency
  url: conventions/octocrawl-dev-conventions.yml
- type: Conventions
  url: conventions/octocrawl-dev-conventions.yml
- type: Authentication
  url: authentication/octocrawl-dev-authentication.yml
- type: ErrorCatalog
  url: errors/octocrawl-dev-problem-types.yml
- type: Conformance
  url: conformance/octocrawl-dev-conformance.yml
- type: LLMsTxt
  url: llms/octocrawl-dev-llms.txt
- type: MCPServer
  url: mcp/octocrawl-dev-mcp.yml
- type: WellKnown
  url: well-known/octocrawl-dev-well-known.yml
- type: Hosts
  url: hosts/octocrawl-dev-hosts.yml
- type: Vendors
  url: vendors/octocrawl-dev-vendors.yml
- type: Packages
  url: packages/octocrawl-dev-packages.yml
- type: TermsOfService
  url: https://octocrawl.dev/docs/terms/
- type: SDKs
  url: https://www.npmjs.com/package/@octocrawl/sdk
- type: PrivacyPolicy
  url: https://octocrawl.dev/docs/privacy/
- type: Login
  url: https://octocrawl.dev/signin/
- type: Documentation
  url: https://octocrawl.dev/docs/
- type: ChangeLog
  url: https://octocrawl.dev/changelog/
- type: Blog
  url: https://octocrawl.dev/blog/
- type: APIReference
  url: https://octocrawl.dev/docs/reference/
- type: DeveloperPortal
  url: https://api.octocrawl.dev`
- type: DomainSecurity
  url: security/octocrawl-dev-domain-security.yml
- type: Website
  url: https://octocrawl.dev/
apis:
- name: Octocrawl API
  description: Octocrawl REST API for scraping, mapping, crawling and batch jobs.
  humanURL: https://octocrawl.dev/docs/reference/
  baseURL: https://api.octocrawl.dev
  aid: octocrawl-dev:octocrawl-api
- aid: octocrawl-dev:octocrawl-dev-api
  name: octocrawl-dev API
  description: 'octocrawl-dev API as documented publicly: 31 operations. Contract generated from the documentation by API
    Evangelist (2026-10-30);

    not the provider''s own document.'
  humanURL: https://octocrawl.dev/docs/reference
  baseURL: https://api.octocrawl.dev
  properties:
  - type: OpenAPI
    url: openapi/_ae-authored/octocrawl-dev-openapi-generated.yml
    x-provenance: generated
  - type: JSONSchema
    url: json-schema/octocrawl-dev-post-crawl-request-schema.json
  - type: JSONSchema
    url: json-schema/octocrawl-dev-post-v1-logins-import-response-schema.json
  - type: JSONSchema
    url: json-schema/octocrawl-dev-get-api-quota-response-schema.json
  - type: JSONSchema
    url: json-schema/octocrawl-dev-post-v1-scrape-request-schema.json
  - type: JSONSchema
    url: json-schema/octocrawl-dev-post-v1-logins-import-request-schema.json
  - type: JSONSchema
    url: json-schema/octocrawl-dev-get-api-capability-response-schema.json
x-coverage:
  state: unreadable
  reason: no-machine-readable-spec
  detail: No OpenAPI or other machine-readable contract found despite probing known spec URLs on the API host.
  evidence:
  - url: https://api.octocrawl.dev/openapi.json
    status: 403
  checked: '2026-10-31'
x-enrichment:
  date: '2026-10-30'
  status: enriched
  artifacts_added: 38
  pass: modular-v1
  engine: identity=oss,contract=oss,surfaces=oss,docs=oss,regulatory=oss

octocrawl-dev API — frequently asked questions

Does octocrawl-dev have an API?

Yes. APIs.io lists 2 APIs from octocrawl-dev, including Octocrawl API and octocrawl-dev API.

Is there an OpenAPI specification for the octocrawl-dev API?

Yes. APIs.io indexes 1 OpenAPI specification for octocrawl-dev, each linked for download from this page.

How do I authenticate with the octocrawl-dev API?

octocrawl-dev documents its authentication (1 scheme). The details are profiled on APIs.io as Octocrawl Dev Authentication.

Does octocrawl-dev have an MCP server?

Yes. APIs.io lists 1 MCP server for octocrawl-dev: Remote MCP server at mcp.octocrawl.dev.

Is the octocrawl-dev API ready for AI agents?

octocrawl-dev has a Kin Score of 57.5 out of 100 (strong) and an Agent Readiness score of 47.4 (agent-native), scored by API Evangelist on 2026-10-31.

Work with this as data

Every provider here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for providers

9 MCP tools reach this

find_providersBrowse and filter every provider in the catalog.
get_provider_artifactsEvery artifact this provider publishes, grouped by type.
get_provider_operationsEvery operation across all of their OpenAPIs — one call instead of parsing every spec.
get_provider_toolsEvery MCP tool they ship, with the operation each wraps.
get_provider_evidenceHow each part of their score was established. Free — the basis for a claim should not sit behind it.
get_provider_ratingPRO — composite, band, trend and facet scores.
apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
resolveTurn a domain, URL or GitHub org into the provider it belongs to.
find_cohortsEvery scored population of providers in the catalog.

All 92 tools →

Call it yourself

curl for this page

This provider

curl "https://apis.io/api/v1/providers/octocrawl-dev"

All providers

curl "https://apis.io/api/v1/providers?limit=25"

Every operation they expose

curl "https://apis.io/api/v1/providers/octocrawl-dev/operations?limit=25"

How their score was established

curl "https://apis.io/api/v1/providers/octocrawl-dev/evidence"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.