Mohammad Movahedi Crawlers API
The Crawlers API from Mohammad Movahedi — 3 operation(s) for crawlers.
The Crawlers API from Mohammad Movahedi — 3 operation(s) for crawlers.
Every API here is available over the APIs.io API and to AI agents over MCP.
One button, every client — Claude, Cursor, VS Code and the rest.
https://apis.io/mcp
find_apisBrowse and filter every API in the catalog.get_api_artifactsOne API's artifacts, grouped by type.get_openapiThe primary OpenAPI for this API.find_similar_apisAPIs that look like this one.apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.resolveTurn a domain, URL or GitHub org into the provider it belongs to.find_cohortsEvery scored population of providers in the catalog.curl "https://apis.io/api/v1/apis/movahedi-ca:movahedi-ca-crawlers-api"
curl "https://apis.io/api/v1/apis?limit=25"
Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.
Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. By signing up you are subscribing to the APIs.io and API Evangelist newsletters — you can unsubscribe from any issue. See our Privacy Policy and Terms.
A second provider on the same verified email joins the account you already have.
openapi: 3.2.0
info:
title: movahedi.ca Public Crawlers API
version: 1.0.0
description: 'Free read-only API: Canadian privacy enforcement actions, a privacy glossary, Quebec Law 25 compliance guidance, a cookie and tracker pattern knowledge base, and the AI Crawler Registry. Glossary definitions support English and French via lang=en or lang=fr; other endpoint language coverage is documented per response. Anonymous quota 2,000 calls/day/IP; free API keys raise it to 10,000/day.'
contact:
url: https://movahedi.ca/developers
license:
name: Terms
url: https://movahedi.ca/api/v1/terms
servers:
- url: https://movahedi.ca/api/v1
tags:
- name: Crawlers
paths:
/crawlers:
get:
summary: List AI crawler registry records
parameters:
- name: q
in: query
schema:
type: string
maxLength: 128
- name: operator
in: query
schema:
type: string
maxLength: 60
- name: purpose
in: query
schema:
type: string
enum:
- training
- search-index
- user-triggered
- search-and-training
- other
- name: status
in: query
schema:
type: string
enum:
- active
- retired
- renamed
- name: limit
in: query
schema:
type: integer
minimum: 1
maximum: 200
default: 50
- name: offset
in: query
schema:
type: integer
minimum: 0
default: 0
responses:
'200':
description: OK
headers:
X-RateLimit-Limit:
schema:
type: integer
X-RateLimit-Remaining:
schema:
type: integer
X-RateLimit-Reset:
schema:
type: integer
content:
application/json:
schema:
type: object
required:
- lang
- data
- meta
properties:
lang:
type: string
enum:
- en
- fr
description: 'Language of the payload: the requested language, or the resolved entry language for single-term lookups.'
data:
type: object
required:
- results
- total_count
- has_more
- next_offset
properties:
results:
type: array
items:
type: object
required:
- token
- operator
- purpose
- honors_robots_txt
- vendor_doc_url
- first_documented
- status
- verified_at
properties:
token:
type: string
operator:
type: string
purpose:
type: string
enum:
- training
- search-index
- user-triggered
- search-and-training
- other
honors_robots_txt:
type: string
enum:
- 'yes'
- 'no'
- disputed
vendor_doc_url:
type: string
format: uri
ip_ranges_url:
type:
- string
- 'null'
format: uri
first_documented:
type: string
status:
type: string
enum:
- active
- retired
- renamed
successor:
type:
- string
- 'null'
verified_at:
type: string
format: date
notes:
type: string
source_note:
type: string
dataset_version:
type: string
title: Crawler
total_count:
type: integer
has_more:
type: boolean
next_offset:
type:
- integer
- 'null'
dataset_version:
type: string
dataset_updated:
type: string
format: date
meta:
type: object
required:
- version
- corpus_version
properties:
version:
type: string
description: API data bundle version, pinned to the site corpus version.
corpus_version:
type: string
description: movahedi.ca corpus version every tool output pins.
'400':
description: Invalid parameter
content:
application/json:
schema:
type: object
required:
- error
properties:
error:
type: object
required:
- code
- message
- docs
properties:
code:
type: string
message:
type: string
docs:
type: string
format: uri
key_url:
type: string
format: uri
tags:
- Crawlers
operationId: getCrawlers
x-operation-id-source: derived
/crawlers/meta:
get:
summary: 'AI crawler registry metadata: version, counts, coverage, citation'
responses:
'200':
description: OK
headers:
X-RateLimit-Limit:
schema:
type: integer
X-RateLimit-Remaining:
schema:
type: integer
X-RateLimit-Reset:
schema:
type: integer
content:
application/json:
schema:
type: object
required:
- lang
- data
- meta
properties:
lang:
type: string
enum:
- en
- fr
description: 'Language of the payload: the requested language, or the resolved entry language for single-term lookups.'
data:
type: object
required:
- dataset
- dataset_version
- record_count
properties:
dataset:
type: string
dataset_version:
type: string
dataset_updated:
type: string
format: date
record_count:
type: integer
operator_count:
type: integer
operators:
type: array
items:
type: string
purposes:
type: array
items:
type: string
by_purpose:
type: object
additionalProperties:
type: integer
statuses:
type: array
items:
type: string
last_verified:
type: string
format: date
citation:
type: string
title: CrawlerRegistryMeta
meta:
type: object
required:
- version
- corpus_version
properties:
version:
type: string
description: API data bundle version, pinned to the site corpus version.
corpus_version:
type: string
description: movahedi.ca corpus version every tool output pins.
tags:
- Crawlers
operationId: getCrawlersMeta
x-operation-id-source: derived
/crawlers/{token}:
get:
summary: One AI crawler registry record by user-agent token (case-insensitive)
parameters:
- name: token
in: path
required: true
schema:
type: string
maxLength: 80
responses:
'200':
description: OK
headers:
X-RateLimit-Limit:
schema:
type: integer
X-RateLimit-Remaining:
schema:
type: integer
X-RateLimit-Reset:
schema:
type: integer
content:
application/json:
schema:
type: object
required:
- lang
- data
- meta
properties:
lang:
type: string
enum:
- en
- fr
description: 'Language of the payload: the requested language, or the resolved entry language for single-term lookups.'
data:
type: object
required:
- token
- operator
- purpose
- honors_robots_txt
- vendor_doc_url
- first_documented
- status
- verified_at
properties:
token:
type: string
operator:
type: string
purpose:
type: string
enum:
- training
- search-index
- user-triggered
- search-and-training
- other
honors_robots_txt:
type: string
enum:
- 'yes'
- 'no'
- disputed
vendor_doc_url:
type: string
format: uri
ip_ranges_url:
type:
- string
- 'null'
format: uri
first_documented:
type: string
status:
type: string
enum:
- active
- retired
- renamed
successor:
type:
- string
- 'null'
verified_at:
type: string
format: date
notes:
type: string
source_note:
type: string
dataset_version:
type: string
title: Crawler
meta:
type: object
required:
- version
- corpus_version
properties:
version:
type: string
description: API data bundle version, pinned to the site corpus version.
corpus_version:
type: string
description: movahedi.ca corpus version every tool output pins.
'404':
description: Not found
content:
application/json:
schema:
type: object
required:
- error
properties:
error:
type: object
required:
- code
- message
- docs
properties:
code:
type: string
message:
type: string
docs:
type: string
format: uri
key_url:
type: string
format: uri
tags:
- Crawlers
operationId: getCrawlersByToken
x-operation-id-source: derived