Every API here is available over the APIs.io API and to AI agents over MCP.
openapi: 3.2.0
info:
title: Writ Agent Crawl API
version: 0.1.0
license:
name: MIT
identifier: MIT
description: 'Loopback HTTP API of the Writ local agent (`writ-agentd`), the daemon embedded in the
Writ desktop app and the self-host build. This document is generated against the Rust
daemon source (`writ-agent/src/local/`), which is the wire truth.
## Transport
- Plain HTTP on `127.0.0.1:8131` (default; `WRIT_PORT` env overrides).
- Optional HTTPS twin on `127.0.0.1:8132`, served with a certificate minted by a
per-install, name-constrained local CA (`~/.writ/tls/ca.pem`). Clients must trust that
CA file (or pin it) to use the TLS port; the payloads are identical on both ports.
- The daemon enforces a DNS-rebind guard: `Host`/`Origin` must be loopback. Talk to
`127.0.0.1`, not `localhost`.
## Errors
Domain errors are JSON `{"error": "", "code": ""}` (see the
`Error` schema). Codes include `bad_request`, `unauthorized`, `forbidden`, `not_found`,
`vault_locked` (HTTP 423), `too_many_requests`, `device_capacity` (HTTP 409),
`captcha_required` (HTTP 402), `internal`.
**Caveat:** some 4xx bodies are `text/plain`, not JSON — axum framework rejections
(e.g. a JSON body that fails to deserialize returns
`Failed to deserialize the JSON body into the target type: missing field ''url''` as
plain text with a 4xx status). Clients must tolerate non-JSON error bodies.
## List envelopes
The wire is deliberately documented as-is (SDKs normalize; this spec does not):
- `{"data": [...], "count": n}` — workflows, personas, persona runs, secrets, files, keys
- `{"data": [...], "count": n, "total": n}` — runs
- **bare JSON array** — monitors, monitor selectors, extractors, automations,
`/v1/changes/recent`
- bespoke objects — `/v1/monitors/{id}/changes`, `/v1/data`, the workflow-data table'
servers:
- url: http://127.0.0.1:8131
description: Plain-HTTP loopback listener (default port; WRIT_PORT overrides).
- url: https://127.0.0.1:8132
description: HTTPS twin of the same API, terminated with a certificate from the per-install local CA at ~/.writ/tls/ca.pem. Trust that CA (or pin it) in your HTTP client.
security:
- bearerAuth: []
tags:
- name: crawl
description: Dragnet whole-site crawl — start/list/poll/cancel; pages aggregate under a synthetic workflow.
paths:
/v1/crawl:
get:
operationId: listCrawls
tags:
- crawl
summary: List crawls
description: 'Newest-first Dragnet crawl list. Does NOT use the `{data, count}` envelope — it returns `{crawls: [CrawlJob…]}` (mirrors the `/v1/data` non-Page shape). Each item is the `to_view` projection (parsed `include_paths`/`exclude_paths` arrays, the `Dragnet` brand, the `data_workflow_id` alias and `is_terminal`).'
parameters:
- name: limit
in: query
description: Page size (clamped to 1..=500; default 50).
schema:
type: integer
format: int64
default: 50
responses:
'200':
description: Crawl list under a `crawls` key.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlList'
'401':
$ref: '#/components/responses/Unauthorized'
post:
operationId: startCrawl
tags:
- crawl
summary: Start a crawl
description: Queues a Dragnet whole-site crawl from a seed `url`, fanning it across a bounded in-process worker pool; extracted pages aggregate under a synthetic per-crawl workflow (read back via `GET /v1/workflows/{data_workflow_id}/data`). An empty `url` is a 400. Returns the queued crawl view.
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlStartRequest'
responses:
'200':
description: The queued crawl view.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlJob'
'400':
$ref: '#/components/responses/BadRequest'
'401':
$ref: '#/components/responses/Unauthorized'
/v1/crawl/{id}:
parameters:
- $ref: '#/components/parameters/CrawlId'
get:
operationId: getCrawl
tags:
- crawl
summary: Get one crawl
description: One crawl view (with live progress counters) or 404.
responses:
'200':
description: The crawl view.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlJob'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
/v1/crawl/{id}/cancel:
post:
operationId: cancelCrawl
tags:
- crawl
summary: Request cancellation of a crawl
description: Signals the crawl to stop. Always returns the refreshed crawl view (never a 409) plus `cancel_requested_now` — `true` iff this call flipped a live crawl into `stopping`, `false` when it was already terminal.
parameters:
- $ref: '#/components/parameters/CrawlId'
responses:
'200':
description: The refreshed crawl view plus `cancel_requested_now`.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlCancelResult'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
/v1/crawl/definitions:
get:
operationId: listCrawlDefinitions
tags:
- crawl
summary: List saved crawls
description: 'Newest-first saved crawls. A saved crawl is a stored crawl configuration with a stable slug: a crawl row is one RUN whose id dies with that run, so a definition is what makes a crawl callable and re-runnable. Envelope is `{definitions: [...]}`, not a page.'
parameters:
- name: limit
in: query
schema:
type: integer
minimum: 1
maximum: 500
default: 50
responses:
'200':
description: The saved crawls.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlDefinitionList'
'401':
$ref: '#/components/responses/Unauthorized'
post:
operationId: saveCrawlDefinition
tags:
- crawl
summary: Save a crawl configuration
description: 'Supply EITHER `config` (a start-crawl body) OR `from_crawl_id`. Prefer `from_crawl_id` when capturing an existing crawl: a crawl''s status view does not echo every knob it ran with (politeness, shard sizing, path filters), so a config rebuilt by a client would silently substitute defaults and save a crawl that behaves differently from the one you pointed at.'
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/SaveCrawlBody'
responses:
'201':
description: The saved crawl.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlDefinition'
'400':
description: Neither `config` nor `from_crawl_id` was supplied.
'401':
$ref: '#/components/responses/Unauthorized'
/v1/crawl/definitions/{ref}:
parameters:
- $ref: '#/components/parameters/CrawlDefinitionRef'
get:
operationId: getCrawlDefinition
tags:
- crawl
summary: Get one saved crawl
responses:
'200':
description: The saved crawl.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlDefinition'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
patch:
operationId: updateCrawlDefinition
tags:
- crawl
summary: Update a saved crawl
description: Sparse update — omitted fields stay untouched.
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/SaveCrawlBody'
responses:
'200':
description: The updated saved crawl.
content:
application/json:
schema:
$ref: '#/components/schemas/CrawlDefinition'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
delete:
operationId: deleteCrawlDefinition
tags:
- crawl
summary: Delete a saved crawl
description: Removes the reusable configuration only. The runs it launched and the data they collected survive — they simply stop pointing at a definition.
responses:
'204':
description: Deleted.
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
/v1/crawl/definitions/{ref}/run:
post:
operationId: runCrawlDefinition
tags:
- crawl
summary: Run a saved crawl (with optional freshness reuse)
description: 'Runs the saved settings. `max_age` is a freshness CONTRACT, not a cache flag: "data collected within this many seconds is acceptable, otherwise go get it again". On a hit the last completed run''s rows are returned inline with `cached: true` and nothing is crawled or metered; `max_age: 0` (or `Cache-Control: no-cache`) always re-crawls.
Only a `completed` run with at least one fetched page is reusable — a crawl that converged having fetched nothing (the shape a fully-blocked host produces) must not pin an empty answer behind a long window.
A cold call answers **202** with a crawl id and `status_url`, because a whole-site crawl outlives an HTTP request. Pass `wait: true` to block instead; an overrun answers **504 carrying the same collectable handle**, never a dead end.'
parameters:
- $ref: '#/components/parameters/CrawlDefinitionRef'
- name: max_age
in: query
description: 'Alternative spelling of the body field (also accepts `Cache-Control: max-age=N`).'
schema:
type: integer
minimum: 0
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/RunSavedCrawlBody'
responses:
'200':
description: A freshness hit — collected data, nothing crawled.
content:
application/json:
schema:
$ref: '#/components/schemas/SavedCrawlRun'
'202':
description: A crawl was dispatched; poll `status_url`.
content:
application/json:
schema:
$ref: '#/components/schemas/SavedCrawlRun'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
'504':
description: '`wait: true` outlived its budget. The crawl is STILL RUNNING; the body carries `crawl_id` and `status_url` so it stays collectable. Do not retry — that would start a second crawl.'
/v1/crawl/definitions/{ref}/data:
get:
operationId: getCrawlDefinitionData
tags:
- crawl
summary: Read what a saved crawl already collected
description: The rows from the saved crawl's most recent completed run. A pure read at any age — never dispatches a crawl. Use the run endpoint with `max_age` when you need a recency guarantee instead.
parameters:
- $ref: '#/components/parameters/CrawlDefinitionRef'
- name: limit
in: query
schema:
type: integer
minimum: 1
maximum: 500
default: 50
responses:
'200':
description: The collected rows (or nulls when it has never completed).
content:
application/json:
schema:
$ref: '#/components/schemas/SavedCrawlData'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
components:
schemas:
CrawlCancelResult:
description: The refreshed crawl view plus `cancel_requested_now` — `true` iff this cancel call flipped a live crawl into `stopping`, `false` when it was already terminal.
allOf:
- $ref: '#/components/schemas/CrawlJob'
- type: object
required:
- cancel_requested_now
properties:
cancel_requested_now:
type: boolean
description: Whether this call transitioned the crawl to `stopping`.
CrawlDefinition:
type: object
description: A SAVED crawl — a stored crawl configuration with a stable slug. A CrawlJob is one RUN and its id dies with that run, so this is what makes a crawl callable and re-runnable, and what gives `max_age` a history to answer from.
required:
- id
- slug
- name
- seed_url
properties:
id:
type: integer
format: int64
slug:
type: string
description: URL-safe stable ref used by the callable endpoint.
name:
type: string
description:
type: string
nullable: true
seed_url:
type: string
config:
type: object
additionalProperties: true
description: The saved start-crawl body. Send it back verbatim to edit.
default_max_age_seconds:
type: integer
nullable: true
description: Freshness used when a caller omits `max_age`; null means always re-crawl.
created_at:
type: string
nullable: true
updated_at:
type: string
nullable: true
last_run_at:
type: string
nullable: true
run_url:
type: string
data_url:
type: string
Error:
type: object
description: 'The daemon''s domain error body. NOTE: some 4xx responses (axum framework rejections) are `text/plain` instead of this JSON shape. Certain codes attach extra fields (e.g. `device_capacity` adds `would_use_units_per_min` / `budget_units_per_min`; `captcha_required` adds `captcha_type` / `url`).'
required:
- error
- code
properties:
error:
type: string
description: 'Human-readable message (e.g. `not found: workflow 999999`).'
code:
type: string
description: 'Stable machine code: `bad_request`, `unauthorized`, `forbidden`, `not_found`, `vault_locked`, `too_many_requests`, `captcha_required`, `device_capacity`, `cloud_unreachable`, `db_error`, `io_error`, `json_error`, `internal`, …'
additionalProperties: true
CrawlDefinitionList:
type: object
description: The saved-crawl list payload (non-Page — a `definitions` array).
required:
- definitions
properties:
definitions:
type: array
items:
$ref: '#/components/schemas/CrawlDefinition'
RunSavedCrawlBody:
type: object
description: DELIVERY controls only — the crawl settings are the saved ones.
properties:
max_age:
type: integer
minimum: 0
description: Reuse the last completed crawl if it finished within this many seconds. 0 always re-crawls.
wait:
type: boolean
default: false
description: Block until the crawl converges. Default false, because a whole-site crawl is slow.
timeout:
type: integer
minimum: 5
maximum: 300
default: 120
limit:
type: integer
minimum: 1
maximum: 500
default: 50
CrawlJob:
type: object
description: 'A Dragnet crawl job (`crawl_jobs` row), in the `to_view` projection: the JSON-TEXT scope columns are parsed into arrays, and the `Dragnet` brand, the `data_workflow_id` alias (of `workflow_id`) and the derived `is_terminal` are added. SQLite booleans arrive as `0`/`1` integers — kept typed as ints, not coerced.'
required:
- id
- name
- seed_url
- include_paths
- exclude_paths
- max_depth
- same_domain
- allow_subdomains
- extract_mode
- respect_robots
- delay_ms
- max_concurrent
- page_budget
- status
- pages_discovered
- pages_done
- pages_failed
- pages_skipped
- workers_active
- current_depth
- cancel_requested
- brand
- is_terminal
- created_at
properties:
id:
type: integer
format: int64
name:
type: string
seed_url:
type: string
include_paths:
type: array
items:
type: string
description: Parsed path-regex allowlist.
exclude_paths:
type: array
items:
type: string
description: Parsed path-regex denylist.
max_depth:
type: integer
format: int64
same_domain:
type: integer
description: 0/1 integer flag.
allow_subdomains:
type: integer
description: 0/1 integer flag.
extract_mode:
type: string
description: '`markdown` | `schema`.'
extract_schema:
type:
- object
- 'null'
description: The extraction schema (present when `extract_mode` is `schema`).
persona_id:
type:
- integer
- 'null'
format: int64
respect_robots:
type: integer
description: 0/1 integer flag.
delay_ms:
type: integer
format: int64
max_concurrent:
type: integer
format: int64
page_budget:
type: integer
format: int64
workflow_id:
type:
- integer
- 'null'
format: int64
description: The synthetic per-crawl workflow the extracted pages aggregate under.
data_workflow_id:
type:
- integer
- 'null'
format: int64
description: Alias of `workflow_id` — read the crawl's data via this workflow id.
concierge_session_id:
type:
- integer
- 'null'
format: int64
status:
type: string
description: 'Open string enum: `queued` | `mapping` | `crawling` | `stopping` | `completed` | `failed` | `cancelled` (the last three are terminal).'
enum:
- queued
- mapping
- crawling
- stopping
- completed
- failed
- cancelled
pages_discovered:
type: integer
format: int64
pages_done:
type: integer
format: int64
pages_failed:
type: integer
format: int64
pages_skipped:
type: integer
format: int64
workers_active:
type: integer
format: int64
current_depth:
type: integer
format: int64
error:
type:
- string
- 'null'
cancel_requested:
type: integer
description: 0/1 integer flag.
brand:
type: string
description: Always `Dragnet`.
is_terminal:
type: boolean
description: Whether `status` is one of the terminal states.
created_at:
type: string
updated_at:
type:
- string
- 'null'
started_at:
type:
- string
- 'null'
completed_at:
type:
- string
- 'null'
SavedCrawlData:
type: object
description: A pure read of a saved crawl's latest completed run; never crawls.
required:
- definition
properties:
definition:
$ref: '#/components/schemas/CrawlDefinition'
crawl:
allOf:
- $ref: '#/components/schemas/CrawlJob'
nullable: true
age_seconds:
type: number
nullable: true
data_url:
type: string
nullable: true
data:
allOf:
- $ref: '#/components/schemas/CrawlDataTable'
nullable: true
CrawlDataTable:
type: object
description: A page of a crawl's collected rows, in the Workflow Data API's shape.
properties:
columns:
type: array
items:
type: string
rows:
type: array
items:
type: object
additionalProperties: true
total:
type: integer
truncated:
type: boolean
SaveCrawlBody:
type: object
description: Supply exactly one of `config` / `from_crawl_id`.
properties:
name:
type: string
maxLength: 200
slug:
type: string
maxLength: 120
description:
type: string
default_max_age_seconds:
type: integer
minimum: 0
nullable: true
config:
$ref: '#/components/schemas/CrawlStartRequest'
from_crawl_id:
type: integer
format: int64
description: 'Capture the settings an existing crawl ran with. Preferred over rebuilding `config`: a crawl''s status view does not echo every knob, so a client-rebuilt config silently substitutes defaults.'
CrawlStartRequest:
type: object
description: Start a Dragnet whole-site crawl. Only `url` is required; every other field falls to the column default. `include_paths`/`exclude_paths` are path-regex allow/deny lists.
required:
- url
properties:
url:
type: string
description: The seed URL to crawl (empty → 400).
name:
type: string
description: 'Display name (defaults to a `Dragnet: <host>` label).'
extract_mode:
type: string
enum:
- markdown
- schema
default: markdown
description: Whether each page is captured as markdown or against `extract_schema`.
extract_schema:
type: object
description: The extraction schema, used when `extract_mode` is `schema`.
persona_id:
type: integer
format: int64
description: Optional persona to crawl as (login identity).
include_paths:
type: array
items:
type: string
description: Path-regex allowlist — only matching paths are crawled.
exclude_paths:
type: array
items:
type: string
description: Path-regex denylist — matching paths are skipped.
max_depth:
type: integer
format: int64
default: 3
description: Maximum link depth from the seed.
page_budget:
type: integer
format: int64
default: 500
description: Hard cap on pages fetched for the whole crawl.
max_concurrent:
type: integer
format: int64
default: 4
description: Worker-pool size (concurrent fetches).
delay_ms:
type: integer
format: int64
default: 250
description: Politeness delay between fetches, in milliseconds.
respect_robots:
type: boolean
default: true
description: Honour the target's `robots.txt`.
same_domain:
type: boolean
default: true
description: Restrict the crawl to the seed's domain.
allow_subdomains:
type: boolean
default: true
description: Allow sibling subdomains of the seed domain (when `same_domain`).
content:
type: object
description: 'Content-selection spec applied to every crawled page: {preset, include_comments, exclude_selectors, include_selectors, keep}. Omit for default extraction. Forwarded to the cloud when linked; honored locally otherwise.'
CacheStamp:
type: object
description: Freshness provenance. Stamped into the BODY, not only into headers, because an MCP tool or SDK client receives a payload and would never see a header.
required:
- hit
properties:
hit:
type: boolean
description: True when the answer reused already-collected data (nothing was crawled).
age_seconds:
type: integer
source_crawl_id:
type: integer
format: int64
SavedCrawlRun:
type: object
description: 'Two shapes behind one call. On a freshness HIT (`cached: true`) `data` is inline and nothing was crawled. On a MISS a crawl was dispatched and `data` is absent.'
required:
- cached
- definition
- crawl
properties:
cached:
type: boolean
_cache:
$ref: '#/components/schemas/CacheStamp'
definition:
$ref: '#/components/schemas/CrawlDefinition'
crawl:
$ref: '#/components/schemas/CrawlJob'
status_url:
type: string
nullable: true
data_url:
type: string
nullable: true
data:
allOf:
- $ref: '#/components/schemas/CrawlDataTable'
nullable: true
CrawlList:
type: object
description: The Dragnet crawl list payload (non-Page — a `crawls` array).
required:
- crawls
properties:
crawls:
type: array
items:
$ref: '#/components/schemas/CrawlJob'
parameters:
CrawlDefinitionRef:
name: ref
in: path
required: true
description: A saved crawl's numeric id or its slug.
schema:
type: string
CrawlId:
name: id
in: path
required: true
description: Crawl job row id.
schema:
type: integer
format: int64
responses:
NotFound:
description: The addressed resource does not exist.
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
Unauthorized:
description: Missing/invalid bearer token, or the Origin/Host guard rejected the request.
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
BadRequest:
description: 'Invalid input. Domain validation errors use the JSON Error shape; axum body-deserialize rejections are `text/plain` with a bare message (e.g. `Failed to deserialize the JSON body into the target type: missing field ''url''`).'
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
text/plain:
schema:
type: string
securitySchemes:
bearerAuth:
type: http
scheme: bearer
description: "Every route (including `/v1/agent` and `/v1/health`) requires\n`Authorization: Bearer <token>`. Token families:\n\n- `wlt_…` — the runtime token from `runtime.json` (full access, used by the desktop\n UI and SDK discovery).\n- `wlk_…` — a scoped API key minted via `POST /v1/keys` (CSV scopes `read|run|admin`;\n `/v1/keys` itself requires the full-access token).\n- `wlo_…` — an OAuth 2.1 access token from the daemon's local authorization server\n (`run` scope).\n\nTreat the token as an opaque string. The daemon additionally enforces a loopback\nOrigin/Host DNS-rebind guard.\n"