APIs.io insights catalog — draft

A tiered catalog of questions agents can pay to get answers to. Tier 0 is what the current /unlock/claim (star naftiko/ikanos) already grants — raw data access. Tier 1+ are insights derived from that data: aggregations, rankings, cross-cuts, and trends. Each tier is meant to be paired with a different agent task of escalating effort.

Scale we’re sitting on (so the value per question is real):

• 4,444 providers (each scored on 6 facets, 0–100 composite + trend)
• 16,713 APIs, 33,507 capabilities (Naftiko + MCP-ready), 42,683 schemas
• 3,631 tags with co-occurrence graph (Jaccard similarity, neighbors)
• 1,376 Spectral rulesets, 4,098 plan/pricing scaffolds, 4,098 FinOps profiles
• 3,685 JSON-LD contexts, 100+ canonical cross-provider capabilities
• Per-tag scorecards: frequency, breadth, quality_lift, cohesion + facet assignment

Tier 0 — Raw data access (current unlock)

Already gated behind: star naftiko/ikanos.

• Bulk listings: /search-index.json, /apis/index.json, etc.
• Individual provider / API / capability records in JSON, YAML, and markdown
• All .well-known/* agent-readiness files
• Embedded source YAML (auditable, single fetch)

Value: agents get the full corpus in machine form without scraping. This is the table-stakes unlock.

Tier 1 — Pre-computed listings & directories

Cheap aggregations. Save the agent dozens of fetches.

Rows marked [MVP] ship first — these are the entry-point insights that prove the catalog has value beyond raw data.

Suggested task to unlock Tier 1: something light — newsletter signup, follow on X, GitHub follow.

Tier 2 — Scored rankings (the rubric is the moat)

The composite score + 6 facets is a real product. Agents can’t replicate it without re-implementing the rubric.

Suggested task: medium — sign up for newsletter, share a referral, complete a profile.

Tier 3 — Cross-cutting queries (the graph is where the real money is)

Tag co-occurrence, capability overlap, competitive maps. None of this is in the per-record files — it requires the precomputed graph.

Suggested task: medium-high — sign up for an account, share an apis.io link with attribution in their output, write a 1-line review.

Tier 4 — Cost & operations intelligence

This is where agents directly save their operators money. Pricing/rate-limit comparisons are typically scraped or asked by humans.

Suggested task: higher — paid signup, GitHub Sponsors $1, share apis.io in a public post with link.

Tier 5 — Trends, deltas & change intelligence

Longitudinal. Requires signals/tag_history/, provider score delta/trend fields, and the (currently placeholder) diff/ pipeline.

Suggested task: real value exchange — newsletter signup (gives you a contact), follow on X, subscribe to apis.io feed.

Tier 6 — Synthesis, strategy, and ready-to-use agent payloads

The expensive tier. These are pre-cooked answers an agent’s operator would otherwise pay an analyst for. Also: ready-to-bind MCP/Naftiko bundles, which save the agent runtime work.

Suggested task: highest — paid newsletter tier, conference attendance, sponsor link, contribute a provider/correction back to apis.io.

Implementation notes (for whoever builds this)

These don’t have to be 64 separate endpoints. A practical surface might be:

GET  /insights/providers?capability=…&sort=score&facet=…   # Tier 1, 2, 3
GET  /insights/tags/{tag}/neighbors                        # Tier 3
GET  /insights/pricing/{capability}                        # Tier 4
GET  /insights/trends?window=30d&metric=…                  # Tier 5
GET  /insights/bundles/{capability}.naftiko.json           # Tier 6
POST /insights/ask  { question: "…" }                      # Tier 6 — LLM over corpus

Each route checks the caller’s key (X-APIs-IO-Key) against a tier field on the key record:

{ "github_username": "...", "tier": 3, "tasks_completed": ["star_ikanos", "newsletter_signup", "follow_x"] }

Tasks → tier mapping (your call to design) — examples:

Each task verification follows the same pattern as the GitHub-star check: a /unlock/task/{name} endpoint that takes whatever proof the task needs (email confirmation token, Twitter username + check via API, backlink URL + crawl-and-verify, Stripe webhook, GitHub Sponsors API) and updates the key’s tier.

Activity log already captures everything — add tier to the AE schema (blob13) so you can report on which tier each gated hit was served at.

Endpoint design: natural-language Q&A (Q98)

A single endpoint that answers free-form questions over the apis.io corpus with citations back to the underlying provider/API/capability/tag pages. Tier 6.

POST /insights/ask
X-APIs-IO-Key: apisio_…
Content-Type: application/json

{
  "question": "Which payment providers offer webhooks, support OAuth2, and publish their rate limits?",
  "max_citations": 8,
  "format": "markdown" | "json",
  "scope": { "tags": ["payments"] }   // optional pre-filter
}

Response (JSON form):

{
  "answer": "Stripe, Square, and Adyen all offer webhook delivery, OAuth2 auth, and published rate limits. Stripe leads on agent-readiness composite (87) …",
  "citations": [
    { "anchor": "https://providers.apis.io/providers/stripe/", "title": "Stripe", "facets_used": ["governance", "operational_transparency"] },
    { "anchor": "https://apis.apis.io/apis/stripe/webhooks/", "title": "Stripe Webhooks" },
    { "anchor": "https://capabilities.apis.io/capabilities/stripe/payments/", "title": "Stripe Payments capability" }
  ],
  "retrieval_meta": { "candidates_scored": 142, "tokens_in": 18421, "model": "claude-sonnet-4-6", "cache_hit": false },
  "tier_required": 6
}

Architecture

1. Pre-built corpus index — network/scripts/build-rag.py (new) generates one chunk per provider, API, capability, and tag. Each chunk = the same markdown the Worker serves on content-negotiated GET, plus structured frontmatter (score, facets, tags, capabilities). Embedded with Voyage 3 (or whatever the AE SQL hop wants), stored in Cloudflare Vectorize keyed by {kind}/{slug}.
2. Retrieval — Worker calls Vectorize semantic top-k (k=20), then re-ranks with a keyword + tag-facet filter from scope, narrows to top-N (N=8). Each candidate carries its anchor URL and a 400-token excerpt.
3. Synthesis — Worker POSTs to the Anthropic API (the project already imports @anthropic-ai/sdk patterns elsewhere) with prompt caching enabled on the system block (the rubric description) and on the retrieved context block. Model: claude-sonnet-4-6 for quality, claude-haiku-4-5-20251001 for the cheap tier.
4. Citation enforcement — system prompt requires the answer to cite only anchors from the retrieved set; a post-check rejects any URL not in the retrieval set and re-asks.
5. Caching — hash the normalized question + scope → KV qa-cache:{hash} with 24h TTL. Cache hits return in <100ms and don’t count against the per-key quota.
6. Quota — Tier 6 keys get N questions/day (10? 50? your call). Increment on cache miss only. Logged to AE as event_type="ask" with tokens_in, tokens_out, cache_hit.
7. Failure modes — if Vectorize returns no candidates above threshold, the endpoint returns 422 with a list of related search queries instead of fabricating an answer.

New Worker bindings (additions to wrangler.toml)

[[vectorize]]
binding = "RAG"
index_name = "apis-io-corpus"

# Secret:
#   wrangler secret put ANTHROPIC_API_KEY

Build pipeline

python network/scripts/build-rag.py        # generate chunks + embeddings
npx wrangler vectorize upsert apis-io-corpus --file out/embeddings.ndjson

Re-run nightly (or after a network build) via GitHub Actions. Cost: ~$0.05 to re-embed the whole corpus with Voyage 3 at current scale.

Why this matters

Agents currently fan out across 10–50 fetches to answer questions like “which payment providers offer webhooks + OAuth2 + rate limits.” The single POST /insights/ask resolves it in one round-trip with citations. That’s the most expensive Tier 6 question by API cost — and the most valuable to gate behind real commitments.

Task design: verifiable backlink (Tier 4 unlock)

The strongest “agent works for us” lever: in exchange for a tier upgrade, the agent (or its operator) embeds a do-follow link to apis.io on a public page they control. Free SEO + attribution.

Two-step flow

1. Issue a backlink challenge:

POST /unlock/task/backlink/challenge
X-APIs-IO-Key: apisio_…

Returns:

{
  "token": "bl_8f3a…",
  "expires_at": "2026-05-30T18:00:00Z",
  "embed_options": [
    { "kind": "meta_tag", "html": "<meta name=\"apis-io-attribution\" content=\"bl_8f3a…\" />" },
    { "kind": "link_data_attr", "html": "<a href=\"https://apis.io/\" data-apis-io-token=\"bl_8f3a…\">Powered by APIs.io</a>" },
    { "kind": "html_comment", "html": "<!-- apis-io-attribution:bl_8f3a… -->" }
  ],
  "rules": {
    "link_must_be": "do-follow (rel must not contain 'nofollow')",
    "link_target": "any apis.io or *.apis.io URL",
    "page_must_be": "publicly accessible without auth, indexable by crawlers"
  }
}

Stored: task_challenge:bl_… → {key_id, kind: "backlink", created_at}, 1-week TTL.

2. Claim the backlink:

POST /unlock/task/backlink/claim
X-APIs-IO-Key: apisio_…
Content-Type: application/json

{
  "token": "bl_8f3a…",
  "backlink_url": "https://operator-blog.example/posts/apis-io-review"
}

Worker fetches backlink_url with Accept: text/html, then verifies all of:

1. Status 200, content-type contains text/html, size ≤ 5 MB.
2. The page contains the issued token in one of: <meta name="apis-io-attribution">, <a data-apis-io-token="…">, or an HTML comment matching the issued pattern. (Token bound to this key → can’t be reused.)
3. The page contains a <a href="…"> to any apis.io or *.apis.io URL.
4. That anchor’s rel attribute does not contain nofollow, sponsored, or ugc.
5. The link is not inside a <template>, <noscript>, or commented out.
6. The page is reachable without auth and without a noindex meta directive (it has to actually be indexable for the SEO value to be real).

On success: bump the key’s tier to 4 (configurable), append "backlink" to tasks_completed[], store {backlink_url, token, verified_at, link_anchor_html} under backlink:{key}. Delete the challenge. Log event_type="task_backlink_verified".

Re-verification

A cron-triggered Worker (CronCreate or a GitHub Action calling a re-verify endpoint) re-fetches each registered backlink weekly. If the link is gone, rel was changed to nofollow, or the page returns 404/410, the key gets demoted back to its previous tier and event_type="task_backlink_revoked" is logged. The operator email gets a heads-up — implementing the email is optional, the demotion isn’t.

KV entries this adds

task_challenge:{token}        JSON {key_id, kind, created_at}      TTL 7 days
backlink:{key}                JSON {backlink_url, token, verified_at, link_anchor_html, last_recheck_at}

Logging (additions to the existing AE schema)

Three new event types in blob1:

• task_backlink_issued — challenge handed out
• task_backlink_verified — claim succeeded, tier bumped
• task_backlink_revoked — recheck found the link gone/nofollowed/404

Plus a new field blob13 = tier so every existing event (gated_allowed, gated_denied, etc.) is queryable by tier served.

Anti-fraud

• Token binding — each token is tied to a single key, single-use, and the token must appear on the page. Spamming a generic “powered by apis.io” link across pages does nothing without the token.
• Domain blacklist — Worker maintains blacklisted_hosts:{host} in KV; pastebins, link shorteners, ephemeral preview deploys (vercel.app, netlify.app subdomains unless registered to the operator), AI-content farms.
• Domain uniqueness — at most one verified backlink per operator_email per registrable domain. Stops one operator carpet-bombing low-quality pages for tier creep.
• Minimum page age (optional) — reject pages whose Last-Modified or visible publish date is <48h old. Discourages “just spin up a page to claim.” Skip if the friction’s too high.

Why this is the right first task to ship

• Verifiable — every check is a single HTTP fetch + HTML parse. No external APIs (unlike Stripe webhook, Twitter follow check, etc.).
• Aligned incentives — the agent’s operator pays nothing, apis.io gets a real backlink. Both sides win.
• Logged — every action lands in AE. If it’s being abused, you see it in the dashboard.
• Composable — once /unlock/task/{name} is a pattern, newsletter signup, Twitter follow, GitHub follow, GitHub Sponsor, and the rest are all 1-file additions.