Columbia University Agentic Access

generated: '2026-08-19'
method: probed
source: >-
  Live probes of Columbia University hosts on 2026-08-19 with and without a browser User-Agent,
  covering the well-known discovery paths, the agent-facing conventions and every institution-
  operated surface found.
provider: Columbia University
providerId: columbia
description: >-
  What an autonomous agent can actually do with Columbia University. The answer is narrow and it
  is unusually well-defined: exactly one API is agent-callable end to end, and the rest of the
  institution's estate is actively defended against automated clients by two different
  challenge products.
callable_without_human:
- surface: columbia:library-hours
  baseURL: https://hours.library.columbia.edu/api/v1
  x-operator: institution
  verdict: fully_callable
  detail: >-
    No credential, no registration, no click-through, no rate limit encountered. Both operations
    return application/json with Access-Control-Allow-Origin: * — Columbia's own source sets the
    CORS header deliberately on exactly these two actions — so an agent can call it from a
    browser context as well as a server. Correct 400 and 404 status codes on the error paths.
    The one trap is the missing-parameter path, which returns HTTP 200 with an HTML body; an
    agent that trusts the status code will store a web page as data.
- surface: columbia:identity
  baseURL: https://shibboleth.columbia.edu/idp/shibboleth
  x-operator: institution
  verdict: readable
  detail: >-
    SAML 2.0 metadata is served anonymously as application/xml and is machine-parseable. It is
    readable, not actionable — an agent can learn how Columbia federates identity but cannot
    obtain an identity.
- surface: columbia:clio-opendata
  baseURL: https://lito.cul.columbia.edu/extracts/ColumbiaLibraryCatalog/full/
  x-operator: institution
  verdict: harvestable
  detail: >-
    108 gzipped MARCXML files plus a deletes list, served from an open Apache directory index
    under CC0 1.0. An agent can enumerate and download the entire Columbia catalogue with no
    credential. There is no manifest, no checksum file and no change feed, so an agent must diff
    the directory listing to detect updates.
blocked_to_agents:
- surface: opendataservice.columbia.edu, www.cuit.columbia.edu, provost.columbia.edu, ai.columbia.edu, vergil.registrar.columbia.edu
  x-operator: institution
  product: Cloudflare managed challenge
  status: 403
  detail: >-
    Columbia's central Drupal web estate returns a "Just a moment..." interstitial with HTTP 403
    to any non-browser client, including with a full browser User-Agent and navigation headers.
    The Open Data Service — Columbia's own developer-facing service — is inside this perimeter,
    so an agent cannot read the documentation for the feeds it is being invited to consume.
- surface: clio.columbia.edu, academiccommons.columbia.edu, geodata.library.columbia.edu
  x-operator: institution
  product: Anubis proof-of-work challenge
  status: 200
  detail: >-
    Columbia University Libraries has deployed Anubis in front of its entire Blacklight discovery
    estate. This is the more consequential of the two because it returns HTTP 200 with the
    challenge body, so an agent that checks only the status code records a success and ingests a
    bot-check page. It is the direct reason the OAI-PMH endpoint could not be verified, and it
    sits in tension with the Libraries' own CC0 bulk release of the same catalogue.
- surface: iridl.ldeo.columbia.edu
  x-operator: institution
  product: login redirect
  detail: >-
    The IRI/LDEO Climate Data Library front page is anonymous but every data path — /SOURCES/,
    OPeNDAP, OGC WMS GetCapabilities — 302s to /auth/login. Presence without access.
discovery:
  llms_txt:
    status: absent
    evidence:
    - url: https://www.columbia.edu/llms.txt
      status: 404
    - url: https://library.columbia.edu/llms.txt
      status: 404
  ai_txt:
    status: absent
    evidence:
    - url: https://www.columbia.edu/ai.txt
      status: 404
  well_known_api_catalog:
    status: absent
    evidence:
    - url: https://hours.library.columbia.edu/.well-known/api-catalog
      status: 404
  security_txt:
    status: absent
    evidence:
    - url: https://www.columbia.edu/.well-known/security.txt
      status: 404
    - url: https://library.columbia.edu/.well-known/security.txt
      status: 404
  robots_txt:
    status: partial
    detail: >-
      hours.library.columbia.edu serves a robots.txt, but it is the stock Rails template
      containing only the commented documentation link and no directives. www.columbia.edu and
      library.columbia.edu both return 404 for robots.txt. So Columbia challenges agents at the
      edge while publishing no machine-readable statement anywhere about what agents may do.
    evidence:
    - url: https://hours.library.columbia.edu/robots.txt
      status: 200
    - url: https://www.columbia.edu/robots.txt
      status: 404
  openapi_published: false
  mcp_server: false
  agent_card: false
  agent_skills: false
summary:
  fully_callable_surfaces: 1
  readable_surfaces: 2
  challenge_blocked_hosts: 8
  agent_discovery_files: 0
notes: >-
  The shape here is worth stating precisely because it is becoming the common institutional
  pattern: the one thing an agent can use is a small campus-life API that nobody governs, and the
  scholarly resources the institution most wants read are the ones sitting behind an anti-crawler
  challenge. Columbia has taken a deliberate position on automated access at the edge and has
  published no statement of that position anywhere a machine can read it — no llms.txt, no
  ai.txt, no meaningful robots.txt, no terms naming a sanctioned bulk route.