CoreStory pre_ingestion API
The pre_ingestion API from CoreStory — 10 operation(s) for pre_ingestion.
OpenAPI Specification
openapi: 3.1.0
info:
title: Crowdbotics API Documentation admin pre_ingestion API
description: "# CoreStory API Overview\n\nThe CoreStory API provides programmatic access to structured insights derived from software source code. It is designed for engineering, architecture, and product teams who need to extract business logic, technical specifications, architectural relationships, and system-level context from complex or legacy codebases.\n\nBy interfacing directly with CoreStory API, users can:\n\n- Generate full Product Requirements Documents (PRDs) from a codebase\n- Extract detailed Technical Specifications for modernization or migration\n- Query a codebase using natural language to surface implementation details, logic paths, dependencies, and architectural insights\n- Construct and visualize knowledge graphs of systems, components, and relationships\n- Ingest, store, and retrieve content from a vector store for AI-powered retrieval-augmented generation (RAG) workflows\n\nCoreStory is especially valuable in contexts where:\n\n- Codebases are large, unstructured, or under-documented\n- Product and engineering teams are onboarding to unfamiliar systems\n- Technical stakeholders require visibility into system design or feature coverage\n- Modernization, replatforming, or M&A due diligence is underway\n\nThe API can be integrated into existing CI/CD pipelines, internal developer portals, architecture review boards, product planning processes, or documentation automation workflows.\n\nAll endpoints support secure authentication and are designed for high-availability, asynchronous workloads. Output formats include JSON, Markdown, and PDF, with flexible formatting options for downstream consumption.\n\nFor step-by-step usage instructions and workflows, refer to the [CoreStory API Quick Start Guide](#section/CoreStory-API-Quick-Start-Guide).\n\n## Features\n\n* **Document Generation**\n * Generate complete Product Requirements Documents (PRD)\n * Generate Technical Specifications\n * Generate individual document sections (executive overview, user personas, etc.)\n * Format documents in JSON or Markdown\n\n* **Vector Store Management**\n * Ingest and manage documents in vector store\n * Query and retrieve document content\n * Manage document embeddings\n\n* **Context Management**\n * Manage context for different document types\n * Set and retrieve PRD context\n * Support for future context types (technical specs, user stories, etc.)\n\n* **Knowledge Graph**\n * Generate knowledge graphs from codebase queries\n * Visualize relationships between code entities\n * Analyze code structure and dependencies\n\n* **API Endpoints**\n * `/api/documents/*` - Document generation and formatting\n * `/api/generate/*` - Document formatting utilities\n * `/api/vector-store/*` - Vector store management\n * `/api/agents/*` - AI agent operations\n * `/api/context/*` - Context management for documents\n * `/api/graph/*` - Knowledge graph generation and analysis\n\nNote: JWT authentication support coming soon. All endpoints currently provide detailed OpenAPI documentation.\n\n* **Observability**\n * OpenTelemetry instrumentation for distributed tracing\n * Prometheus metrics endpoint at `/metrics`\n * Comprehensive request/response tracking\n * Custom business metrics\n\n\n\n# CoreStory System Architecture\n\nThis document provides a high-level overview of how the CoreStory platform ingests, analyzes, and serves structured software intelligence through its API. The system is designed to process codebases and related technical artifacts in order to generate product specifications, technical documents, and knowledge graph representations.\n\n---\n\n## Core Components\n\n### 1. **Ingestion Layer**\n\n- Accepts Git URLs, file uploads, or local archives\n- Extracts source files, documentation, and metadata\n- Initiates chunking and embedding via the vector store pipeline\n\n### 2. **Vector Store and Context Engine**\n\n- Stores chunked and embedded documents\n- Supports semantic search and document retrieval\n- Feeds LLMs with scoped context for generation and reasoning\n\n### 3. **LLM Orchestration Layer**\n\n- Fan-out to multiple large language models\n- Handles prompt templating, context injection, and result normalization\n- Evaluates and scores generated outputs\n\n### 4. **Document Generation Engine**\n\n- Produces:\n - PRDs\n - Technical Specs\n - Individual document sections\n - Markdown and PDF exports\n- Format-flexible output for portals, APIs, and documentation pipelines\n\n### 5. **Graph Construction Module**\n\n- Extracts static and dynamic code relationships\n- Builds DAGs and call graphs of components, services, and files\n- Allows natural-language queries that return graph subviews and LLM explanations\n\n### 6. **Query Engine**\n\n- Accepts free-form questions about codebases\n- Routes queries to code graph, document store, or LLMs\n- Returns structured JSON answers and explanatory text\n\n### 7. **API Gateway**\n\n- Authenticates and rate-limits clients\n- Routes requests to internal services\n- Serves all external-facing endpoints in REST format\n\n---\n\n## Security and Privacy\n\n- All data transfer is encrypted via HTTPS\n- API access is secured using provided auth credentials (JWT tokens coming soon)\n- Uploaded code is only used to generate customer-requested outputs and is not retained indefinitely\n\n---\n\n## Extensibility\n\nThe platform is modular. It can be extended to:\n\n- Add new LLMs for ensemble generation\n- Integrate alternative vector databases\n- Connect to internal developer portals or product management tools via webhook or export endpoints\n\nFor more information, see the Quick Start or contact the CoreStory team.\n\n\n# CoreStory API Quick Start Guide\n\nThe CoreStory API provides programmatic access to structured code intelligence, enabling your team to extract business and technical specifications directly from source code. You can use the API to integrate CoreStory-generated documents into existing workflows, support software modernization and maintenance efforts, or enrich internal platforms with architectural and feature-level metadata.\n\n---\n\n## Overview\n\nThe CoreStory API enables:\n\n- Ingestion of content into a **vector store** for LLM-driven analysis\n- Generation of **Product Requirements Documents (PRDs)** and **Technical Specifications** from code\n- Interrogation of your codebase via a natural language **query engine**\n- Construction of **knowledge graphs** describing code relationships and system architecture\n- Management of **custom document context** and **custom prompts** for improved output relevance and quality\n\nThese capabilities support SDLC workflows such as:\n\n- Application modernization, migration, and maintenance\n- Documentation and auditability of legacy systems\n- Alignment of product, engineering, and architecture teams and systems\n- Developer onboarding and code comprehension\n\n---\n\n**Note for POV customers:** If your codebase has already been ingested as part of a proof-of-value (POV) engagement, you can use your existing authentication credentials and skip step 2 (”Ingest a Codebase or Document Source) of this guide.\n\n## 1. Authentication\n\n```\nPOST /api/auth/login\n```\n\nObtain a JWT for future API calls.\n\n**Request:**\n\n```\n{\n \"email\": \"you@example.com\",\n \"password\": \"your-password\"\n}\n```\n\n**Response:**\n\n```\n{\n \"access_token\": \"...\",\n \"token_type\": \"bearer\"\n}\n```\n\nInclude this token in all subsequent requests:\n\n```\nAuthorization: Bearer <access_token>\n```\n\n---\n\n## 2. Ingest a Codebase or Document Source\n\n```\nPOST /api/vector-store/ingest\n```\n\nTrigger ingestion of code from a Git repo, ZIP file, or local document directory.\n\n**Example Payload (GitHub):**\n\n```\n{\n \"source_type\": \"url\",\n \"source\": \"https://github.com/example/repo.git\",\n \"force\": true\n}\n```\n\nIngestion automatically chunks, embeds, and summarizes source files for downstream analysis.\n\n<details>\n<summary>Supported File Extensions</summary>\n\n### Programming Languages\n- Python: `.py`, `.pyw`, `.pyx`, `.pyi`\n- JavaScript/TypeScript: `.js`, `.mjs`, `.cjs`, `.ts`, `.jsx`, `.tsx`\n- Java: `.java`, `.class`, `.jar`\n- C/C++: `.c`, `.cpp`, `.cc`, `.cxx`, `.c++`, `.h`, `.hpp`, `.hh`, `.hxx`, `.h++`\n- .NET: `.cs`, `.csx`, `.vb`, `.fs`, `.fsx`, `.fsi`\n- PHP: `.php`, `.php3`, `.php4`, `.php5`, `.phtml`\n- Ruby: `.rb`, `.rbw`, `.rake`, `.gemspec`\n- Go: `.go`, `.mod`, `.sum`\n- Rust: `.rs`, `.rlib`\n- Swift: `.swift`\n- Kotlin: `.kt`, `.kts`, `.ktm`\n- Scala: `.scala`, `.sc`\n- Clojure: `.clj`, `.cljs`, `.cljc`, `.edn`\n- Haskell: `.hs`, `.lhs`\n- OCaml: `.ml`, `.mli`, `.mll`, `.mly`\n- R: `.r`, `.R`, `.rmd`, `.rnw`\n- Objective-C: `.m`, `.mm`\n- Perl: `.pl`, `.pm`, `.t`, `.pod`\n- Shell scripts: `.sh`, `.bash`, `.zsh`, `.fish`, `.ksh`, `.csh`, `.tcsh`\n- PowerShell: `.ps1`, `.psm1`, `.psd1`\n- Windows batch: `.bat`, `.cmd`\n- Lua: `.lua`\n- Dart: `.dart`\n- Elm: `.elm`\n- Elixir: `.ex`, `.exs`\n- Erlang: `.erl`, `.hrl`\n- Julia: `.jl`\n- Nim: `.nim`, `.nims`\n- Crystal: `.cr`\n- D: `.d`\n- Pascal: `.pas`, `.pp`, `.inc`\n- Fortran: `.f`, `.f90`, `.f95`, `.f03`, `.f08`, `.for`, `.ftn`, `.fpp`\n- COBOL: `.cob`, `.cbl`, `.cpy`, `.cobol`, `.bms`, `.ctl`, `.jcl`, `.proc`\n- IBM i (AS/400) RPG ILE: `.rpgle`, `.rpgleinc`\n- IBM i (AS/400) CLLE: `.clle`\n- IBM i (AS/400) DDS: `.pf`, `.lf`, `.dspf`\n- PowerBuilder: `.pbt`, `.pbs`, `.pbr`, `.srw`, `.srd`, `.sru`, `.sra`, `.srp`, `.srf`, `.srq`, `.srs`, `.srm`, `.srj`\n- Ada: `.ada`, `.adb`, `.ads`\n- Assembly: `.asm`, `.s`, `.S`\n- Verilog/SystemVerilog: `.v`, `.vh`, `.sv`, `.svh`\n- VHDL: `.vhd`, `.vhdl`\n- Tcl/Tk: `.tcl`, `.tk`\n- Groovy: `.groovy`, `.gvy`, `.gy`, `.gsh`\n- CoffeeScript: `.coffee`, `.litcoffee`\n- PureScript: `.purs`\n- ReasonML: `.reason`, `.re`, `.rei`\n- Racket/Scheme: `.rkt`, `.scm`, `.ss`\n- Lisp: `.lisp`, `.lsp`, `.l`, `.cl`, `.fasl`\n- Prolog: `.prolog`, `.pro`, `.P`\n- MATLAB: `.matlab`, `.fig`\n- Mathematica: `.mathematica`, `.nb`, `.wl`, `.wls`\n- SageMath: `.sage`, `.spyx`\n- Zig: `.zig`\n- Odin: `.odin`\n- V: `.v3`, `.v2`, `.v1`\n- Pony: `.pony`\n- Red: `.red`, `.reds`\n- Io: `.io`\n- Factor: `.factor`\n- Forth: `.forth`, `.fth`, `.4th`\n- J/K/Q/APL: `.j`, `.k`, `.q`, `.apl`\n- Chapel: `.chapel`, `.chpl`\n- X10: `.x10`\n- Arduino/Processing: `.pde`, `.ino`\n\n### Web Technologies\n- HTML: `.html`, `.htm`, `.xhtml`, `.shtml`\n- CSS/Preprocessors: `.css`, `.scss`, `.sass`, `.less`, `.styl`, `.stylus`\n- Frontend frameworks: `.vue`, `.svelte`\n- JSP: `.jsp`, `.jspx`, `.tag`, `.tagx`\n- ASP.NET: `.asp`, `.aspx`, `.ascx`, `.asax`, `.ashx`, `.asmx`\n- Ruby templates: `.erb`, `.haml`, `.slim`\n- PHP templates: `.twig`, `.blade`\n- Template engines: `.mustache`, `.hbs`, `.handlebars`, `.ejs`, `.pug`, `.jade`, `.liquid`, `.ftl`, `.ftlh`, `.vm`, `.vtl`, `.thymeleaf`\n\n### Configuration/Data\n- JSON: `.json`, `.json5`, `.jsonc`, `.jsonl`, `.ndjson`\n- XML: `.xml`, `.xsd`, `.xsl`, `.xslt`, `.dtd`, `.rng`, `.rnc`\n- YAML/TOML: `.yaml`, `.yml`, `.toml`\n- INI/Cfg/Conf: `.ini`, `.cfg`, `.conf`, `.config`, `.properties`\n- Environment: `.env`, `.envrc`, `.env.local`, `.env.development`, `.env.production`\n- Property lists: `.plist`\n- HOCON: `.hocon`\n- RON/HJSON/CSON: `.ron`, `.hjson`, `.cson`\n\n### Documentation/Text\n- Markdown: `.md`, `.markdown`, `.mdown`, `.mkd`, `.mkdn`\n- Plain text: `.txt`, `.text`\n- reStructuredText: `.rst`, `.rest`\n- AsciiDoc: `.asciidoc`, `.adoc`, `.asc`\n- LaTeX: `.tex`, `.latex`, `.ltx`, `.sty`, `.cls`, `.bib`\n- Org mode: `.org`\n- Wiki markup: `.wiki`, `.mediawiki`\n- Textile/Creole/RDoc: `.textile`, `.creole`, `.rdoc`\n\n### Data Formats\n- Delimited: `.csv`, `.tsv`, `.psv`\n- SQL: `.sql`, `.ddl`, `.dml`, `.plsql`, `.psql`, `.mysql`\n- GraphQL: `.graphql`, `.gql`\n- Protocols: `.proto`, `.protobuf`, `.avro`, `.avsc`, `.avdl`, `.thrift`, `.capnp`, `.fbs`\n- Columnar: `.parquet`, `.orc`, `.arrow`\n\n### Build/Dependency\n- Gradle: `.gradle`, `.gradle.kts`\n- Maven: `.maven`, `.mvn`\n- SBT: `.sbt`\n- CMake: `.cmake`, `.cmake.in`\n- Make: `.make`, `.mk`, `.mak`\n- Docker: `.dockerfile`, `.containerfile`, `.dockerignore`\n- Git: `.gitignore`, `.gitattributes`, `.gitmodules`, `.gitkeep`\n- Mercurial/Subversion/Bazaar: `.hgignore`, `.hgrc`, `.svnignore`, `.bzrignore`\n- EditorConfig: `.editorconfig`\n- Linters/Formatters: `.eslintrc`, `.eslintignore`, `.prettierrc`, `.prettierignore`, `.babelrc`, `.babelignore`\n- Node.js: `.npmrc`, `.npmignore`, `.yarnrc`, `.yarnignore`\n- Python: `.pipfile`, `.pipfile.lock`, `.requirements`, `.requirements.txt`, `.poetry.lock`, `.pyproject.toml`, `.setup.py`, `.setup.cfg`, `.manifest.in`, `.tox.ini`, `.noxfile.py`, `.pre-commit-config.yaml`\n- CI/CD: `.github`, `.gitlab-ci.yml`, `.travis.yml`, `.appveyor.yml`, `.circleci`, `.jenkins`, `.jenkinsfile`, `.azure-pipelines.yml`, `.buildkite.yml`\n- Package Managers: `package.json`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `composer.json`, `composer.lock`, `gemfile`, `gemfile.lock`, `cargo.toml`, `cargo.lock`, `go.mod`, `go.sum`, `requirements.txt`, `pipfile`, `pipfile.lock`, `poetry.lock`, `pyproject.toml`, `build.gradle`, `build.gradle.kts`, `settings.gradle`, `gradle.properties`, `pom.xml`, `settings.xml`, `build.sbt`, `plugins.sbt`, `makefile`, `gnumakefile`, `makefile.am`, `makefile.in`, `dockerfile`, `containerfile`, `vagrantfile`, `rakefile`, `guardfile`, `capfile`, `gulpfile.js`, `gruntfile.js`, `webpack.config.js`, `rollup.config.js`, `tsconfig.json`, `jsconfig.json`, `deno.json`, `deno.jsonc`, `mix.exs`, `mix.lock`, `rebar.config`, `rebar.lock`, `project.clj`, `deps.edn`, `stack.yaml`, `cabal.config`, `setup.hs`, `dub.json`, `dub.sdl`, `nimble`, `config.nims`, `shard.yml`, `shard.lock`, `pubspec.yaml`, `pubspec.lock`, `elm.json`, `elm-package.json`, `project.json`, `global.json`, `nuget.config`, `packages.config`, `paket.dependencies`, `paket.lock`, `.log`, `.lock`, `.pid`, `.tmp`, `.temp`, `.patch`, `.diff`, `.spec`, `.rpm`, `.deb`, `.control`, `.pkgbuild`, `.ebuild`, `.formula`, `.port`, `.portfile`, `.nix`, `.bzl`, `.bazel`, `.workspace`, `.buck`, `.buckconfig`, `.pants`, `.pants.ini`, `.please`, `.plzconfig`, `.gn`, `.gni`, `.ninja`, `.waf`, `.wscript`, `.scons`, `.sconstruct`, `.sconscript`\n\n</details>\n\n<details>\n<summary>Configuration Files</summary>\n\n- `requirements.txt`\n- `.gitignore`\n- `.dockerignore`\n- `.editorconfig`\n- `.eslintrc.js`\n- `.eslintrc.json`\n- `.prettierrc.js`\n- `.prettierrc.json`\n- `.babelrc.js`\n- `.babelrc.json`\n\n</details>\n\n---\n\n## 3. Generate a Product Requirements Document (PRD)\n\n**Note**: Generating a PRD or technical specification will replace the existing PRD or technical specification. The new outputs will document your codebase as it was when it was most recently ingested.\n\n```\nPOST /api/documents/prd?format=json\n```\n\nGenerates a structured PRD with:\n\n- Executive Overview\n- User Personas\n- High-Level Requirements\n- User Stories\n- Business Value\n\n**Example Response (truncated):**\n\n```\n{\n \"content\": {\n \"executive_overview\": \"Modernize internal authentication platform\",\n \"user_personas\": [...],\n \"user_stories\": [...]\n }\n}\n```\n\n**Tip:** For PRDs or technical specs, use `format=markdown` to return a Markdown-formatted document instead.\n\n---\n\n## 4. Generate a Technical Specification\n\n```\nPOST /api/documents/technical-spec?format=json\n```\n\nReturns system-level implementation details including:\n\n- Architecture and Components\n- Data Models and Relationships\n- User Interface Screens and Components\n- APIs and Integration Points\n- System Interfaces and Specifications\n- Security Considerations\n- Implementation and Deployment Strategy\n\n---\n\n## 5. Query Your Code\n\n```\nPOST /api/graph/query-engine\n```\n\nAsk natural language questions about your codebase to return a structured graph of relationships.\n\n**Example Query:**\n\n```\n{\n \"query\": \"How does the system determine whether a user has admin privileges when accessing the audit log?\"\n}\n```\n\n**Response (truncated):**\n\n```\n{\n \"summary\": \"When a user requests access to the audit log, the system checks their assigned roles in `AuthService.getUserRoles`. If 'admin' is present, access is granted. This check occurs in `AuditLogController` before querying the log database. The middleware `CheckPermissions` is also invoked on this route.\"\n}\n```\n\n---\n\n## 6. Format and Retrieve Document Sections\n\nUse these endpoints to extract or reformat specific document sections:\n\n- `/api/documents/sections/{doc_type}` → generate a specific section\n- `/api/documents/format/markdown-output/{doc_type}` → format JSON as Markdown\n- `/api/documents/format/pdf-output/{doc_type}` → export PDF\n\nExample section names:\n\n- `executive-overview`\n- `data-models`\n- `system-architecture`\n- `integration-points`\n\n---\n\n## Questions?\n\nAfter reviewing this guide and the API docs, we recommend scheduling a short Q&A call if you have integration-specific needs or want assistance mapping this to your environment.\n"
version: 1.26.2
servers:
- url: /
description: Current server
security:
- BearerAuth: []
tags:
- name: pre_ingestion
paths:
/api/pre-ingestion/start:
post:
tags:
- pre_ingestion
summary: Start Pre-Ingestion Process
description: "Start the pre-ingestion process for a specific repository. This process\nwill analyze the repository and return a summary of the supported files.\n\nArgs:\n request (PreIngestionRequest): The incoming request object.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n db (Session): The database session.\n\nReturns:\n PreIngestionResponse: Details about the initiated pre-ingestion process."
operationId: start_pre_ingestion
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/StartPreIngestionRequest'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/start/upload-repository:
post:
tags:
- pre_ingestion
summary: Upload repository files and run the pre-ingestion analysis
description: Upload a single zipped repository file to Azure Blob Storage or Local directory,create a new project, and trigger pre-ingestion analysis.
operationId: start_pre_ingestion_from_upload
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_start_pre_ingestion_from_upload'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/upload-url:
post:
tags:
- pre_ingestion
summary: Get a pre-signed URL for direct file upload
description: Returns a time-limited pre-signed URL that allows the client to upload a file directly to storage, bypassing server-side proxying and Cloudflare's upload size limits.
operationId: get_upload_url
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/UploadUrlRequest'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/UploadUrlResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/start/from-storage:
post:
tags:
- pre_ingestion
summary: Start pre-ingestion from an already-uploaded file
description: Triggers the pre-ingestion analysis for a file that was uploaded directly to storage via a pre-signed URL obtained from the /upload-url endpoint.
operationId: start_pre_ingestion_from_storage
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/StartFromStorageRequest'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/{job_id}/status:
get:
tags:
- pre_ingestion
summary: Get Pre-Ingestion Job Status
description: "Get the status of a pre-ingestion job by its ID.\n\nArgs:\n job_id (int): The ID of the pre-ingestion job.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n db (Session): The database session.\n\nReturns:\n PreIngestionResponse: Details about the pre-ingestion job status."
operationId: get_pre_ingestion_status
parameters:
- name: job_id
in: path
required: true
schema:
type: integer
title: Job Id
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/{job_id}/coverage:
get:
tags:
- pre_ingestion
summary: Get Pre-Ingestion Job Results
description: "Get the coverage results of a pre-ingestion job by its ID.\n\nArgs:\n job_id (int): The ID of the pre-ingestion job.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n db (Session): The database session.\n\nReturns:\n PreIngestionCoverageResponse: Coverage details of the pre-ingestion job."
operationId: get_pre_ingestion_coverage
parameters:
- name: job_id
in: path
required: true
schema:
type: integer
title: Job Id
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionCoverageResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/{job_id}/excluded-files:
get:
tags:
- pre_ingestion
summary: Get Excluded File Paths from Pre-Ingestion Job
description: "Get the list of excluded file paths from a pre-ingestion job by its ID.\n\nArgs:\n job_id (int): The ID of the pre-ingestion job.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n db (Session): The database session.\n file_name (str, optional): Filter files by name.\n\nReturns:\n list[str]: List of excluded file paths."
operationId: get_pre_ingestion_excluded_files
parameters:
- name: job_id
in: path
required: true
schema:
type: integer
title: Job Id
- name: page
in: query
required: false
schema:
type: integer
minimum: 1
description: Page number (1-based)
default: 1
title: Page
description: Page number (1-based)
- name: size
in: query
required: false
schema:
type: integer
maximum: 1000
minimum: 1
description: Items per page
default: 100
title: Size
description: Items per page
- name: file_name
in: query
required: false
schema:
anyOf:
- type: string
- type: 'null'
title: File Name
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PaginatedFilePathResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/{job_id}/included-files:
get:
tags:
- pre_ingestion
summary: Get Included File Paths from Pre-Ingestion Job
description: "Get the list of included file paths from a pre-ingestion job by its ID.\n\nArgs:\n job_id (int): The ID of the pre-ingestion job.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n db (Session): The database session.\n file_name (str, optional): Filter files by name.\n\nReturns:\n list[str]: Paginated list of included file paths."
operationId: get_pre_ingestion_included_files
parameters:
- name: job_id
in: path
required: true
schema:
type: integer
title: Job Id
- name: page
in: query
required: false
schema:
type: integer
minimum: 1
description: Page number (1-based)
default: 1
title: Page
description: Page number (1-based)
- name: size
in: query
required: false
schema:
type: integer
maximum: 1000
minimum: 1
description: Items per page
default: 100
title: Size
description: Items per page
- name: file_name
in: query
required: false
schema:
anyOf:
- type: string
- type: 'null'
title: File Name
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PaginatedFilePathResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/{job_id}:
delete:
tags:
- pre_ingestion
summary: Delete Pre-Ingestion Job
description: "Delete a pre-ingestion job by its ID.\n\nArgs:\n job_id (int): The ID of the pre-ingestion job.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n db (Session): The database session.\n storage (StorageBackend): Storage backend for artifact cleanup.\n\nReturns:\n dict: Confirmation message upon successful deletion."
operationId: delete_pre_ingestion_job
parameters:
- name: job_id
in: path
required: true
schema:
type: integer
title: Job Id
responses:
'200':
description: Successful Response
content:
application/json:
schema:
type: object
additionalProperties: true
title: Response Delete Pre Ingestion Job
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
patch:
tags:
- pre_ingestion
summary: Update Pre-Ingestion Job
description: "Update a pre-ingestion job by its ID.\n\nArgs:\n job_id (int): The ID of the pre-ingestion job.\n user (ClerkUser): The authenticated user.\n organization (Organization): The organization context.\n application_context (str, optional): New application context.\n db (Session): The database session.\n\nReturns:\n PreIngestionResponse: Updated details of the pre-ingestion job."
operationId: update_pre_ingestion_job
parameters:
- name: job_id
in: path
required: true
schema:
type: integer
title: Job Id
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionUpdateRequest'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PreIngestionResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
/api/pre-ingestion/{pre_ingestion_id}/estimate-ingestion:
get:
tags:
- pre_ingestion
summary: Estimate CIU cost for a number of LOC and files count.
description: Estimate the CIU cost for ingesting a codebase based on the number of lines of code and files.
operationId: estimate_ciu_cost_for_ingestion
parameters:
- name: pre_ingestion_id
in: path
required: true
schema:
type: integer
title: Pre Ingestion Id
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/EstimateIngestionResponse'
default:
description: Error
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorSchema'
components:
schemas:
ErrorSchema:
properties:
error:
$ref: '#/components/schemas/ErrorBody'
type: object
required:
- error
title: ErrorSchema
PaginatedFilePathResponse:
properties:
results:
items:
type: string
type: array
title: Results
description: List of filepaths in pre-ingestion job
pagination:
anyOf:
- additionalProperties: true
type: object
- type: 'null'
title: Pagination
type: object
required:
- results
- pagination
title: PaginatedFilePathResponse
PreIngestionCoverageResponse:
properties:
project_id:
anyOf:
- type: integer
- type: 'null'
title: Project Id
repository_url:
anyOf:
- type: string
- type: 'null'
title: Repository Url
branch:
anyOf:
- type: string
- type: 'null'
title: Branch
total_files:
type: integer
title: Total Files
default: 0
total_directories:
type: integer
title: Total Directories
default: 0
code_analysis:
additionalProperties: true
type: object
title: Code Analysis
description: Dict with ingestion files analysis and detections.
file_coverage:
type: number
title: File Coverage
description: Coverage percentage from 0.0 to 100.0
default: 0.0
included_count:
additionalProperties: true
type: object
title: Included Count
description: List of objects representing file type and its count
excluded_count:
additionalProperties: true
type: object
title: Excluded Count
description: List of objects representing file type and its count
exclusion_reasons:
additionalProperties: true
type: object
title: Exclusion Reasons
description: Breakdown of files by exclusion reason (e.g., minified, third_party_vendored)
# --- truncated at 32 KB (39 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/corestory/refs/heads/main/openapi/corestory-pre-ingestion-api-openapi.yml