Cumulus Labs

Cumulus Inference Gateway

OpenAI-compatible HTTP inference gateway. One client works against every upstream provider; per-workflow routing rules pick the model, provider, and infrastructure, with a layered exact/prefix/semantic cache, request-level observability, and the Ion engine serving open-weight models and fine-tunes.

GitHub

Documentation

📖
Documentation
https://docs.cumuluslabs.io/
📖
GettingStarted
https://docs.cumuluslabs.io/getting-started/
📖
APIReference
https://docs.cumuluslabs.io/inference/overview/
📖
Authentication
https://raw.githubusercontent.com/api-evangelist/cumulus-labs/refs/heads/main/authentication/cumulus-labs-authentication.yml

Other Resources

🔗
APIsJSON
https://raw.githubusercontent.com/api-evangelist/cumulus-labs/refs/heads/main/apis.yml

API entry from apis.yml

apis.yml Raw ↑ 
name: Cumulus Inference Gateway
description: OpenAI-compatible HTTP inference gateway. One client works against every upstream provider;
  per-workflow routing rules pick the model, provider, and infrastructure, with a layered exact/prefix/semantic
  cache, request-level observability, and the Ion engine serving open-weight models and fine-tunes.
humanURL: https://docs.cumuluslabs.io/
baseURL: https://api.cumuluslabs.io/v1
tags:
- Inference
- LLM
- OpenAI-Compatible
- Gateway
- GPU
properties:
- type: Documentation
  url: https://docs.cumuluslabs.io/
- type: GettingStarted
  url: https://docs.cumuluslabs.io/getting-started/
- type: APIReference
  url: https://docs.cumuluslabs.io/inference/overview/
- type: Authentication
  url: authentication/cumulus-labs-authentication.yml