ngrok AI Gateway

ngrok AI Gateway provides traffic management and security for AI APIs including multi-provider routing, automatic failover, LLM prompt inspection, rate limiting, caching, observability, PII redaction, and access control. It enables teams to manage, secure, and monitor traffic to AI model providers (OpenAI, Anthropic, Google, DeepSeek) and self-hosted models such as Ollama and vLLM through an OpenAI-compatible interface.

1 APIs 11 Features

AIAI GatewayAPI GatewayLLMOpenAI CompatibleRoutingSecurityTraffic Management

APIs

ngrok AI Gateway

ngrok AI Gateway exposes an OpenAI-compatible HTTP interface for routing requests across multiple AI providers and self-hosted models. Each AI Gateway instance has a unique base...

Collections

ngrok AI Gateway API

OPEN

Pricing Plans

Ngrok Ai Plans Pricing

3 plans

PLANS

Rate Limits

Ngrok Ai Rate Limits

5 limits

RATE LIMITS

FinOps

Ngrok Ai Finops

FINOPS

Features

Multi-Provider Routing

Direct requests to AI providers including OpenAI, Anthropic, Google, and DeepSeek through a single gateway endpoint.

Automatic Failover

If one provider or model fails, the gateway automatically tries the next configured model.

OpenAI SDK Compatibility

Works with official and third-party OpenAI SDKs by changing only the base URL.

Self-Hosted Model Support

Route requests to local systems such as Ollama or vLLM alongside hosted providers.

Automatic Model Selection

Use ngrok/auto for intelligent model picking based on configured strategies.

CEL-Based Selection Strategies

Define custom routing logic using Common Expression Language expressions.

Cost-Based Routing

Direct traffic to the cheapest available model option meeting requirements.

Access Control

Restrict which providers and models clients can use by API key, identity, or policy.

PII Redaction

Inspect and modify content to remove personally identifiable information from prompts and responses.

Response Sanitization

Modify and filter responses before they reach clients.

No Provider Account Required

Access OpenAI and Anthropic models without individual provider signup, using ngrok credits.

Use Cases

Centralized AI API Management

Manage all AI provider traffic through a single gateway with unified observability and policy enforcement.

Cost Optimization

Route traffic to the most cost-effective model that meets quality requirements.

Compliance and Data Protection

Enforce PII redaction and prompt inspection policies before requests leave the organization.

Multi-Provider Resilience

Failover automatically across providers to maintain AI service availability.

Local and Hybrid Model Routing

Route between hosted providers and self-hosted models such as Ollama or vLLM.

Resources

GitHubOrganization

Sources

opencollection: 1.0.0
info:
  name: ngrok AI Gateway API
  version: v1
items:
- info:
    name: OpenAI
    type: folder
  items:
  - info:
      name: OpenAI-compatible chat completions
      type: http
    http:
      method: POST
      url: https://{gatewayDomain}/v1/chat/completions
      body:
        type: json
        data: '{}'
      auth:
        type: bearer
        token: '{{bearerToken}}'
    docs: 'Sends a chat completion request in OpenAI''s Chat Completions

      format. The gateway routes the request to the configured

      upstream model provider.

      '
- info:
    name: Anthropic
    type: folder
  items:
  - info:
      name: Anthropic-compatible messages
      type: http
    http:
      method: POST
      url: https://{gatewayDomain}/v1/messages
      headers:
      - name: anthropic-version
        value: ''
      body:
        type: json
        data: '{}'
      auth:
        type: apikey
        key: x-api-key
        value: '{{x-api-key}}'
        placement: header
    docs: 'Sends a message creation request in Anthropic''s Messages

      API format. The gateway routes the request to the configured

      upstream model provider.

      '
bundled: true