Mango LLMBoost Inference Server API

aid: mangoboost:llmboost-inference
name: Mango LLMBoost Inference Server API
description: 'LLMBoost is MangoBoost''s enterprise LLM inference server. It serves the OpenAI REST API
  on /v1 so an existing OpenAI client migrates with a base-URL change: POST /v1/chat/completions, POST
  /v1/completions, POST /v1/embeddings, POST /v1/responses, POST /v1/audio/transcriptions and /translations,
  GET /v1/models, plus GET /health and GET /metrics (Prometheus). Streaming, JSON-schema structured output,
  tool/function calling and multimodal image input are supported. The server is self-hosted on the customer''s
  own AMD Instinct GPUs — there is no MangoBoost-hosted endpoint — and is started with `lbh serve <Repo/Model>`
  or `llmboost serve <model>` inside the LLMBoost container.'
humanURL: https://llmboost.mangoboost.io/docs/features/openai-api
baseURL: http://localhost:8000/v1
x-deployment: self-hosted
x-default-port: 8000
tags:
- Inference
- Artificial Intelligence
- LLM
- OpenAI Compatible
- Self Hosted
properties:
- type: Documentation
  url: https://llmboost.mangoboost.io/docs/
- type: GettingStarted
  url: https://llmboost.mangoboost.io/docs/quickstart
- type: APIReference
  url: https://llmboost.mangoboost.io/docs/features/openai-api