Scalable Inference Serving

BentoML REST API

BentoML is an open-source unified inference platform for deploying and scaling AI models. It auto-generates RESTful APIs from Python service definitions, provides built-in OpenAPI/Swagger documentation, supports adaptive batching, and integrates with KServe for Kubernetes deployment. BentoML 1.0 introduced the Runner abstraction for parallelizing inference workloads with adaptive batching and independent scaling of pre/post-processing from model inference.

GitHub

Documentation

📖
Documentation
https://docs.bentoml.com/en/latest/
📖
GettingStarted
https://docs.bentoml.com/en/latest/get-started/quickstart.html
📖
APIReference
https://docs.bentoml.com/en/latest/reference/index.html

Other Resources

🔗
GitHub
https://github.com/bentoml/BentoML
🔗
Pricing
https://www.bentoml.com/pricing
🔗
APIsJSON
https://raw.githubusercontent.com/api-evangelist/scalable-inference-serving/refs/heads/main/apis.yml

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools

Call it yourself

curl for this page
This API 
curl "https://apis.io/api/v1/apis/bentoml-rest-api"
All apis 
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no email required.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have.

API entry from apis.yml

apis.yml Raw ↑ 
name: BentoML REST API
description: BentoML is an open-source unified inference platform for deploying and scaling AI models.
  It auto-generates RESTful APIs from Python service definitions, provides built-in OpenAPI/Swagger documentation,
  supports adaptive batching, and integrates with KServe for Kubernetes deployment. BentoML 1.0 introduced
  the Runner abstraction for parallelizing inference workloads with adaptive batching and independent
  scaling of pre/post-processing from model inference.
image: https://www.bentoml.com/favicon.ico
humanUrl: https://www.bentoml.com/
baseUrl: https://api.bentoml.example.com
tags:
- Batching
- Inference
- Model Serving
- Open-Source
- Python
- REST API
tags_raw:
- Batching
- Inference
- Model Serving
- Open Source
- Python
- REST API
properties:
- type: Documentation
  url: https://docs.bentoml.com/en/latest/
- type: GitHub
  url: https://github.com/bentoml/BentoML
- type: GettingStarted
  url: https://docs.bentoml.com/en/latest/get-started/quickstart.html
- type: Pricing
  url: https://www.bentoml.com/pricing
- type: APIReference
  url: https://docs.bentoml.com/en/latest/reference/index.html
contact:
- type: Community
  url: https://l.bentoml.com/join-slack
- type: GitHubIssues
  url: https://github.com/bentoml/BentoML/issues