Lucidworks Tokenization API

Tokenize text by model

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools

Call it yourself

curl for this page
This API
curl "https://apis.io/api/v1/apis/lucidworks-tokenization-api"
All apis
curl "https://apis.io/api/v1/apis?limit=25"

Discovery needs no key. Ratings and market analysis are Pro.

Get an API key

Free tier, no email required.

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

lucidworks-tokenization-api-openapi.yml Raw ↑
openapi: 3.1.0
info:
  title: Lucidworks AI Platform Chunking Tokenization API
  description: The Lucidworks AI Platform API exposes prediction endpoints for FAQ enrichment, keyword extraction, named entity recognition (NER), retrieval augmented generation (RAG), summarization, passthrough LLM calls, and standalone query rewriting. Predictions are submitted by use case and fetched asynchronously by prediction ID.
  version: 1.0.0
  contact:
    name: Lucidworks Support
    url: https://lucidworks.com/support
  license:
    name: Lucidworks Terms of Service
    url: https://lucidworks.com/terms
servers:
- url: https://api.lucidworks.ai
  description: Lucidworks AI production environment
security:
- bearerAuth: []
tags:
- name: Tokenization
  description: Tokenize text by model
paths:
  /ai/tokenize/{modelId}:
    post:
      tags:
      - Tokenization
      summary: Tokenize text
      description: Return token IDs and tokens for the input text.
      operationId: tokenize
      parameters:
      - name: modelId
        in: path
        required: true
        schema:
          type: string
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
              - input
              properties:
                input:
                  type: string
      responses:
        '200':
          description: Tokens
          content:
            application/json:
              schema:
                type: object
                properties:
                  tokens:
                    type: array
                    items:
                      type: string
                  ids:
                    type: array
                    items:
                      type: integer
components:
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT
externalDocs:
  description: Lucidworks AI Platform API Reference
  url: https://doc.lucidworks.com/api-reference