Elastic Stack (ELK Stack) Inference API
The inference API from Elastic Stack (ELK Stack) — 40 operation(s) for inference.
Operations 48
Each operation below carries the questions people ask an LLM about it and the instructions they give an agent to run it. Generated by API Evangelist overlay
POST
/_inference/chat_completion/{inference_id}/_stream
Stream a chat completion from an inference endpoint · Perform chat completion inference on the service
#
Ask an LLM
“Can I stream a multi-turn chat conversation through an Elasticsearch chat_completion endpoint?”
“Does the streaming chat API let me pass tools and a tool_choice to the model?”
Tell an agent
Stream a chat reply from chat_completion endpoint {inference_id} for these messages: {messages}.
Send {messages} to chat endpoint {inference_id} with temperature {temperature} and stream the answer back.
POST
/_inference/completion/{inference_id}
Get a text completion from an inference endpoint · Perform completion inference on the service
#
Ask an LLM
“How do I get a non-streaming text completion from a completion inference endpoint?”
“Can I send a prompt to a completion endpoint and get the whole answer back in one response?”
Tell an agent
Complete the prompt {input} using completion endpoint {inference_id} and return the full response.
Run a one-shot completion on {inference_id} for {input}.
Ask an LLM
“What service and settings is a given inference endpoint configured with?”
“Can I look up one inference endpoint by its ID without knowing its task type?”
Tell an agent
Show me the configuration of inference endpoint {inference_id}.
Look up inference endpoint {inference_id} by ID alone.
PUT
/_inference/{inference_id}
Create an inference endpoint without a task type · Create an inference endpoint
#
Ask an LLM
“Can I create an inference endpoint just by ID, letting the service determine the task type?”
“What do I need to supply in service and service_settings to register a generic inference endpoint?”
Tell an agent
Create inference endpoint {inference_id} using service {service} with settings {service_settings}, no task type in the path.
Register a new inference endpoint named {inference_id} on the {service} service.
POST
/_inference/{inference_id}
Run inference against an endpoint by ID · Perform inference on the service
#
Ask an LLM
“Can I send input to an inference endpoint by ID and let it perform whatever task it was configured for?”
“Which input do I pass to run a generic inference call when I don't want to name the task type?”
Tell an agent
Run inference endpoint {inference_id} on the input {input}.
Call {inference_id} with {input} using its configured task, no task type given.
DELETE
/_inference/{inference_id}
Delete an inference endpoint by ID · Delete an inference endpoint
#
Ask an LLM
“How do I remove an inference endpoint I no longer need?”
“Can I do a dry run to see which ingest pipelines still reference an inference endpoint before deleting it?”
Tell an agent destructive · confirm first
Delete inference endpoint {inference_id}.
Dry-run deleting inference endpoint {inference_id} to see what references it.
GET
/_inference/{task_type}/{inference_id}
Look up an inference endpoint by task type and ID · Get an inference endpoint
#
Ask an LLM
“Can I fetch an inference endpoint scoped to a specific task type like text_embedding?”
“What configuration does my rerank endpoint have when I look it up under its task type?”
Tell an agent
Get the {task_type} inference endpoint {inference_id}.
Show the settings for {inference_id} under task type {task_type}.
PUT
/_inference/{task_type}/{inference_id}
Create an inference endpoint for a task type · Create an inference endpoint
#
Ask an LLM
“How do I create an inference endpoint for a specific task type such as sparse_embedding or completion?”
“Can I set chunking settings when creating an inference endpoint for a given task type?”
Tell an agent
Create a {task_type} inference endpoint {inference_id} on service {service}.
Set up {inference_id} for task type {task_type} with service settings {service_settings}.
POST
/_inference/{task_type}/{inference_id}
Run inference for a given task type · Perform inference on the service
#
Ask an LLM
“Can I run inference while naming the task type explicitly in the request path?”
“What happens if I call an endpoint with a task type it wasn't created for?”
Tell an agent
Run {task_type} inference on endpoint {inference_id} with input {input}.
Use the {task_type} endpoint {inference_id} to process {input}.
DELETE
/_inference/{task_type}/{inference_id}
Delete an inference endpoint of a task type · Delete an inference endpoint
#
Ask an LLM
“Can I delete an inference endpoint by naming both its task type and ID?”
“Is there a force option to delete a task-typed inference endpoint that pipelines still use?”
Tell an agent destructive · confirm first
Delete the {task_type} inference endpoint {inference_id}.
Force-delete {inference_id} under task type {task_type} even if it is referenced.
Ask an LLM
“Which geographic regions is inference currently restricted to on my cluster?”
“Is there a region policy set for inference right now?”
Tell an agent
Show me the current inference region policy.
Check which regions inference is allowed to run in.
PUT
/_inference/_region_policy
Set the inference region policy · Create or update the inference region policy
#
Ask an LLM
“How do I restrict inference to certain cloud regions or geographic areas?”
“Can I change the allowed inference regions after a policy already exists?”
Tell an agent
Restrict inference to the regions in {region_policy}.
Update the inference region policy to {region_policy}, forcing the change.
DELETE
/_inference/_region_policy
Remove the inference region policy · Delete the inference region policy
#
Ask an LLM
“How can I lift the geographic restriction on where inference runs?”
“What removes the inference region policy entirely?”
Tell an agent destructive · confirm first
Delete the inference region policy.
Remove all regional restrictions on inference.
POST
/_inference/embedding/{inference_id}
Generate dense embeddings for input · Perform dense embedding inference on the service
#
Ask an LLM
“Can I get dense vector embeddings from an embedding task endpoint?”
“Does the dense embedding call accept an input_type to mark the text as a query or a document?”
Tell an agent
Generate dense embeddings for {input} using embedding endpoint {inference_id}.
Embed {input} as dense vectors with {inference_id}, input type {input_type}.
Ask an LLM
“What inference endpoints exist on my cluster across every task type?”
“Can I see every inference endpoint, including the preconfigured ELSER and E5 ones?”
Tell an agent
List all my inference endpoints.
Show every inference endpoint on the cluster regardless of task type.
GET
/_inference/{task_type}/_all
List inference endpoints for one task type · Get an inference endpoint
#
Ask an LLM
“Which inference endpoints do I have for text_embedding?”
“Can I list only the rerank endpoints on my cluster?”
Tell an agent
List all {task_type} inference endpoints.
Show every endpoint configured for task type {task_type}.
PUT
/_inference/{task_type}/{ai21_inference_id}
Create an AI21 inference endpoint · Create a AI21 inference endpoint
#
Ask an LLM
“How do I connect Elasticsearch to AI21 models for completion or chat?”
“What service settings does an ai21 inference endpoint need?”
Tell an agent
Create an AI21 {task_type} inference endpoint named {ai21_inference_id}.
Set up AI21 endpoint {ai21_inference_id} for {task_type} with settings {service_settings}.
PUT
/_inference/{task_type}/{alibabacloud_inference_id}
Create an AlibabaCloud AI Search inference endpoint · Create an AlibabaCloud AI Search inference endpoint
#
Ask an LLM
“Can I use AlibabaCloud AI Search models for embeddings or reranking in Elasticsearch?”
“What's needed to register an alibabacloud-ai-search inference endpoint?”
Tell an agent
Create an AlibabaCloud AI Search {task_type} endpoint {alibabacloud_inference_id}.
Register AlibabaCloud AI Search endpoint {alibabacloud_inference_id} for {task_type} using {service_settings}.
PUT
/_inference/{task_type}/{amazonbedrock_inference_id}
Create an Amazon Bedrock inference endpoint · Create an Amazon Bedrock inference endpoint
#
Ask an LLM
“How do I hook Amazon Bedrock models into Elasticsearch inference?”
“Do I have to re-enter my Amazon Bedrock access and secret keys after creating the endpoint?”
Tell an agent
Create an Amazon Bedrock {task_type} endpoint called {amazonbedrock_inference_id}.
Connect Amazon Bedrock as {amazonbedrock_inference_id} for {task_type} with settings {service_settings}.
PUT
/_inference/{task_type}/{amazonsagemaker_inference_id}
Create an Amazon SageMaker inference endpoint · Create an Amazon SageMaker inference endpoint
#
Ask an LLM
“Can I point an Elasticsearch inference endpoint at a model hosted on Amazon SageMaker?”
“What settings does the amazon_sagemaker inference service expect?”
Tell an agent
Create an Amazon SageMaker {task_type} endpoint {amazonsagemaker_inference_id}.
Wire my SageMaker model in as {amazonsagemaker_inference_id} for {task_type} using {service_settings}.
PUT
/_inference/{task_type}/{anthropic_inference_id}
Create an Anthropic inference endpoint · Create an Anthropic inference endpoint
#
Ask an LLM
“How do I use Anthropic models for completion through Elasticsearch?”
“What does an anthropic inference endpoint need in its service settings?”
Tell an agent
Create an Anthropic {task_type} endpoint named {anthropic_inference_id}.
Register Anthropic endpoint {anthropic_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{azureaistudio_inference_id}
Create an Azure AI Studio inference endpoint · Create an Azure AI studio inference endpoint
#
Ask an LLM
“Can I use models deployed in Azure AI Studio for embeddings or completion in Elasticsearch?”
“What's required to create an azureaistudio inference endpoint?”
Tell an agent
Create an Azure AI Studio {task_type} endpoint {azureaistudio_inference_id}.
Connect Azure AI Studio as {azureaistudio_inference_id} for {task_type} using {service_settings}.
PUT
/_inference/{task_type}/{azureopenai_inference_id}
Create an Azure OpenAI inference endpoint · Create an Azure OpenAI inference endpoint
#
Ask an LLM
“How do I use my Azure OpenAI deployment for embeddings or chat completion in Elasticsearch?”
“Which Azure OpenAI chat models can an azureopenai inference endpoint point at?”
Tell an agent
Create an Azure OpenAI {task_type} endpoint named {azureopenai_inference_id}.
Connect my Azure OpenAI deployment as {azureopenai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{cohere_inference_id}
Create a Cohere inference endpoint · Create a Cohere inference endpoint
#
Ask an LLM
“Can I use Cohere embeddings or rerank models inside Elasticsearch?”
“What goes in service_settings for a cohere inference endpoint?”
Tell an agent
Create a Cohere {task_type} endpoint {cohere_inference_id}.
Register Cohere as {cohere_inference_id} for {task_type} using {service_settings}.
PUT
/_inference/{task_type}/{contextualai_inference_id}
Create a Contextual AI inference endpoint · Create an Contextual AI inference endpoint
#
Ask an LLM
“How do I use Contextual AI rerank models from Elasticsearch?”
“What settings does a Contextual AI rerank endpoint need?”
Tell an agent
Create a Contextual AI {task_type} endpoint named {contextualai_inference_id}.
Set up Contextual AI reranking as {contextualai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{custom_inference_id}
Create a custom inference endpoint · Create a custom inference endpoint
#
Ask an LLM
“Can I connect Elasticsearch to an inference service that has no dedicated integration?”
“How do I define my own request and response format for an external model with the custom service?”
Tell an agent
Create a custom {task_type} inference endpoint {custom_inference_id} for an unsupported external service.
Define custom endpoint {custom_inference_id} for {task_type} with request and response settings {service_settings}.
PUT
/_inference/{task_type}/{deepseek_inference_id}
Create a DeepSeek inference endpoint · Create a DeepSeek inference endpoint
#
Ask an LLM
“Can I use DeepSeek models for completion through Elasticsearch inference?”
“What does a deepseek inference endpoint need to be configured?”
Tell an agent
Create a DeepSeek {task_type} endpoint {deepseek_inference_id}.
Register DeepSeek as {deepseek_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{elasticsearch_inference_id}
Create an Elasticsearch-hosted model inference endpoint · Create an Elasticsearch inference endpoint
#
Ask an LLM
“How do I deploy a model running inside my own cluster, like E5 or one uploaded with Eland, as an inference endpoint?”
“Do I need to create an endpoint for ELSER or E5 if the deployment already has preconfigured ones?”
Tell an agent
Create an elasticsearch-service {task_type} endpoint {elasticsearch_inference_id} for a model in my cluster.
Deploy the in-cluster model as {elasticsearch_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{elser_inference_id}
Create an ELSER inference endpoint · Create an ELSER inference endpoint
#
Ask an LLM
“How do I set up an ELSER endpoint for sparse semantic search?”
“Is creating an ELSER endpoint through the elser service still the recommended way to deploy ELSER?”
Tell an agent
Create an ELSER {task_type} endpoint named {elser_inference_id}.
Deploy ELSER as {elser_inference_id} for {task_type} with allocation settings {service_settings}.
PUT
/_inference/{task_type}/{fireworksai_inference_id}
Create a Fireworks AI inference endpoint · Create a Fireworks AI inference endpoint
#
Ask an LLM
“Can I call Fireworks AI models from an Elasticsearch inference endpoint?”
“What settings does the fireworksai service require?”
Tell an agent
Create a Fireworks AI {task_type} endpoint {fireworksai_inference_id}.
Register Fireworks AI as {fireworksai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{googleaistudio_inference_id}
Create a Google AI Studio inference endpoint · Create an Google AI Studio inference endpoint
#
Ask an LLM
“How do I use Google AI Studio models for completion or embeddings in Elasticsearch?”
“What does a googleaistudio endpoint need in service_settings?”
Tell an agent
Create a Google AI Studio {task_type} endpoint {googleaistudio_inference_id}.
Connect Google AI Studio as {googleaistudio_inference_id} for {task_type} using {service_settings}.
PUT
/_inference/{task_type}/{googlevertexai_inference_id}
Create a Google Vertex AI inference endpoint · Create a Google Vertex AI inference endpoint
#
Ask an LLM
“Can I use Google Vertex AI embeddings or rerankers from Elasticsearch?”
“Which settings are required for a googlevertexai inference endpoint?”
Tell an agent
Create a Google Vertex AI {task_type} endpoint {googlevertexai_inference_id}.
Connect Vertex AI as {googlevertexai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{groq_inference_id}
Create a Groq inference endpoint · Create a Groq inference endpoint
#
Ask an LLM
“How can I run Groq-hosted models through Elasticsearch inference?”
“What does the groq service need to create an endpoint?”
Tell an agent
Create a Groq {task_type} endpoint {groq_inference_id}.
Register Groq as {groq_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{huggingface_inference_id}
Create a Hugging Face inference endpoint · Create a Hugging Face inference endpoint
#
Ask an LLM
“Can I use a Hugging Face Inference Endpoint for text embeddings in Elasticsearch?”
“Does the hugging_face service support chat_completion as well as text_embedding?”
Tell an agent
Create a Hugging Face {task_type} endpoint {huggingface_inference_id}.
Connect my Hugging Face endpoint URL as {huggingface_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{jinaai_inference_id}
Create a JinaAI inference endpoint · Create an JinaAI inference endpoint
#
Ask an LLM
“Can I use JinaAI rerankers or embedding models in Elasticsearch?”
“What goes in the settings for a jinaai inference endpoint?”
Tell an agent
Create a JinaAI {task_type} endpoint {jinaai_inference_id}.
Register JinaAI as {jinaai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{llama_inference_id}
Create a Llama inference endpoint · Create a Llama inference endpoint
#
Ask an LLM
“How do I connect Llama models to Elasticsearch inference?”
“What service settings does a llama endpoint need?”
Tell an agent
Create a Llama {task_type} endpoint {llama_inference_id}.
Register Llama as {llama_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{mistral_inference_id}
Create a Mistral inference endpoint · Create a Mistral inference endpoint
#
Ask an LLM
“Can I use Mistral embeddings or completion models in Elasticsearch?”
“What does the mistral service need in service_settings?”
Tell an agent
Create a Mistral {task_type} endpoint {mistral_inference_id}.
Register Mistral as {mistral_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{nvidia_inference_id}
Create an Nvidia inference endpoint · Create an Nvidia inference endpoint
#
Ask an LLM
“How do I use Nvidia-hosted models for inference in Elasticsearch?”
“What settings does an nvidia inference endpoint require?”
Tell an agent
Create an Nvidia {task_type} endpoint {nvidia_inference_id}.
Register Nvidia as {nvidia_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{openai_inference_id}
Create an OpenAI inference endpoint · Create an OpenAI inference endpoint
#
Ask an LLM
“Can I use OpenAI or an OpenAI-compatible API for embeddings and chat in Elasticsearch?”
“What goes in service_settings for an openai inference endpoint?”
Tell an agent
Create an OpenAI {task_type} endpoint {openai_inference_id}.
Register an OpenAI-compatible API as {openai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{openshiftai_inference_id}
Create an OpenShift AI inference endpoint · Create an OpenShift AI inference endpoint
#
Ask an LLM
“How do I connect models served on OpenShift AI to Elasticsearch inference?”
“What does the openshift_ai service need to be configured?”
Tell an agent
Create an OpenShift AI {task_type} endpoint {openshiftai_inference_id}.
Register OpenShift AI as {openshiftai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{voyageai_inference_id}
Create a VoyageAI inference endpoint · Create a VoyageAI inference endpoint
#
Ask an LLM
“Can I use VoyageAI embedding or rerank models in Elasticsearch?”
“Why shouldn't I create several VoyageAI endpoints for the same model?”
Tell an agent
Create a VoyageAI {task_type} endpoint {voyageai_inference_id}.
Register VoyageAI as {voyageai_inference_id} for {task_type} with {service_settings}.
PUT
/_inference/{task_type}/{watsonx_inference_id}
Create a watsonx inference endpoint · Create a Watsonx inference endpoint
#
Ask an LLM
“How do I use IBM watsonx.ai models for inference in Elasticsearch?”
“Do I need an IBM Cloud Databases for Elasticsearch deployment to use the watsonxai service?”
Tell an agent
Create a watsonx {task_type} endpoint {watsonx_inference_id}.
Register watsonx.ai as {watsonx_inference_id} for {task_type} with {service_settings}.
POST
/_inference/rerank/{inference_id}
Rerank documents against a query · Perform reranking inference on the service
#
Ask an LLM
“Can I rerank a list of candidate documents by relevance to a query?”
“Is there a way to only get back the top N results after reranking?”
Tell an agent
Rerank {input} against the query {query} with rerank endpoint {inference_id}.
Rerank these documents for {query} using {inference_id} and keep only the top {top_n}.
POST
/_inference/sparse_embedding/{inference_id}
Generate sparse embeddings for text · Perform sparse embedding inference on the service
#
Ask an LLM
“How do I get sparse token-weight vectors for text, like ELSER produces?”
“Can I generate sparse embeddings for several strings in one request?”
Tell an agent
Generate sparse embeddings for {input} with endpoint {inference_id}.
Expand {input} into sparse token weights using {inference_id}.
POST
/_inference/completion/{inference_id}/_stream
Stream a text completion incrementally · Perform streaming completion inference on the service
#
Ask an LLM
“Can I stream a completion answer token by token instead of waiting for all of it?”
“Does streaming completion work with endpoints of the completion task type only?”
Tell an agent
Stream the completion for {input} from endpoint {inference_id} as it is generated.
Incrementally stream a completion of {input} using {inference_id}.
POST
/_inference/text_embedding/{inference_id}
Generate text embeddings for input · Perform text embedding inference on the service
#
Ask an LLM
“How do I turn text into embedding vectors with a text_embedding endpoint?”
“Can I tell the text embedding endpoint whether my input is a search query or an ingested document?”
Tell an agent
Create text embeddings for {input} with text_embedding endpoint {inference_id}.
Vectorize {input} using {inference_id} with input type {input_type}.
PUT
/_inference/{inference_id}/_update
Update an inference endpoint's settings · Update an inference endpoint
#
Ask an LLM
“Can I rotate the API key stored in an existing inference endpoint?”
“How can I change task settings or the number of allocations on an endpoint without recreating it?”
Tell an agent
Update inference endpoint {inference_id} with new service settings {service_settings}.
Change the task settings of {inference_id} to {task_settings}.
PUT
/_inference/{task_type}/{inference_id}/_update
Update a task-typed inference endpoint · Update an inference endpoint
#
Ask an LLM
“Can I update an inference endpoint while naming its task type in the path?”
“What parts of a text_embedding endpoint can I modify after creation?”
Tell an agent
Update the {task_type} endpoint {inference_id} with service settings {service_settings}.
Modify task settings on {task_type} endpoint {inference_id} to {task_settings}.