The webhook contract for the Guardrail feature in kgateway, Agentgateway Enterprise, and Gloo Gateway. Two endpoints — /request and /response — intercept prompts on the way to a large language model and responses on the way back, and return a Pass, Mask, or Reject action so operators can implement their own content filtering and guardrails across LLM providers.
openapi: 3.1.0
info:
title: GuardRail Webhook API
description: "\nThis API specification defines the webhook endpoints for the \
\ Guardrail feature for AI use cases in kgateway, agentgateway enterprise, and Gloo Gateway. \
\ The Guardrail feature provides a way to intercept\
\ and process both requests to and responses from Large Language Models (LLMs).\
\ This way, you can implement your own advanced guardrails and content filtering.\n\
\nThe Guardrail feature consists of two main webhook endpoints:\n\n1. `/request`\
\ - Processes request prompts before they are sent to the LLM\n2. `/response`\
\ - Processes responses from the LLM before they are sent back to the user\n\n\
Each endpoint supports different actions:\n\n* `PassAction`: Allow the content\
\ to pass through unchanged\n* `MaskAction`: Modify the content by masking sensitive\
\ information\n* `RejectAction`: Block the content and return an error response\n\
\nThe API is designed to work with various LLM providers by normalizing their\
\ different request and response formats into a consistent schema.\n "
version: 0.1.0
paths:
/request:
post:
tags:
- Webhooks
summary: Process Prompts
description: 'This webhook intercepts requests from the user before they are
sent to the LLM. You can use it to:
- Validate and filter content
- Mask sensitive information
- Reject requests based on policy rules
The webhook receives normalized prompt messages regardless of the original
LLM provider''s format.
It can return one of three actions:
1. `PassAction`: Allow the request to proceed unchanged
2. `MaskAction`: Return modified prompts with sensitive information masked
3. `RejectAction`: Block the request with a specified HTTP status code and
message'
operationId: process_prompts_request_post
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GuardrailsPromptRequest'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/GuardrailsPromptResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/response:
post:
tags:
- Webhooks
summary: Process Responses
description: "This webhook intercepts responses from the LLM before they are\
\ returned to the user. The `role` and `content` are extracted from the response\
\ into the `ResponseChoices` JSON object, regardless of the API format from\
\ various providers.\n\nFor streaming responses from the LLM, this webhook\
\ is called multiple times for a single response. The gateway buffers and\
\ detects the semantic boundary of the content before making the webhook call.\n\
\nTwo types of responses are possible by returning one of the following JSON\
\ objects:\n\n1. `PassAction`: \n - Indicates that no action is taken for\
\ the response\n - The response is allowed to be sent to the user unchanged\n\
\n2. `MaskAction`: \n - Indicates that some information is masked in the\
\ response\n - The `ResponseChoices` JSON object can be modified in place\n\
\ - The modified object should be sent back in the body field of the response\n\
\ - The number of choices inside `ResponseChoices` MUST be the same as in\
\ the request\n - If content needs to be deleted, set an empty content field"
operationId: process_responses_response_post
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GuardrailsResponseRequest'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/GuardrailsResponseResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
components:
schemas:
GuardrailsPromptRequest:
properties:
body:
$ref: '#/components/schemas/PromptMessages'
description: The body object is a list of the Message JSON objects from
the prompts in the request.
type: object
required:
- body
title: GuardrailsPromptRequest
description: GuardrailsPromptRequest is the request model for the Guardrails
prompt API.
GuardrailsPromptResponse:
properties:
action:
anyOf:
- $ref: '#/components/schemas/PassAction'
- $ref: '#/components/schemas/MaskAction'
- $ref: '#/components/schemas/RejectAction'
title: Action
description: "\n The action to be taken based on the request. The\
\ following actions are available:\n\n 1. PassAction: \n \
\ - No action is required\n - The request proceeds unchanged\
\ to the LLM\n\n 2. MaskAction: \n - Some content in\
\ the request needs to be masked\n - The modified request is\
\ returned in the body field\n - The number of messages must\
\ match the original request\n\n 3. RejectAction: \n \
\ - The request should be rejected\n - A specific HTTP status\
\ code and message are returned\n - The request will not be\
\ sent to the LLM\n "
type: object
required:
- action
title: GuardrailsPromptResponse
description: GuardrailsPromptResponse is the response model for the Guardrails
prompt API.
GuardrailsResponseRequest:
properties:
body:
$ref: '#/components/schemas/ResponseChoices-Input'
description: The body object is a list of the Choice JSON objects that have
the response content from the LLM.
type: object
required:
- body
title: GuardrailsResponseRequest
description: GuardrailsResponseRequest is the request model for the Guardrails
response API.
GuardrailsResponseResponse:
properties:
action:
anyOf:
- $ref: '#/components/schemas/PassAction'
- $ref: '#/components/schemas/MaskAction'
title: Action
description: "\n The action to be taken on the response. The following\
\ actions are available:\n\n 1. PassAction: \n - No action\
\ is required\n - The response proceeds unchanged to the user\n\
\n 2. MaskAction: \n - Some content in the response needs\
\ to be masked\n - The modified response is returned in the\
\ body field\n - The number of choices must match the original\
\ response\n "
type: object
required:
- action
title: GuardrailsResponseResponse
description: GuardrailsResponseResponse is the response model for the Guardrails
response API.
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
MaskAction:
properties:
body:
anyOf:
- $ref: '#/components/schemas/PromptMessages'
- $ref: '#/components/schemas/ResponseChoices-Output'
title: Body
description: The body has the modified messages that masked out some of
the original content. When used in a GuardrailPromptResponse, this should
be PromptMessages. When used in GuardrailResponseResponse, this should
be ResponseChoices.
reason:
anyOf:
- type: string
- type: 'null'
title: Reason
description: The reason is a human readable string that explains the reason
for the action.
type: object
required:
- body
title: MaskAction
description: 'The response model for the Mask action, which indicates the message
has been modified.
This can be used in GuardrailsPromptResponse or GuardrailsResponseResponse
when responding to a GuardrailsPromptRequest or a GuardrailsResponseRequest
respectively.'
Message:
properties:
role:
type: string
title: Role
description: 'The role of the message sender in the conversation. Common
values include:
- ''system'': System instructions or context
- ''user'': Messages from the end user
- ''assistant'': Responses from the AI assistant'
examples:
- system
- user
- assistant
content:
type: string
title: Content
description: The actual text content of the message. Depending on the role,
the content can be a question, instruction, or response.
examples:
- You are a helpful AI assistant.
- What is the capital of France?
- The capital of France is Paris.
type: object
required:
- role
- content
title: Message
description: 'A single message in a conversation with an LLM.
Each message has a role (who sent it) and content (what was sent).'
PassAction:
properties:
reason:
anyOf:
- type: string
- type: 'null'
title: Reason
description: The reason is a human readable string that explains the reason
for the action.
type: object
title: PassAction
description: The response model for the Pass action, which indicates no modification
is done to the messages.
PromptMessages:
properties:
messages:
items:
$ref: '#/components/schemas/Message'
type: array
title: Messages
description: A sequence of messages that form a conversation or prompt.
The order of messages matters, because it represents the conversation
history.
examples:
- - content: You are a helpful AI assistant that provides concise answers.
role: system
- content: What is 2+2?
role: user
- content: 2+2 is 4.
role: assistant
- content: What about 3+3?
role: user
type: object
title: PromptMessages
description: 'A complete conversation or prompt to be sent to an LLM.
The `messages` array represents the conversation history in chronological
order.'
RejectAction:
properties:
body:
type: string
title: Body
description: The rejection message that to be used for the HTTP error response
body.
status_code:
type: integer
title: Status Code
description: The HTTP status code to be returned in the HTTP error response.
reason:
anyOf:
- type: string
- type: 'null'
title: Reason
description: The reason is a human readable string that explains the reason
for the action.
type: object
required:
- body
- status_code
title: RejectAction
description: 'The response model for the Reject action, which indicates the
request should be rejected.
This action causes a HTTP error response to be sent back to the end user.'
ResponseChoice:
properties:
message:
$ref: '#/components/schemas/Message'
description: The AI assistant's response to the user's prompt. The `role`
is typically 'assistant' and the `content` has the response text.
examples:
- content: The sum of 2 and 2 is 4.
role: assistant
type: object
required:
- message
title: ResponseChoice
description: 'Represents a single possible response from the LLM.
Some LLMs might provide multiple alternative responses.'
ResponseChoices-Input:
properties:
choices:
items:
$ref: '#/components/schemas/ResponseChoice'
type: array
title: Choices
description: A list of possible responses from the LLM. Some models might
provide multiple alternative responses.
examples:
- - message:
content: The sum of 2 and 2 is 4.
role: assistant
- message:
content: When you add 2 to 2, you get 4.
role: assistant
type: object
title: ResponseChoices
description: 'Contains all possible responses from the LLM for a given prompt.
The `choices` array might contain one or more alternative responses.'
ResponseChoices-Output:
properties:
choices:
items:
$ref: '#/components/schemas/ResponseChoice'
type: array
title: Choices
description: A list of possible responses from the LLM. Some models might
provide multiple alternative responses.
examples:
- - message:
content: The sum of 2 and 2 is 4.
role: assistant
- message:
content: When you add 2 to 2, you get 4.
role: assistant
type: object
title: ResponseChoices
description: 'Contains all possible responses from the LLM for a given prompt.
The `choices` array might contain one or more alternative responses.'
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError