Solo.io Webhooks API

openapi: 3.2.0
info:
  title: GuardRail Webhook Webhooks API
  description: "\nThis API specification defines the webhook endpoints for the  Guardrail feature for AI use cases in kgateway, agentgateway enterprise, and Gloo Gateway.  The Guardrail feature provides a way to intercept and process both requests to and responses from Large Language Models (LLMs). This way, you can implement your own advanced guardrails and content filtering.\n\nThe Guardrail feature consists of two main webhook endpoints:\n\n1. `/request` - Processes request prompts before they are sent to the LLM\n2. `/response` - Processes responses from the LLM before they are sent back to the user\n\nEach endpoint supports different actions:\n\n* `PassAction`: Allow the content to pass through unchanged\n* `MaskAction`: Modify the content by masking sensitive information\n* `RejectAction`: Block the content and return an error response\n\nThe API is designed to work with various LLM providers by normalizing their different request and response formats into a consistent schema.\n    "
  version: 0.1.0
tags:
- name: Webhooks
paths:
  /request:
    post:
      tags:
      - Webhooks
      summary: Process Prompts
      description: 'This webhook intercepts requests from the user before they are sent to the LLM. You can use it to:

        - Validate and filter content

        - Mask sensitive information

        - Reject requests based on policy rules


        The webhook receives normalized prompt messages regardless of the original LLM provider''s format.

        It can return one of three actions:

        1. `PassAction`: Allow the request to proceed unchanged

        2. `MaskAction`: Return modified prompts with sensitive information masked

        3. `RejectAction`: Block the request with a specified HTTP status code and message'
      operationId: process_prompts_request_post
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/GuardrailsPromptRequest'
        required: true
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/GuardrailsPromptResponse'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
  /response:
    post:
      tags:
      - Webhooks
      summary: Process Responses
      description: "This webhook intercepts responses from the LLM before they are returned to the user. The `role` and `content` are extracted from the response into the `ResponseChoices` JSON object, regardless of the API format from various providers.\n\nFor streaming responses from the LLM, this webhook is called multiple times for a single response. The gateway buffers and detects the semantic boundary of the content before making the webhook call.\n\nTwo types of responses are possible by returning one of the following JSON objects:\n\n1. `PassAction`: \n   - Indicates that no action is taken for the response\n   - The response is allowed to be sent to the user unchanged\n\n2. `MaskAction`: \n   - Indicates that some information is masked in the response\n   - The `ResponseChoices` JSON object can be modified in place\n   - The modified object should be sent back in the body field of the response\n   - The number of choices inside `ResponseChoices` MUST be the same as in the request\n   - If content needs to be deleted, set an empty content field"
      operationId: process_responses_response_post
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/GuardrailsResponseRequest'
        required: true
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/GuardrailsResponseResponse'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
components:
  schemas:
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    Message:
      properties:
        role:
          type: string
          title: Role
          description: 'The role of the message sender in the conversation. Common values include:

            - ''system'': System instructions or context

            - ''user'': Messages from the end user

            - ''assistant'': Responses from the AI assistant'
          examples:
          - system
          - user
          - assistant
        content:
          type: string
          title: Content
          description: The actual text content of the message. Depending on the role, the content can be a question, instruction, or response.
          examples:
          - You are a helpful AI assistant.
          - What is the capital of France?
          - The capital of France is Paris.
      type: object
      required:
      - role
      - content
      title: Message
      description: 'A single message in a conversation with an LLM.

        Each message has a role (who sent it) and content (what was sent).'
    ResponseChoices-Output:
      properties:
        choices:
          items:
            $ref: '#/components/schemas/ResponseChoice'
          type: array
          title: Choices
          description: A list of possible responses from the LLM. Some models might provide multiple alternative responses.
          examples:
          - - message:
                content: The sum of 2 and 2 is 4.
                role: assistant
            - message:
                content: When you add 2 to 2, you get 4.
                role: assistant
      type: object
      title: ResponseChoices
      description: 'Contains all possible responses from the LLM for a given prompt.

        The `choices` array might contain one or more alternative responses.'
    GuardrailsPromptRequest:
      properties:
        body:
          $ref: '#/components/schemas/PromptMessages'
          description: The body object is a list of the Message JSON objects from the prompts in the request.
      type: object
      required:
      - body
      title: GuardrailsPromptRequest
      description: GuardrailsPromptRequest is the request model for the Guardrails prompt API.
    RejectAction:
      properties:
        body:
          type: string
          title: Body
          description: The rejection message that to be used for the HTTP error response body.
        status_code:
          type: integer
          title: Status Code
          description: The HTTP status code to be returned in the HTTP error response.
        reason:
          anyOf:
          - type: string
          - type: 'null'
          title: Reason
          description: The reason is a human readable string that explains the reason for the action.
      type: object
      required:
      - body
      - status_code
      title: RejectAction
      description: 'The response model for the Reject action, which indicates the request should be rejected.

        This action causes a HTTP error response to be sent back to the end user.'
    ResponseChoice:
      properties:
        message:
          $ref: '#/components/schemas/Message'
          description: The AI assistant's response to the user's prompt. The `role` is typically 'assistant' and the `content` has the response text.
          examples:
          - content: The sum of 2 and 2 is 4.
            role: assistant
      type: object
      required:
      - message
      title: ResponseChoice
      description: 'Represents a single possible response from the LLM.

        Some LLMs might provide multiple alternative responses.'
    GuardrailsResponseResponse:
      properties:
        action:
          anyOf:
          - $ref: '#/components/schemas/PassAction'
          - $ref: '#/components/schemas/MaskAction'
          title: Action
          description: "\n        The action to be taken on the response. The following actions are available:\n\n        1. PassAction: \n           - No action is required\n           - The response proceeds unchanged to the user\n\n        2. MaskAction: \n           - Some content in the response needs to be masked\n           - The modified response is returned in the body field\n           - The number of choices must match the original response\n        "
      type: object
      required:
      - action
      title: GuardrailsResponseResponse
      description: GuardrailsResponseResponse is the response model for the Guardrails response API.
    GuardrailsResponseRequest:
      properties:
        body:
          $ref: '#/components/schemas/ResponseChoices-Input'
          description: The body object is a list of the Choice JSON objects that have the response content from the LLM.
      type: object
      required:
      - body
      title: GuardrailsResponseRequest
      description: GuardrailsResponseRequest is the request model for the Guardrails response API.
    PromptMessages:
      properties:
        messages:
          items:
            $ref: '#/components/schemas/Message'
          type: array
          title: Messages
          description: A sequence of messages that form a conversation or prompt. The order of messages matters, because it represents the conversation history.
          examples:
          - - content: You are a helpful AI assistant that provides concise answers.
              role: system
            - content: What is 2+2?
              role: user
            - content: 2+2 is 4.
              role: assistant
            - content: What about 3+3?
              role: user
      type: object
      title: PromptMessages
      description: 'A complete conversation or prompt to be sent to an LLM.

        The `messages` array represents the conversation history in chronological order.'
    ResponseChoices-Input:
      properties:
        choices:
          items:
            $ref: '#/components/schemas/ResponseChoice'
          type: array
          title: Choices
          description: A list of possible responses from the LLM. Some models might provide multiple alternative responses.
          examples:
          - - message:
                content: The sum of 2 and 2 is 4.
                role: assistant
            - message:
                content: When you add 2 to 2, you get 4.
                role: assistant
      type: object
      title: ResponseChoices
      description: 'Contains all possible responses from the LLM for a given prompt.

        The `choices` array might contain one or more alternative responses.'
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
            - type: string
            - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
      type: object
      required:
      - loc
      - msg
      - type
      title: ValidationError
    PassAction:
      properties:
        reason:
          anyOf:
          - type: string
          - type: 'null'
          title: Reason
          description: The reason is a human readable string that explains the reason for the action.
      type: object
      title: PassAction
      description: The response model for the Pass action, which indicates no modification is done to the messages.
    MaskAction:
      properties:
        body:
          anyOf:
          - $ref: '#/components/schemas/PromptMessages'
          - $ref: '#/components/schemas/ResponseChoices-Output'
          title: Body
          description: The body has the modified messages that masked out some of the original content. When used in a GuardrailPromptResponse, this should be PromptMessages. When used in GuardrailResponseResponse, this should be ResponseChoices.
        reason:
          anyOf:
          - type: string
          - type: 'null'
          title: Reason
          description: The reason is a human readable string that explains the reason for the action.
      type: object
      required:
      - body
      title: MaskAction
      description: 'The response model for the Mask action, which indicates the message has been modified.

        This can be used in GuardrailsPromptResponse or GuardrailsResponseResponse when responding to a GuardrailsPromptRequest or a GuardrailsResponseRequest respectively.'
    GuardrailsPromptResponse:
      properties:
        action:
          anyOf:
          - $ref: '#/components/schemas/PassAction'
          - $ref: '#/components/schemas/MaskAction'
          - $ref: '#/components/schemas/RejectAction'
          title: Action
          description: "\n        The action to be taken based on the request. The following actions are available:\n\n        1. PassAction: \n           - No action is required\n           - The request proceeds unchanged to the LLM\n\n        2. MaskAction: \n           - Some content in the request needs to be masked\n           - The modified request is returned in the body field\n           - The number of messages must match the original request\n\n        3. RejectAction: \n           - The request should be rejected\n           - A specific HTTP status code and message are returned\n           - The request will not be sent to the LLM\n        "
      type: object
      required:
      - action
      title: GuardrailsPromptResponse
      description: GuardrailsPromptResponse is the response model for the Guardrails prompt API.