Observe.AI Evaluations API

openapi: 3.2.0
info:
  title: Reporting Evaluations API
  description: "Reporting APIs are a collection of RESTful APIs that provide a convenient and secure way for integrating reporting data from Observe.AI into external system of your choice. These APIs are asynchronous and render data in a JSON format. Observe.AI uses OAuth 2.0 protocol for authentication of APIs. Please refer to Authentication section to obtain credentials to access Reporting APIs. Below are the 3 APIs that we support -\n      <ol>\n          <li><a href=\"#tag/Interactions\">Interactions API</a>- To fetch Moments, Transcripts and Interactions(Voice calls, Webchat, Email) metadata</li>\n          <li><a href=\"#tag/Evaluations\">Evaluations API</a> - To fetch Evaluation forms and submitted Evaluations along with question-wise scores and aggregate scores.</li>\n          <li><a href=\"#tag/Coachings\"> Coachings API </a> - To fetch Coaching sessions with mapped Evaluations and related Notes.</li>\n          <li><a href=\"#tag/Ack Dispute Flow\"> Ack Dispute API </a> - To fetch Ack/Dispute state changes for Evaluations.</li>\n          <li><a href=\"#tag/Summary\"> Summarization AI API </a> - To fetch GenAI-based Summaries along with Moments, Transcripts and metadata for all Interactions(Voice calls, Webchat).</li>\n      </ol>\n"
  x-logo:
    url: https://cdn.observe.ai/observeaiLogo.png
servers:
- url: https://{base_url}
  description: Generated server url, where `{base_url}` corresponds to the specific cluster's base URL.
tags:
- name: Evaluations
  x-displayName: Evaluations API
  description: "Evaluations API can be used to pull all the Evaluations(Manual and Auto QA)  done on Observe AI platform.\n\nPlease note -\n    <ol>\n      <li>Allow 24 hrs to pass before pulling Evaluations. ie., For Evaluations completed on 14-Jan by 12:00AM PST, they would be available via APIs by 15-Jan by 12:00AM PST.</li>\n      <li>You can pull Evaluations for 90 days at a time i.e., the duration between the end_date and the start_date should not exceed 90 days.</li>\n      <Li>There is no limit as to how far back in the past you want to go for pulling Evaluations. You can go back to the first ever evaluation on the platform unless data retention limits have been set.</li>\n      <li> All the Evaluations that were Submitted, Edited and Deleted in the date-range will be available in the API response.</li>\n        <ul>\n          <li>Lets take an example to explain it further - when you pull Evaluations for 15 days(say for start_date - 15-Jan and end_date 30-Jan) on 15-Feb via an API call, you get all Evaluations that were -</li>\n            <ul>\n              <li>submitted between 15-jan and 30-jan with \"status\": \"SUBMITTED\"</li>\n              <li>submitted before 15-jan but modified/deleted between 15-jan and 30-jan with \"status\"=\"EDITED\" / \"status\"=\"DELETED\"</li>\n              <li>submitted between 15-jan and 30-jan but modified after 30-jan until the pull date(15-Feb) with \"status\"=\"EDITED\"</li>\n            </ul>\n        </ul>\n      <li>The S3 URL obtained in the API response (with the evaluations) is valid for 24 hours.\n                                </li>\n      <li>If you have been using Basic Auth for the APIs, we request you to switch to OAuth2.0 based Auth for the APIs by 31-Dec-2022. Detailed instructions to create API Credentials and make the switch to OAuth are documented <a href=\"#tag/Authentication\">here</a>. Please reach out to your CSM for more details.</li>\n    </ol>\n"
paths:
  /v1/data/reports/evaluations:
    post:
      tags:
      - Evaluations
      operationId: Create Evaluation Request
      parameters:
      - $ref: '#/components/parameters/Page'
      - $ref: '#/components/parameters/Size'
      - $ref: '#/components/parameters/EvaluationType'
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/RawJobRequest'
      responses:
        '200':
          description: Ok
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/PostJobResponse'
              examples:
                '0':
                  value: '{"request_id":"3058bc09-f9fc-4586-bc7c-498b95a63ec2","start_date":"2022-03-03T00:00:12.000+00:00","end_date":"2022-04-04T00:16:00.000+00:00","status":"INPROGRESS"}'
        '400':
          description: Bad Request
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"error_code":"bad_request","error_description":"similar job already exists with request id 3058bc09-b974-4468-ae1f-f7b9b28067f1"}'
        '401':
          description: Unauthorized
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"error_code": "auth_token_error", "error_description": "invalid auth token"}'
        '403':
          description: Access Denied
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"error_code":"access_denied","error_description":"unauthorized_user"}'
        '404':
          description: Request Not found
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"timestamp":"2022-05-12T06:27:39.587+00:00","status":404,"error":"Not Found","message":"No message available","path":"/v1/data/reports/evaluation"}'
        '405':
          description: Method Not Allowed
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"timestamp": "2022-05-17T04:21:37.669+00:00", "status": 405, "error": "Method Not Allowed", "message": "Request method ''GET'' not supported", "path": "/v1/data/reports/evaluations"}'
        '429':
          description: Rate Limit Exceeded
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/RateLimitMessage'
              examples:
                '0':
                  value: '{"message": "API rate limit exceeded"}'
        '500':
          description: Internal Server Error
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
  /v1/data/reports/evaluations/{request_id}:
    get:
      tags:
      - Evaluations
      operationId: Get Evaluations
      parameters:
      - name: request_id
        in: path
        required: true
        schema:
          type: string
      responses:
        '200':
          description: "\nReport field Description:\n<table>\n  <tr>\n    <td>Param Name</td>\n    <td>Type</td>\n    <td>Description</td>\n  </tr>\n  <tr>\n    <td>evaluation_forms</td>\n    <td>List</td>\n    <td>List of evaluation forms.</td>\n  </tr>\n  <tr>\n    <td>evaluation_forms.template</td>\n    <td>Object</td>\n    <td>Template of each evaluation form</td>\n  </tr>\n  <tr>\n    <td>template.id</td>\n    <td>String</td>\n    <td>ID of the template/evaluation form.</td>\n  </tr>\n  <tr>\n    <td>template.name</td>\n    <td>String</td>\n    <td>Name of the template/evaluation form.</td>\n  </tr>\n  <tr>\n    <td>template.max_score</td>\n    <td>double</td>\n    <td>Max score of the template/evaluation form.</td>\n  </tr>\n  <tr>\n    <td>template.sections</td>\n    <td>List</td>\n    <td>List of sections for each template/evaluation form</td>\n  </tr>\n  <tr>\n    <td>sections.section_id</td>\n    <td>String</td>\n    <td>Section id.</td>\n  </tr>\n  <tr>\n    <td>sections.section_name</td>\n    <td>String</td>\n    <td>Section name.</td>\n  </tr>\n  <tr>\n    <td>sections.max_score</td>\n    <td>double</td>\n    <td>Max score for a section.</td>\n  </tr>\n  <tr>\n    <td>sections.questions</td>\n    <td>List</td>\n    <td>List of questions for each section.</td>\n  </tr>\n  <tr>\n    <td>questions.id</td>\n    <td>String</td>\n    <td>ID of the question.</td>\n  </tr>\n  <tr>\n    <td>questions.max_score</td>\n    <td>double</td>\n    <td>Max score of the question.</td>\n  </tr>\n  <tr>\n    <td>questions.phrase</td>\n    <td>String</td>\n    <td>Question description.</td>\n  </tr>\n  <tr>\n    <td>evaluation_forms.evaluations</td>\n    <td>List</td>\n    <td>List of evaluations for each evaluation form.</td>\n  </tr>\n  <tr>\n    <td>evaluations.channel_meta</td>\n    <td>Object</td>\n    <td>Channel metadata.</td>\n  </tr>\n  <tr>\n    <td>evaluations.evaluation_list</td>\n    <td>List</td>\n    <td>List of evaluations for a template.</td>\n  </tr>\n  <tr>\n    <td>evaluation_list.evaluation_meta</td>\n    <td>Object</td>\n    <td>Evaluation metadata.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.id</td>\n    <td>String</td>\n    <td>ID of the evaluation.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.evaluation_type</td>\n    <td>String</td>\n    <td>Type of evaluation (Manual/Auto QA)</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.evaluator_id</td>\n    <td>String</td>\n    <td>ID of the evaluator. \"NA\" for Auto QA evaluations.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.evaluator_name</td>\n    <td>String</td>\n    <td>Name of the evaluator. \"Observe.AI\" for Auto QA evaluations.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.partner_evaluator_id</td>\n    <td>String</td>\n    <td>ID of the evaluator in client's system . \"NA\" for Auto QA evaluations.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.evaluator_email </td>\n    <td>String</td>\n    <td>Email ID of the evaluator. \"NA\" for Auto QA evaluations.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.evaluator_status</td>\n    <td>String</td>\n    <td>Evaluator status on the system: ACTIVE or IN_ACTIVE. \"NA\" for Auto QA evaluations.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.evaluation_purpose</td>\n    <td>String</td>\n    <td>Evaluation Purpose on the system: \n    <ul>\n    <li>AGENT_PERFORMANCE - Evaluation done for the purpose of grading Agent performance</li>\n    <li>CALIBRATION -  Evaluation completed by Quality Analysts for Calibration purpose</li>\n    <li>FINAL_CALIBRATION - Final Evaluation typically done by Quality Manager or evaluators group based on discussion for Calibration purpose</li>\n    <li>null - evaluation_purpose is applicable only for evaluations with evaluation_type= Manual.</li>\n    </ul>\n    </td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.agent_id</td>\n    <td>String</td>\n    <td>ID of the agent.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.partner_agent_id</td>\n    <td>String</td>\n    <td>ID of the agent in client system.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.ack_status</td>\n    <td>String</td>\n    <td>\n      Captures current status of Evaluation form in the Acknowledgement-Dispute workflow\n      <ul>\n      <li>NOT_INITIATED - Evaluation form is submitted without initiating for Acknowledgement</li>\n      <li>AWAITING_ACK - Agent has been notified of Evaluation. Acknowledgement pending from Agent</li>\n      <li>ACKED - Agent Acknowledged the Evaluation</li>\n      <li>IN_DISPUTE - Dispute raised(typically by Agent or Supervisor) on Evaluation form and awaiting resolution</li>\n      <li>DISPUTE_ACCEPTED - Dispute has been accepted by resolver(typically QA Analyst/QA Manager)</li>\n      <li>DISPUTE_PARTIAL - Dispute has been partially accepted by resolver(typically QA Analyst/QA Manager)</li>\n      <li>DISPUTE_REJECTED - Dispute has been rejected  by resolver(typically QA Analyst/QA Manager)</li>\n      </ul>\n    </td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.ack_init_time</td>\n    <td>Date</td>\n    <td>Acknowledgement Initiated timestamp</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.ack_time</td>\n    <td>Date</td>\n    <td>Acknowledgement confirmation timestamp</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.dispute_raised_at</td>\n    <td>Date</td>\n    <td>Disputed initiated timestamp</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.dispute_raised_by_email</td>\n    <td>String</td>\n    <td>Email ID of the User(typically Agent/Supervisor) who raises dispute</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.dispute_resolved_at</td>\n    <td>Date</td>\n    <td>Disputed resolved timestamp</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.ack_info.dispute_resolved_by_email</td>\n    <td>String</td>\n    <td>Email ID of the User(typically QA Analyst/QA Manager) who resolves the dispute</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.agent_name</td>\n    <td>String</td>\n    <td>Name of the agent.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.agent_email</td>\n    <td>String</td>\n    <td>Email ID of the agent</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.agent_status</td>\n    <td>String</td>\n    <td>Agent status on the system: ACTIVE or IN_ACTIVE.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.created_at</td>\n    <td>Date</td>\n    <td>Evaluation creation time.</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.modified_at</td>\n    <td>Date</td>\n    <td>Evaluation modification time</td>\n  </tr>\n  <tr>\n    <td>evaluation_meta.status</td>\n    <td>String</td>\n    <td>Status of the evaluation - whether submitted, edited, or deleted.</td>\n  </tr>\n  <tr>\n    <td>evaluation_list.response</td>\n    <td>Object</td>\n    <td>Response object for a evaluation.</td>\n  </tr>\n  <tr>\n    <td>response.comment</td>\n    <td>String</td>\n    <td>Comment</td>\n  </tr>\n  <tr>\n    <td>response.sections</td>\n    <td>List</td>\n    <td>List of response sections.</td>\n  </tr>\n  <tr>\n    <td>response.sections.id</td>\n    <td>String</td>\n    <td>Section ID</td>\n  </tr>\n  <tr>\n    <td>response.sections.score_obtained</td>\n    <td>double</td>\n    <td>Score obtained for the section.</td>\n  </tr>\n  <tr>\n    <td>response.sections.questions</td>\n    <td>List</td>\n    <td>List of questions for each section.</td>\n  </tr>\n  <tr>\n    <td>response.sections.questions.id</td>\n    <td>String</td>\n    <td>Question ID.</td>\n  </tr>\n  <tr>\n    <td>response.sections.questions.score_obtained</td>\n    <td>double</td>\n    <td>Score obtained for each question.</td>\n  </tr>\n  <tr>\n    <td>response.sections.questions.comment</td>\n    <td>String</td>\n    <td>Comment for a question response.</td>\n  </tr>\n  <tr>\n    <td>response.sections.questions.fail</td>\n    <td>boolean</td>\n    <td>If the agent has failed that question or not.</td>\n  </tr>\n  <tr>\n    <td>response.sections.questions.value</td>\n    <td>String</td>\n    <td>Answer value for the question</td>\n  </tr>\n  <tr>\n    <td>evaluation_list.scores_obtained</td>\n    <td>Object</td>\n    <td>Score object for each evaluation</td>\n  </tr>\n   <tr>\n    <td>scores_obtained.final_score</td>\n    <td>double</td>\n    <td>Final score for the evaluation</td>\n  </tr>\n  <tr>\n    <td>scores_obtained.total_points</td>\n    <td>double</td>\n    <td>Total points for the evaluation</td>\n  </tr>\n  <tr>\n    <td>scores_obtained.percent_score</td>\n    <td>double</td>\n    <td>Percent score for the evaluation</td>\n  </tr>\n  <tr>\n    <td>scores_obtained.total_bonus</td>\n    <td>double</td>\n    <td>Total bonus for the evaluation</td>\n  </tr>\n  <tr>\n    <td>scores_obtained.total_penalties</td>\n    <td>double</td>\n    <td>Total penalties for the evaluation</td>\n  </tr>\n  <tr>\n    <td>scores_obtained.grade_assigned</td>\n    <td>String</td>\n    <td>Assigned grade</td>\n  </tr>\n  <tr>\n    <td>scores_obtained.auto_fail</td>\n    <td>boolean</td>\n    <td>Is auto fail</td>\n  </tr>\n</table>\n"
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/GetJobResponse'
              examples:
                '0':
                  value: '{"start_date":"2022-03-03T00:00:12.000+00:00","end_date":"2022-04-04T00:14:00.000+00:00","status":"COMPLETED","url":"https://e2e-data-out.s3.amazonaws.com/reports/interactions/3058bc09-6d4d-4f22-9f69-6992f3520e56.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220510T045319Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=YOUR_AWS_ACCESS_KEY_ID%2FYYYYMMDD%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=YOUR_AWS_SIGNATURE_PLACEHOLDER"}'
        '400':
          description: Bad Request
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
        '401':
          description: Unauthorized
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"error_code": "auth_token_error", "error_description": "invalid auth token"}'
        '403':
          description: Access Denied
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"error_code":"access_denied","error_description":"unauthorized_user"}'
        '404':
          description: Not Found
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"error_code":"resource_not_found","error_description":"jobId 3058bc09-da96-490c-824f-f42646160f0a of type EVALUATION not found or does not belong to the account"}'
                '1':
                  value: '{"timestamp": "2022-05-17T04:23:16.247+00:00", "status": 404, "error": "Not Found", "message": "No message available", "path": "/v1/data/reports/evaluation/3058bc09-1a58-44f3-9674-b5be9768d313"}'
        '405':
          description: Method Not Allowed
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
              examples:
                '0':
                  value: '{"timestamp": "2022-05-17T04:21:37.669+00:00", "status": 405, "error": "Method Not Allowed", "message": "Request method ''POST'' not supported", "path": "/v1/data/reports/evaluations/3058bc09-da96-490c-824f-f42646160f0a"}'
        '429':
          description: Rate Limit Exceeded
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/RateLimitMessage'
              examples:
                '0':
                  value: '{"message": "API rate limit exceeded"}'
        '500':
          description: Internal Server Error
          content:
            '*/*':
              schema:
                $ref: '#/components/schemas/ErrorMessage'
components:
  parameters:
    Page:
      in: query
      name: page
      schema:
        type: integer
        minimum: 1
        example: 1
        default: 1
      description: The page number of the paginated response.
    Size:
      in: query
      name: size
      schema:
        type: integer
        minimum: 10
        maximum: 1000
        example: 1000
        default: 1000
      description: Number of evaluations required in a page.
    EvaluationType:
      in: query
      name: evaluationType
      schema:
        type: enum
        default: MANUAL
        example: AUTOQA
      description: Choose from MANUAL, AUTOQA or ALL types of evaluation forms. All manual evaluations are fetched if evaluationType is not specified.
  schemas:
    RawJobRequest:
      required:
      - end_date
      - start_date
      type: object
      properties:
        start_date:
          type: string
          example: '2022-03-03T00:00:12.000+00:00'
          description: The date should be given in the ISO format.
        end_date:
          type: string
          example: '2022-04-04T00:14:00.000+00:00'
          description: The date should be given in the ISO format.
        format:
          type: string
          example: JSON
          enum:
          - JSON
    PostJobResponse:
      type: object
      properties:
        request_id:
          type: string
        start_date:
          type: string
          format: date-time
        end_date:
          type: string
          format: date-time
        status:
          type: string
          enum:
          - QUEUED
          - INPROGRESS
          - COMPLETED
          - EXPIRED
          - FAILED
    RateLimitMessage:
      type: object
      properties:
        message:
          type: string
          description: API rate limit exceeded
    GetJobResponse:
      type: object
      properties:
        start_date:
          type: string
          format: date-time
        end_date:
          type: string
          format: date-time
        status:
          type: string
          enum:
          - QUEUED
          - INPROGRESS
          - COMPLETED
          - EXPIRED
          - FAILED
        url:
          type: string
    ErrorMessage:
      type: object
      properties:
        error_code:
          type: string
        error_description:
          type: string
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT
    DsrDeleteRequest:
      type: object
      required:
      - entity_type
      - rules
      properties:
        entity_type:
          type: string
          description: Entity type to delete
          enum:
          - AUDIO_TRANSCRIPT
          - AUDIO
          - TRANSCRIPT
          - SCREEN_RECORDING
        rules:
          type: array
          items:
            $ref: '#/components/schemas/DsrRule'
    DsrRule:
      type: object
      required:
      - type
      - name
      properties:
        type:
          type: string
          enum:
          - OAI_METADATA
          - CUSTOMER_METADATA
        name:
          type: string
          description: For OAI_METADATA use DURATION, ENTITYTYPE, or STATUS
        values:
          type: array
          items:
            type: string
    DsrDeleteResponse:
      type: object
      properties:
        job_id:
          type: string
        status:
          type: string
          enum:
          - QUEUED
          - CREATED
          - PROGRESS
          - COMPLETED
          - STOPPED
          - FAILED
        message:
          type: string
        requested_at:
          type: string
          format: date-time
        expected_completion_by:
          type: string
          format: date-time
    DsrStatusResponse:
      type: object
      properties:
        request_id:
          type: string
        status:
          type: string
          enum:
          - QUEUED
          - CREATED
          - PROGRESS
          - COMPLETED
          - STOPPED
          - FAILED
        status_message:
          type: string
x-tagGroups:
- name: Reporting APIs
  tags:
  - ReportingService-Overview
  - Authentication
  - Interactions
  - Summary
  - Evaluations
  - Coachings
  - Ack Dispute Flow
  - CallsReportVsInteractions
  - ReleaseNotes
- name: DSR APIs
  tags:
  - DSR-Overview
  - DSR
  - DSR Release Notes
- name: Bulk Export
  tags:
  - Bulk-Export-Overview
  - Bulk-Export-Data-Definitions