Scale AI Evaluation Groups API

openapi: 3.2.0
info:
  title: EGP API V5 Evaluation Groups API
  description: This is the parent API for all EGP APIs. If you are looking for the EGP API, please go to https://api.egp.scale.com/docs.
  contact:
    name: Scale Generative AI Platform
    url: https://scale.com/genai-platform
  version: 0.1.0
servers:
- url: https://api.egp.scale.com
security: []
tags:
- name: Evaluation Groups
paths:
  /v5/evaluation-groups/{group_id}:
    delete:
      tags:
      - Evaluation Groups
      summary: Archive Evaluation Group
      description: 'Archive (soft-delete) an evaluation group and its dependent records.


        This is a soft delete: the group is marked deleted rather than removed, so it no

        longer appears in default list/get results but can still be fetched with the

        include-deleted flags. The same delete timestamp cascades as a soft delete to the

        group''s members, row identifiers, and its dashboard charts and their chart columns;

        the member evaluations themselves are not affected. The archived group is returned in

        the response.'
      operationId: DELETE-V5-/v5/evaluation-groups/group_id
      parameters:
      - name: group_id
        in: path
        required: true
        schema:
          title: Group Id
          type: string
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationGroup'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/archive-evaluation-group
    get:
      tags:
      - Evaluation Groups
      summary: Get Evaluation Group
      description: 'Fetch a single evaluation group by its ID.


        By default only non-archived groups are returned; set include_deleted to also resolve

        a soft-deleted group. The views parameter optionally expands the response with the

        group''s members and/or row_identifiers, and when members are loaded they are enriched

        with each member evaluation''s name, tags, and creation time. Use the schema endpoint

        instead when you need the per-evaluation column schemas of the group''s members.'
      operationId: GET-V5-/v5/evaluation-groups/group_id
      parameters:
      - name: group_id
        in: path
        required: true
        schema:
          title: Group Id
          type: string
      - name: views
        in: query
        required: false
        schema:
          type: array
          items:
            $ref: '#/components/schemas/EvaluationGroupViews'
          description: 'Optional relationships to include: ''members'', ''row_identifiers'''
          default: []
          title: Views
        description: 'Optional relationships to include: ''members'', ''row_identifiers'''
      - name: include_deleted
        in: query
        required: false
        schema:
          type: boolean
          default: false
          title: Include Deleted
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationGroup'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/get-evaluation-group
    patch:
      tags:
      - Evaluation Groups
      summary: Patch Evaluation Group
      description: 'Partially update an evaluation group''s attributes.


        Only the fields present in the request are changed; unset fields are left untouched.

        Unlike the PUT endpoint, metadata is merged into the existing metadata key-by-key

        rather than replaced wholesale, and group membership (evaluation_ids) cannot be

        changed here -- use PUT to replace members. At least one field must be supplied or the

        request fails with a 400. Supplying row_identifiers replaces the group''s entire row

        identifier set (an evaluation_id-to-column-name mapping for cross-dataset joins);

        passing an empty mapping clears it.'
      operationId: PATCH-V5-/v5/evaluation-groups/group_id
      parameters:
      - name: group_id
        in: path
        required: true
        schema:
          title: Group Id
          type: string
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/EvaluationGroupPatchRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationGroup'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/patch-evaluation-group
    put:
      tags:
      - Evaluation Groups
      summary: Update Evaluation Group
      description: 'Replace an evaluation group''s attributes and, optionally, its membership.


        This is a full replacement of the group''s attributes: every attribute field (name,

        description, tags, metadata) is written from the request, so omitting an optional

        field clears it rather than leaving it unchanged. Use PATCH instead to update only

        selected fields and to merge (rather than overwrite) metadata. Membership is replaced

        only when evaluation_ids is supplied: the resulting members become exactly that set

        (added/removed by diff against the current members), it must contain at least one ID

        with no duplicates, and every ID is validated for existence and account access or the

        request fails with a 400. When membership actually changes, a best-effort Temporal

        workflow is started to recompute the group''s dashboard widgets; this is fire-and-forget

        and its failure does not fail the request. Removing members hard-deletes the

        corresponding chart columns and any charts left with no columns.'
      operationId: PUT-V5-/v5/evaluation-groups/group_id
      parameters:
      - name: group_id
        in: path
        required: true
        schema:
          title: Group Id
          type: string
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/EvaluationGroupUpdateRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationGroup'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/update-evaluation-group
  /v5/evaluation-groups:
    post:
      tags:
      - Evaluation Groups
      summary: Create Evaluation Group
      description: 'Create a new evaluation group bundling one or more existing evaluations.


        The request must list at least one evaluation ID, and duplicate IDs are rejected.

        Every listed evaluation is validated for existence and account access before the

        group is created; an unknown or inaccessible evaluation ID fails the whole request

        with a 400 rather than creating a partial group. The named evaluations are added as

        group members in the same transaction, and any supplied row_identifiers (an

        evaluation_id-to-column-name mapping used for cross-dataset joins) are persisted

        alongside them, ignoring entries for evaluation IDs not in the group. The returned

        group includes its members enriched with each evaluation''s name, tags, and creation

        time. Unlike the PUT and PATCH update endpoints, creation does not trigger any

        dashboard-widget recomputation.'
      operationId: POST-V5-/v5/evaluation-groups
      parameters:
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/EvaluationGroupCreateRequest'
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationGroup'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/create-evaluation-group
    get:
      tags:
      - Evaluation Groups
      summary: List Evaluation Groups
      description: 'List the calling account''s evaluation groups with optional filters.


        Results are paginated and scoped to the authenticated account. The optional filters

        combine with AND semantics: name does a case-insensitive partial (substring) match on

        the group name, tags returns only groups whose tag list contains all of the supplied

        tags, and evaluation_id returns only groups that have the given evaluation as an

        active member. By default archived (soft-deleted) groups are excluded; set

        include_deleted to include them. The views parameter optionally expands each group

        with its members and/or row_identifiers, and returned members are enriched with each

        evaluation''s name, tags, and creation time.'
      operationId: GET-V5-/v5/evaluation-groups
      parameters:
      - name: include_deleted
        in: query
        required: false
        schema:
          type: boolean
          default: false
          title: Include Deleted
      - name: tags
        in: query
        required: false
        schema:
          anyOf:
          - items:
              type: string
            type: array
          - type: 'null'
          title: Tags
      - name: name
        in: query
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Name
      - name: evaluation_id
        in: query
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          description: Filter to groups containing this evaluation ID
          title: Evaluation Id
        description: Filter to groups containing this evaluation ID
      - name: views
        in: query
        required: false
        schema:
          type: array
          items:
            $ref: '#/components/schemas/EvaluationGroupViews'
          description: 'Optional relationships to include: ''members'', ''row_identifiers'''
          default: []
          title: Views
        description: 'Optional relationships to include: ''members'', ''row_identifiers'''
      - name: starting_after
        in: query
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Starting After
      - name: ending_before
        in: query
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Ending Before
      - name: limit
        in: query
        required: false
        schema:
          type: integer
          maximum: 10000
          minimum: 1
          default: 100
          title: Limit
      - name: sort_by
        in: query
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Sort By
      - name: sort_order
        in: query
        required: false
        schema:
          anyOf:
          - $ref: '#/components/schemas/SortOrder'
          - type: 'null'
          default: asc
          title: Sort Order
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/PaginatedList_EvaluationGroup_'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/list-evaluation-groups
  /v5/evaluation-groups/{group_id}/schema:
    get:
      tags:
      - Evaluation Groups
      summary: Get Evaluation Group Schema
      description: 'Return a separate column schema for each active member evaluation of the group.


        Rather than a single merged schema, the response holds one schema entry per active

        member evaluation (each with its field list, total item count, and sampling info),

        which lets a caller filter columns down to a chosen subset of the group''s evaluations.

        Schemas are computed from the member evaluations'' items; include_archived controls

        whether archived items are counted in that analysis. A group with no active members

        returns an empty schema list. This differs from the plain get endpoint, which returns

        group metadata and members but not their column schemas.'
      operationId: GET-V5-/v5/evaluation-groups/group_id/schema
      parameters:
      - name: group_id
        in: path
        required: true
        schema:
          title: Group Id
          type: string
      - name: include_archived
        in: query
        required: false
        schema:
          type: boolean
          description: Include archived items in schema analysis
          default: false
          title: Include Archived
        description: Include archived items in schema analysis
      - name: x-selected-account-id
        in: header
        required: false
        schema:
          anyOf:
          - type: string
          - type: 'null'
          title: Account ID Header
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/EvaluationGroupSchema'
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'
      security:
      - APIKeyHeader: []
      x-source: https://docs.gp.scale.com/reference/v5/evaluation-groups/get-evaluation-group-schema
components:
  schemas:
    EvaluationGroupViews:
      type: string
      enum:
      - members
      - row_identifiers
      title: EvaluationGroupViews
      description: Views for evaluation group responses
    EvaluationGroupCreateRequest:
      properties:
        tags:
          title: Tags
          description: The tags associated with the entity
          items:
            type: string
          type: array
        name:
          type: string
          maxLength: 128
          title: Name
          description: Name of the evaluation group
        description:
          title: Description
          description: Optional description
          type: string
        metadata:
          title: Metadata
          description: Optional metadata key-value pairs
          example:
            project: Q4-2025
            team: ml-eval
          additionalProperties: true
          type: object
        evaluation_ids:
          items:
            type: string
          type: array
          minItems: 1
          title: Evaluation Ids
          description: List of evaluation IDs to include in the group
        row_identifiers:
          title: Row Identifiers
          description: Optional mapping of evaluation_id to column name for cross-dataset joins
          example:
            eval-123: user_id
            eval-456: customer_id
          additionalProperties:
            type: string
          type: object
      type: object
      required:
      - name
      - evaluation_ids
      title: EvaluationGroupCreateRequest
      description: Request model for creating an evaluation group
    FieldSchema:
      properties:
        object:
          type: string
          const: field_schema
          title: Object
          default: field_schema
        field_name:
          type: string
          title: Field Name
          description: The flattened JSON key path (e.g., 'metadata.category')
        data_type:
          type: string
          title: Data Type
          description: 'JSON type: ''string'', ''number'', ''boolean'', ''object'', ''array'', or ''null'''
        item_count:
          type: integer
          minimum: 0
          title: Item Count
          description: Number of evaluation items containing this field
        source:
          type: string
          enum:
          - data
          - task_result_cache
          title: Source
          description: 'The source of the field: ''data'' or ''task_result_cache'''
      type: object
      required:
      - field_name
      - data_type
      - item_count
      - source
      description: Schema information for a single field in evaluation item data
    EvaluationGroupMember:
      properties:
        id:
          type: string
          title: Id
          description: Unique identifier of the member record
        object:
          type: string
          const: evaluation_group.member
          title: Object
          default: evaluation_group.member
        evaluation_group_id:
          type: string
          title: Evaluation Group Id
          description: ID of the evaluation group
        evaluation_id:
          type: string
          title: Evaluation Id
          description: ID of the evaluation
        evaluation_name:
          title: Evaluation Name
          description: Name of the evaluation
          type: string
        evaluation_tags:
          title: Evaluation Tags
          description: Tags of the evaluation
          items:
            type: string
          type: array
        evaluation_created_at:
          title: Evaluation Created At
          description: When the evaluation was created
          type: string
          format: date-time
        created_at:
          type: string
          format: date-time
          title: Created At
          description: When this member was added to the group
        deleted_at:
          title: Deleted At
          description: When this membership was soft-deleted (if applicable)
          type: string
          format: date-time
      type: object
      required:
      - id
      - evaluation_group_id
      - evaluation_id
      - created_at
      description: Response model for evaluation group member
    PaginatedList_EvaluationGroup_:
      properties:
        object:
          type: string
          const: list
          title: Object
          default: list
        items:
          items:
            $ref: '#/components/schemas/EvaluationGroup'
          type: array
          title: Items
        limit:
          type: integer
          title: Limit
          description: The maximum number of items to return.
          default: 100
        total:
          type: integer
          title: Total
          description: The total of items that match the query. This is greater than or equal to the number of items returned.
        has_more:
          type: boolean
          title: Has More
          description: Whether there are more items left to be fetched.
      type: object
      required:
      - items
      - total
      - has_more
    EvaluationGroupPatchRequest:
      properties:
        tags:
          title: Tags
          description: The tags associated with the entity
          items:
            type: string
          type: array
        name:
          type: string
          maxLength: 128
          title: Name
          description: Name of the evaluation group
        description:
          title: Description
          description: Optional description
          type: string
        metadata:
          title: Metadata
          description: Optional metadata key-value pairs
          additionalProperties: true
          type: object
        row_identifiers:
          title: Row Identifiers
          description: Optional mapping of evaluation_id to column name for cross-dataset joins
          additionalProperties:
            type: string
          type: object
      type: object
      title: EvaluationGroupPatchRequest
      description: Request model for partial update of evaluation group attributes (name, description, tags, metadata, row_identifiers) (PATCH)
    EvaluationGroupRowIdentifier:
      properties:
        object:
          type: string
          const: evaluation_group.row_identifier
          title: Object
          default: evaluation_group.row_identifier
        evaluation_id:
          type: string
          title: Evaluation Id
          description: ID of the evaluation
        column_name:
          type: string
          title: Column Name
          description: Name of the column used as row identifier
      type: object
      required:
      - evaluation_id
      - column_name
      description: Response model for evaluation group row identifier
    EvaluationSchema:
      properties:
        object:
          type: string
          const: evaluation_schema
          title: Object
          default: evaluation_schema
        evaluation_id:
          type: string
          title: Evaluation Id
          description: The ID of the evaluation
        total_items:
          type: integer
          minimum: 0
          title: Total Items
          description: Total number of evaluation items
        fields:
          items:
            $ref: '#/components/schemas/FieldSchema'
          type: array
          title: Fields
          description: List of all discovered fields, ordered alphabetically by field_name
        is_sampled:
          type: boolean
          title: Is Sampled
          description: Whether schema was computed from a sample of items (for large evaluations)
          default: false
        sample_size:
          title: Sample Size
          description: Number of items sampled for schema inference, if applicable
          type: integer
          minimum: 0
      type: object
      required:
      - evaluation_id
      - total_items
      - fields
      description: Schema information for an evaluation's item data structure
    EvaluationGroup:
      properties:
        id:
          type: string
          title: Id
          description: Unique identifier of the evaluation group
        object:
          type: string
          const: evaluation_group
          title: Object
          default: evaluation_group
        tags:
          title: Tags
          description: The tags associated with the entity
          items:
            type: string
          type: array
        name:
          type: string
          title: Name
          description: Name of the evaluation group
        description:
          title: Description
          description: Optional description
          type: string
        metadata:
          title: Metadata
          description: Optional metadata key-value pairs
          additionalProperties: true
          type: object
        created_at:
          type: string
          format: date-time
          title: Created At
          description: When the group was created
        deleted_at:
          title: Deleted At
          description: When the group was soft-deleted
          type: string
          format: date-time
        account_id:
          type: string
          title: Account Id
          description: Account that owns this evaluation group
        members:
          title: Members
          description: Evaluation members in this group. Populated with 'members' view.
          items:
            $ref: '#/components/schemas/EvaluationGroupMember'
          type: array
        row_identifiers:
          title: Row Identifiers
          description: Row identifier mappings. Populated with 'row_identifiers' view.
          items:
            $ref: '#/components/schemas/EvaluationGroupRowIdentifier'
          type: array
      type: object
      required:
      - tags
      - id
      - name
      - created_at
      - account_id
      description: Response model for evaluation group
    EvaluationGroupUpdateRequest:
      properties:
        tags:
          title: Tags
          description: The tags associated with the entity
          items:
            type: string
          type: array
        name:
          type: string
          maxLength: 128
          title: Name
          description: Name of the evaluation group
        description:
          title: Description
          description: Optional description
          type: string
        metadata:
          title: Metadata
          description: Optional metadata key-value pairs
          additionalProperties: true
          type: object
        evaluation_ids:
          title: Evaluation Ids
          description: Complete list of evaluation IDs to include in group (replaces existing members)
          items:
            type: string
          type: array
          minItems: 1
        row_identifiers:
          title: Row Identifiers
          description: Optional mapping of evaluation_id to column name for cross-dataset joins
          additionalProperties:
            type: string
          type: object
      type: object
      title: EvaluationGroupUpdateRequest
      description: Request model for full update of evaluation group (metadata and members)
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
            - type: string
            - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          title: Error Type
          type: string
        input:
          title: Input
        ctx:
          type: object
          title: Context
          additionalProperties: true
      type: object
      required:
      - loc
      - msg
      - type
      title: ValidationError
    SortOrder:
      type: string
      enum:
      - asc
      - desc
      title: SortOrder
    EvaluationGroupSchema:
      properties:
        object:
          type: string
          const: evaluation_group_schema
          title: Object
          default: evaluation_group_schema
        evaluation_group_id:
          type: string
          title: Evaluation Group Id
          description: The ID of the evaluation group
        evaluation_schemas:
          items:
            $ref: '#/components/schemas/EvaluationSchema'
          type: array
          title: Evaluation Schemas
          description: Schema for each member evaluation in the group, one entry per active evaluation
      type: object
      required:
      - evaluation_group_id
      - evaluation_schemas
      description: Per-evaluation schemas for all members of an evaluation group
  securitySchemes:
    APIKeyHeader:
      type: apiKey
      in: header
      name: x-api-key
x-assembled-from: Per-operation OpenAPI fragments embedded verbatim in docs.gp.scale.com reference pages; source spec https://api.sgp.scale.com/openapi-versions/v5/openapi.json (404 when assembled)