Adobe Suite Batch Ingestion API

openapi: 3.2.0
info:
  title: Batch Ingestion API
  description: "Batch ingestion allows you to ingest data into Adobe Experience Platform as batch files. \nBatches are units of data that consist of one or more files to be ingested as a single \nunit. Once ingested, batches provide metadata that describes the number of records \nsuccessfully ingested, as well as any failed records and associated error messages.\n\nUse the Batch Ingestion API to create batches, upload files, check on upload status, and more.\n\n- **Related Documentation**:\n  - [Data ingestion documentation](http://www.adobe.com/go/data-ingestion-overview-en)\n  - Batch ingestion [prerequisites](https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html#data-ingestion-prerequisites), [best practices](https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html#batch-ingestion-best-practices), and [constraints](https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html#batch-ingestion-constraints)\n  - [Batch ingestion API overview](https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/overview.html)\n  - [Streaming ingestion API reference](https://developer.adobe.com/experience-platform-apis/references/streaming-ingestion)\n\n- **Visualize API calls with Postman (a free, third-party software)**:\n  - [Data Ingestion Service API Postman collection on GitHub](https://github.com/adobe/experience-platform-postman-samples/blob/master/apis/experience-platform/Data%20Ingestion%20API.postman_collection.json)\n  - [Video guide for creating the Postman environment](https://video.tv.adobe.com/v/28832)\n  - [Steps for importing environments and collections in Postman](https://learning.getpostman.com/docs/postman/collection_runs/using_environments_in_collection_runs/)\n\n- **API path**:\n  - Base path for batch ingestion APIs: https://<span>platform.adobe.io/data/foundation/import\n\n- **Required headers**:\n  - All calls require the headers `Authorization`, `x-gw-ims-org-id`,\n  and `x-api-key`. For more information on how to obtain these values, see the\n  [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).\n  - All resources in Experience Platform are isolated to specific virtual sandboxes. \n  All requests to Platform APIs require the header `x-sandbox-name` whose value is \n  the all-lowercase name of the sandbox the operation will take place in (for example,\n  \"prod\\\"). See the [sandboxes overview](https://adobe.com/go/sandbox-overview-en) for \n  more information.\n  - Requests with a payload in the request body (such as POST, PUT, and PATCH calls) \n  may require the header `Content-Type`. Accepted values specific to each call are \n  provided in the call parameters.\n\n- **API error handling**:\n    - Refer to the Experience Platform API troubleshooting guide for [FAQs](https://experienceleague.adobe.com/docs/experience-platform/landing/troubleshooting.html#faq), [API status codes](https://experienceleague.adobe.com/docs/experience-platform/landing/troubleshooting.html#api-status-codes), and [request header errors](https://experienceleague.adobe.com/docs/experience-platform/landing/troubleshooting.html#request-header-errors)."
  version: '1.0'
servers:
- url: //{environment}.adobe.io/data/foundation/import
  variables:
    environment:
      default: platform
      enum:
      - platform
      - platform-stage
tags:
- name: Batch Ingestion
  description: 'Batch ingestion is used to ingest data into Experience Platform as batch files. For example, data being ingested can be the profile data from a flat file in a CRM system (for example: Parquet or JSON) or data that conforms to a known Experience Data Model (XDM) schema within the Schema Registry.'
paths:
  /batches:
    post:
      tags:
      - Batch Ingestion
      summary: Create a new batch
      operationId: createBatch
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: Content-Type
        in: header
        description: The type of content being sent in the body of the request. The value should be 'application/json`.
        required: true
        schema:
          type: string
      requestBody:
        description: The payload that contains information needed to create a batch.
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/IngestBatch'
        required: true
      responses:
        201:
          description: The batch was successfully created.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Batch'
        400:
          description: A bad request. The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The user needs to provide a valid bearer token.
          content: {}
        414:
          description: The URL length exceeds the allowed 2000 characters.
          content: {}
        500:
          description: An internal server error has occurred.
          content: {}
        503:
          description: The service is currently unavailable.
          content: {}
      x-codegen-request-body-name: body
  /batches/{batchId}/datasets/{datasetId}/files/{filePath}:
    put:
      tags:
      - Batch Ingestion
      summary: Upload a small file to a dataset
      description: "This endpoint lets you upload files under datasets for a batch initialized by the bulk \ningestion API. The PUT method creates or updates the entire request stream as the file \nbytes under the path denoted by the `filePath` and is idempotent. Subsequent calls to \nthis endpoint will replace the existing file on the referenced path. \n**Note:** Due to gateway size and timeout limits, there is currently a size threshold of 256MB.\nIf the file is larger than that, you **must** use the large file upload mechanism."
      operationId: uploadSmallFile
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: Content-Type
        in: header
        description: The type of content being sent in the body of the request. The value should be 'application/octet-stream'.
        required: true
        schema:
          type: string
          enum:
          - application/octet-stream
      - name: batchId
        in: path
        description: The ID of the batch that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: datasetId
        in: path
        description: The ID of the dataset that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: filePath
        in: path
        description: The location where the file will be uploaded to, on the Adobe side. **Note:** The directory structure denoted in this path is preserved and can be used to denote partitions that the file data belongs to.
        required: true
        schema:
          type: string
      responses:
        200:
          description: The file was successfully updated.
          content: {}
        201:
          description: The file was successfully created.
          content: {}
        400:
          description: The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The IMS token provided is invalid.
          content: {}
        404:
          description: The batch or dataset resource is not found.
          content: {}
        414:
          description: The URL length exceeds the allowed 2000 characters.
          content: {}
        415:
          description: The uploaded file media type isn't supported.
          content: {}
        503:
          description: The service is unavailable.
          content: {}
    patch:
      tags:
      - Batch Ingestion
      summary: Upload part of a large file to a batch
      description: Files larger than 256MB are uploaded in parts. The PATCH endpoint provides you a way to upload a part of your large file that was initialized via the POST endpoint. File sizes of between 0-512000000 bytes are accepted.
      operationId: uploadLargeFilePart
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: Content-Type
        in: header
        required: true
        description: The type of content being sent in the body of the request. The value should be 'application/octet-stream'.
        schema:
          type: string
          enum:
          - application/octet-stream
      - name: batchId
        in: path
        description: The ID of the batch that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: datasetId
        in: path
        description: The ID of the dataset that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: filePath
        in: path
        description: The location where the file will be uploaded to, on the Adobe side. **Note:** The directory structure denoted in this path is preserved and can be used to denote partitions that the file data belongs to.
        required: true
        schema:
          type: string
      - name: Content-Range
        in: header
        description: The lowest and highest value of bytes in the file being uploaded with this request.
        required: true
        schema:
          type: string
      requestBody:
        content:
          multipart/form-data:
            schema:
              required:
              - file
              properties:
                file:
                  type: string
                  description: The full path and name of the file you are trying to upload. This file path is the local file path with the file format suffix. For example, `@{FILE_PATH_AND_NAME}.parquet`. **Note**, as the 'application/octet-stream' content type is a binary file that contains data in the form of bytes instead of text, no example request body is provided here.
                  format: binary
        required: true
      responses:
        200:
          description: The file was successfully uploaded.
          headers:
            Range:
              description: The accepted range of content bytes is 0 - 512000000.
              schema:
                type: string
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/File'
        400:
          description: The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The IMS token provided is invalid.
          content: {}
        404:
          description: The batch, dataset, or file resource is not found.
          content: {}
        414:
          description: The URL length exceeds the allowed 2000 characters.
          content: {}
        415:
          description: The uploaded file media type isn't supported.
          content: {}
        500:
          description: An internal server error has occurred.
          content: {}
        503:
          description: The service is unavailable.
          content: {}
    post:
      tags:
      - Batch Ingestion
      summary: Either initialize or finish uploading a large file
      description: This endpoint should only be used when uploading a large file to the batch. Large file upload should be used for files larger than 256MB.
      operationId: completeLargeFileUpload
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: Content-Type
        in: header
        description: The type of content being sent in the body of the request. The value should be 'application/json'.
        required: true
        schema:
          type: string
      - name: batchId
        in: path
        description: The ID of the batch that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: datasetId
        in: path
        description: The ID of the dataset that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: filePath
        in: path
        description: The location of the file you want to initialize or complete within the Adobe data lake. **Note:** The directory structure denoted in this path is preserved and can be used to denote partitions that the file data belongs to.
        required: true
        schema:
          type: string
      - name: action
        in: query
        description: The action to perform on the file. The currently supported values are `INITIALIZE` and `COMPLETE`. To begin initializing the file for upload, use the value `INITIALIZE`. To mark the file as finished, use the value `COMPLETE`.
        required: true
        schema:
          type: string
          enum:
          - INITIALIZE
          - COMPLETE
      responses:
        200:
          description: The file is now finished uploading.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/File'
        400:
          description: The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The IMS token provided is invalid.
          content: {}
        404:
          description: The batch, dataset, or file resource is not found.
          content: {}
        414:
          description: The URL length exceeds the allowed 2000 characters.
          content: {}
        500:
          description: An internal server error has occurred.
          content: {}
        503:
          description: The service is unavailable.
          content: {}
    head:
      tags:
      - Batch Ingestion
      summary: Get the current status of the large file
      description: Files larger than 512MB are uploaded in parts. The HEAD endpoint provides a way to get the byte range information received by the server so far.
      operationId: getLargeFileStatus
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: batchId
        in: path
        description: The ID of the batch that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: datasetId
        in: path
        description: The ID of the dataset that the data is being uploaded to.
        required: true
        schema:
          type: string
      - name: filePath
        in: path
        description: The location of the file within the Adobe data lake that you want to retrieve status information for. **Note:** The directory structure denoted in this path is preserved and can be used to denote partitions that the file data belongs to.
        required: true
        schema:
          type: string
      responses:
        200:
          description: The status information was successfully returned.
          headers:
            Range:
              description: The accepted range of content bytes is 0 - 512000000.
              schema:
                type: string
          content: {}
        400:
          description: The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The IMS token provided is invalid.
          content: {}
        404:
          description: The batch, dataset, or file resource is not found.
          content: {}
        414:
          description: URL length exceeds the allowed 2000 characters.
          content: {}
        415:
          description: The uploaded file media type isn't supported.
          content: {}
        500:
          description: An internal server error has occurred.
          content: {}
        503:
          description: The service is unavailable.
          content: {}
  /batches/{batchId}/datasets/{datasetId}/preview:
    get:
      tags:
      - Batch Ingestion
      summary: Retrieve a preview for a batch
      description: This endpoint generates a data preview for the files uploaded to the batch so far. The preview can be generated either collectively for all the batch datasets, or for only selected datasets.
      operationId: retrievePreview
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: batchId
        in: path
        description: The ID of the batch that you are trying to preview.
        required: true
        schema:
          type: string
      - name: datasetId
        in: path
        description: The ID of the dataset that you are trying to preview.
        required: true
        schema:
          type: string
      - name: format
        in: query
        description: The file format for the uploaded file.
        required: true
        schema:
          type: string
      - name: isMultiLineJson
        in: query
        description: This flag indicates whether the files to be ingested are in a multi-line json format.
        schema:
          type: boolean
          default: false
      - name: delimiter
        in: query
        description: The character used to specify boundaries between separate strings when parsing column values.
        schema:
          type: string
          default: ','
      - name: quote
        in: query
        description: The character used to quote a value when parsing data.
        schema:
          type: string
          default: '"'
      - name: escape
        in: query
        description: The character used to escape a reserved character when parsing data.
        schema:
          type: string
          default: \
      - name: charset
        in: query
        description: The character encoding used for parsing data.
        schema:
          type: string
          default: UTF-8
      - name: header
        in: query
        description: The flag to indicate if the header is supplied in the dataset files.
        schema:
          type: boolean
          default: true
      - name: nrow
        in: query
        description: The number of rows to parse.
        schema:
          type: integer
          format: int32
          default: 0
      responses:
        200:
          description: The preview for the batch was successfully generated.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/PreviewResponse'
        400:
          description: The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The IMS token provided is invalid.
          content: {}
        404:
          description: The batch or dataset resource is not found.
          content: {}
        414:
          description: The URL length exceeds the allowed 2000 characters.
          content: {}
        415:
          description: The uploaded file media type isn't supported.
          content: {}
        500:
          description: An internal server error has occurred.
          content: {}
        503:
          description: The service is unavailable.
          content: {}
  /batches/{batchId}:
    post:
      tags:
      - Batch Ingestion
      summary: Complete a batch
      description: After you have finished uploading all of the different parts of the file, signal that the batch ingestion is complete to trigger the downstream data promotion workflow.
      operationId: completeBatch
      parameters:
      - name: Authorization
        in: header
        description: The access token which can be copied from your Experience Platform integration, prefixed with "Bearer ". For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-api-key
        in: header
        description: The API key which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-gw-ims-org-id
        in: header
        description: The Organization ID which can be copied from your Experience Platform integration. For more information on how to obtain this value, visit the [authentication tutorial](http://www.adobe.com/go/platform-api-authentication-en).
        required: true
        schema:
          type: string
      - name: x-sandbox-name
        in: header
        description: The name of the sandbox in which the operation will take place. See the [sandboxes overview documentation](https://adobe.com/go/sandbox-overview-en) for more information.
        required: true
        schema:
          type: string
      - name: batchId
        in: path
        description: The ID of the batch that you want to perform the signal action on.
        required: true
        schema:
          type: string
      - name: action
        in: query
        description: The action to take on the batch. To signal that the batch file ingestion is complete, use the 'COMPLETE' action query parameter.
        required: true
        schema:
          type: string
          enum:
          - COMPLETE
          - ABORT
          - FAIL
          - REVERT
      responses:
        200:
          description: The batch has been successfully promoted.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Batch'
        400:
          description: The request is malformed.
          content: {}
        401:
          description: Access is unauthorized. The IMS token provided is invalid.
          content: {}
        404:
          description: The batch is not found.
          content: {}
        414:
          description: The URL length exceeds the allowed 2000 characters.
          content: {}
        500:
          description: An internal server error has occurred.
          content: {}
        503:
          description: The service is unavailable.
          content: {}
components:
  schemas:
    errors:
      type: object
      properties:
        code:
          type: string
          description: An enumerated code that identifies the type of error encountered.
          example: 405
        rows:
          type: array
          description: Indicates which rows exhibit this error. This could be a single number or a hyphen-delimited range.
          items:
            type: string
            description: The row or range of rows that the error was encountered on.
            example: 13-26
        description:
          type: string
          description: A human-readable description of the error.
          example: HTTP 405 Method Not Allowed
    Batch:
      type: object
      properties:
        id:
          type: string
          description: The ID of the batch.
          example: 01EQ1C48TR4GD4FE7NH2522TCQ
        imsOrg:
          type: string
          description: The ID of your organization.
          example: 5C1328435BF324E90A49402A@AdobeOrg
        created:
          type: integer
          description: The Unix timestamp in **milliseconds** when this batch was created.
          example: '2022-11-09T18:45:31.256Z'
          readOnly: true
        createdClient:
          type: string
          description: The ID of your client that created this batch.
          example: acme_foundation_push
          readOnly: true
        createdUser:
          type: string
          description: The ID of the user who created this object.
          example: acme_foundation_dataTracker@AdobeID
          readOnly: true
        updatedUser:
          type: string
          description: The ID of the user who last changed this object.
          example: acme_foundation_dataTracker@AdobeID
          readOnly: true
        updated:
          type: integer
          description: The Unix timestamp in **milliseconds** of the last update.
          example: '2022-11-09T18:45:31.256Z'
          readOnly: true
        started:
          type: integer
          description: The Unix timestamp in **milliseconds** when the batch processing action was started.
          example: '2022-11-09T18:45:31.256Z'
        completed:
          type: integer
          description: The Unix timestamp in **milliseconds** when the batch processing action was completed. To get the total processing time, subtract the "Started" time from the "Completed" time.
          example: '2022-12-09T18:45:31.256Z'
        status:
          type: string
          description: The current status of this batch.
          example: active
          enum:
          - processing
          - act

# --- truncated at 32 KB (43 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/adobe-suite/refs/heads/main/openapi/adobe-suite-batch-ingestion-api-openapi.yml