openapi: 3.1.0
info:
title: Cognite 3D Asset Mapping Transformations API
description: "# Introduction\nThis is the reference documentation for the Cognite API with\nan overview of all the available methods.\n\n# Postman\nSelect the **Download** button to download our OpenAPI specification to get started.\n\nTo import your data into Postman, select **Import**, and the Import modal opens.\nYou can import items by dragging or dropping files or folders. You can choose how to import your API and manage the import settings in **View Import Settings**.\n\nIn the Import Settings, set the **Folder organization** to **Tags**, select\n**Enable optional parameters** to turn off the settings, and select **Always inherit authentication** to turn on the settings. Select **Import**.\n\nSet the Authorization to **Oauth2.0**. By default, the settings are for Open Industrial Data. Navigate to [Cognite Hub](https://hub.cognite.com/open-industrial-data-211) to understand how to get the credentials for use in Postman.\n\nFor more information, see [Getting Started with Postman](https://developer.cognite.com/dev/guides/postman/).\n\n# Pagination\nMost resource types can be paginated, indicated by the field `nextCursor` in the response.\nBy passing the value of `nextCursor` as the cursor you will get the next page of `limit` results.\nNote that all parameters except `cursor` has to stay the same.\n\n# Parallel retrieval\nAs general guidance, Parallel Retrieval is a technique that should be used when due to query complexity, retrieval of data in a single request is significantly slower than it would otherwise be for a simple request. Parallel retrieval does not act as a speed multiplier on optimally running queries. By parallelizing such requests, data retrieval performance can be tuned to meet the client application needs. \n\nCDF supports parallel retrieval through the `partition` parameter, which has the format `m/n` where `n` is the amount of partitions you would like to split the entire data set into.\nIf you want to download the entire data set by splitting it into 10 partitions, do the following in parallel with `m` running from 1 to 10:\n - Make a request to `/events` with `partition=m/10`.\n - Paginate through the response by following the cursor as explained above. Note that the `partition` parameter needs to be passed to all subqueries.\n\nProcessing of parallel retrieval requests is subject to concurrency quota availability. The request returns the `429` response upon exceeding concurrency limits. See the Request throttling chapter below.\n\nTo prevent unexpected problems and to maximize read throughput, you should at most use 10 partitions. \nSome CDF resources will automatically enforce a maximum of 10 partitions.\nFor more specific and detailed information, please read the ```partition``` attribute documentation for the CDF resource you're using. \n\n# Requests throttling\nCognite Data Fusion (CDF) returns the HTTP `429` (too many requests) response status code when project capacity exceeds the limit.\n\nThe throttling can happen:\n - If a user or a project sends too many (more than allocated) concurrent requests.\n - If a user or a project sends a too high (more than allocated) rate of requests in a given amount of time.\n\nCognite recommends using a retry strategy based on truncated exponential backoff to handle sessions with HTTP response codes 429.\n\nCognite recommends using a reasonable number (up to 10) of `Parallel retrieval` partitions.\n\nFollowing these strategies lets you slow down the request frequency to maximize productivity without having to re-submit/retry failing requests.\n\nSee more [here](https://docs.cognite.com/dev/concepts/resource_throttling).\n\n# API versions\n## Version headers\nThis API uses calendar versioning, and version names follow the `YYYYMMDD` format.\nYou can find the versions currently available by using the version selector at the top of this page.\n\nTo use a specific API version, you can pass the `cdf-version: $version` header along with your requests to the API.\n\n## Beta versions\nThe beta versions provide a preview of what the stable version will look like in the future.\nBeta versions contain functionality that is reasonably mature, and highly likely to become a part of the stable API.\n\nBeta versions are indicated by a `-beta` suffix after the version name. For example, the beta version header for the\n2023-01-01 version is then `cdf-version: 20230101-beta`.\n\n## Alpha versions\nAlpha versions contain functionality that is new and experimental, and not guaranteed to ever become a part of the stable API.\nThis functionality presents no guarantee of service, so its use is subject to caution.\n\nAlpha versions are indicated by an `-alpha` suffix after the version name. For example, the alpha version header for\nthe 2023-01-01 version is then `cdf-version: 20230101-alpha`."
version: v1
contact:
name: Cognite Support
url: https://support.cognite.com
email: support@cognite.com
servers:
- url: https://{cluster}.cognitedata.com/api/v1/projects/{project}
description: The URL for the CDF cluster to connect to
variables:
cluster:
enum:
- api
- az-tyo-gp-001
- az-eastus-1
- az-power-no-northeurope
- westeurope-1
- asia-northeast1-1
- gc-dsm-gp-001
default: api
description: The CDF cluster to connect to
project:
default: publicdata
description: The CDF project name.
security:
- oidc-token:
- https://{cluster}.cognitedata.com/.default
- oauth2-client-credentials:
- https://{cluster}.cognitedata.com/.default
- oauth2-open-industrial-data:
- https://api.cognitedata.com/.default
- oauth2-auth-code:
- https://{cluster}.cognitedata.com/.default
tags:
- name: Transformations
description: 'Transformations enable users to use Spark SQL queries to transform
data from the CDF staging area, RAW, into the CDF data model.
### Concurrency limits
The number of concurrent (parallel) jobs are governed by limits. If a request exceeds one of the limits,
the job fails to run. This limitation also applies to scheduled transformations.
The limits for the transformation service are described in the table below. Note that under high load,
some deviation from the limits might occur for short periods as the service scales up.
| Limit | Per project |
|-----------------------|-----------------------|
| Concurrency | 10 parallel jobs |'
paths:
/transformations:
get:
tags:
- Transformations
summary: List transformations
description: '
> **Required capabilities:** `transformationsAcl:READ`
List transformations. Use nextCursor to paginate through the results.'
operationId: getTransformations
parameters:
- name: limit
in: query
description: Limits the number of results to be returned. The maximum is 1000, default limit is 100.
required: false
schema:
type: integer
format: int32
minimum: 1
maximum: 1000
- name: cursor
in: query
description: Cursor for paging through results.
required: false
schema:
type: string
- name: includePublic
in: query
description: Whether public transformations should be included in the results. The default is true.
required: false
schema:
type: boolean
- name: withJobDetails
in: query
description: Whether transformations should contain information about jobs. The default is true.
required: false
schema:
type: boolean
responses:
'200':
description: Paged response with list of transformations.
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithCursor_TransformationRead'
'400':
description: The response for a failed request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'403':
description: The response for a forbidden request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
x-capability: transformationsAcl:READ
post:
tags:
- Transformations
summary: Create transformations
description: '
> **Required capabilities:** `transformationsAcl:WRITE`
Create a maximum of 1000 transformations per request.'
operationId: createTransformations
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/Items_TransformationCreate'
required: true
responses:
'200':
description: Response with list of transformations.
content:
application/json:
schema:
$ref: '#/components/schemas/Items_TransformationRead'
'400':
description: The response for a failed request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'403':
description: The response for a forbidden request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'409':
description: The response for a conflict.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
x-capability: transformationsAcl:WRITE
x-code-samples:
- lang: Python
label: Python SDK
source: "from cognite.client.data_classes import TransformationWrite, TransformationDestination\nfrom cognite.client.data_classes.transformations.common import ViewInfo, EdgeType, DataModelInfo\ntransformations = [\n TransformationWrite(\n external_id=\"transformation1\",\n name=\"transformation1\",\n ignore_null_fields=False,\n destination=TransformationDestination.assets()\n ),\n TransformationWrite(\n external_id=\"transformation2\",\n name=\"transformation2\",\n ignore_null_fields=False,\n destination=TransformationDestination.raw(\"myDatabase\", \"myTable\")\n ),\n TransformationWrite(\n external_id=\"transformation3\",\n name=\"transformation3\",\n ignore_null_fields=False,\n view = ViewInfo(space=\"TypeSpace\", external_id=\"TypeExtId\", version=\"version\"),\n destination=TransformationDestination.nodes(view, \"InstanceSpace\")\n ),\n TransformationWrite(\n external_id=\"transformation4\",\n name=\"transformation4\",\n ignore_null_fields=False,\n view = ViewInfo(space=\"TypeSpace\", external_id=\"TypeExtId\", version=\"version\"),\n destination=TransformationDestination.edges(view, \"InstanceSpace\")\n ),\n TransformationWrite(\n external_id=\"transformation5\",\n name=\"transformation5\",\n ignore_null_fields=False,\n edge_type = EdgeType(space=\"TypeSpace\", external_id=\"TypeExtId\"),\n destination=TransformationDestination.edges(edge_type,\"InstanceSpace\")\n ),\n TransformationWrite(\n external_id=\"transformation6\",\n name=\"transformation6\",\n ignore_null_fields=False,\n data_model = DataModelInfo(space=\"modelSpace\", external_id=\"modelExternalId\",version=\"modelVersion\",destination_type=\"viewExternalId\"),\n destination=TransformationDestination.instances(data_model,\"InstanceSpace\")\n ),\n TransformationWrite(\n external_id=\"transformation7\",\n name=\"transformation7\",\n ignore_null_fields=False,\n data_model = DataModelInfo(space=\"modelSpace\", external_id=\"modelExternalId\",version=\"modelVersion\",destination_type=\"viewExternalId\", destination_relationship_from_type=\"connectionPropertyName\"),\n destination=TransformationDestination.instances(data_model,\"InstanceSpace\")\n ),\n]\nres = client.transformations.create(transformations)\n"
- lang: Java
label: Java SDK
source: "Transformation.Builder builder = Transformation.newBuilder() \n .setName(\"TransformationTestSDK-\") \n .setQuery(\"select * from teste\") \n .setConflictMode(\"upsert\") \n .setIsPublic(true) \n .setExternalId(\"TestSKD-\") \n .setIgnoreNullFields(true) \n .setDataSetId(dataSetId) \n .setDestination(Transformation.Destination.newBuilder() \n .setType(\"raw\") \n .setDatabase(\"Test\") \n .setTable(\"Test\") \n .build() \n ); \nList<Transformation> createdList = client.transformations().upsert(List.of(builder.build())); \n\n"
/transformations/byids:
post:
tags:
- Transformations
summary: Retrieve transformations
description: '
> **Required capabilities:** `transformationsAcl:READ`
Retrieve a maximum of 1000 transformations by ids and externalIds per request.'
operationId: getTransformationsByIds
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithByIdsFlags_CogniteId'
required: true
responses:
'200':
description: Response with list of transformations.
content:
application/json:
schema:
$ref: '#/components/schemas/Items_TransformationRead'
'400':
description: The response for a failed request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'403':
description: The response for a forbidden request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'409':
description: The response for a conflict.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
x-capability: transformationsAcl:READ
x-code-samples:
- lang: Python
label: Python SDK
source: 'res = client.transformations.retrieve(id=1)
res = client.transformations.retrieve(external_id="1")
res = client.transformations.retrieve_multiple(ids=[1,2,3], external_ids=[''transform-1'',''transform-2''])
'
- lang: Java
label: Java SDK
source: "List<Item> byExternalIds = List.of(Item.newBuilder() \n .setExternalId(\"10\") \n .build()); \nclient.transformations().retrieve(byExternalIds);"
/transformations/filter:
post:
tags:
- Transformations
summary: Filter transformations
description: '
> **Required capabilities:** `transformationsAcl:READ`
Filter transformations. Use nextCursor to paginate through the results.'
operationId: filterTransformations
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/TransformationAdvancedList'
required: true
responses:
'200':
description: Paged response with list of transformations.
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithCursor_TransformationRead'
'400':
description: The response for a failed request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'403':
description: The response for a forbidden request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
x-capability: transformationsAcl:READ
x-code-samples:
- lang: Python
label: Python SDK
source: 'transformations_list = client.transformations.list()
'
- lang: Java
label: Java SDK
source: "List<Transformation> listTransformationResults = new ArrayList<>(); \nclient.transformations()\n .list(Request.create().withFilterParameter(\"isPublic\", true))\n .forEachRemaining(listTransformationResults::addAll); \n\n"
/transformations/delete:
post:
tags:
- Transformations
summary: Delete transformations
description: '
> **Required capabilities:** `transformationsAcl:WRITE`
Delete a maximum of 1000 transformations by ids and externalIds per request.'
operationId: deleteTransformations
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithIgnoreUnknownIds_CogniteId'
required: true
responses:
'200':
description: Empty response.
content:
application/json:
schema:
type: object
'400':
description: The response for a failed request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of objects that are not unique.
'403':
description: The response for a forbidden request.
content:
application/json:
schema:
required:
- error
type: object
properties:
error:
required:
- code
- message
type: object
properties:
code:
type: integer
description: HTTP status code.
message:
type: string
description: Error message.
missing:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteInternalId'
description: List of lookup objects that do not match any results.
duplicated:
type: array
items:
oneOf:
- $ref: '#/components/schemas/TransformationCogniteExternalId'
- $ref: '#/components/schemas/TransformationCogniteI
# --- truncated at 32 KB (82 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/cognite/refs/heads/main/openapi/cognite-transformations-api-openapi.yml