A hosted extractor writes to a **destination**. The destination contains credentials for CDF, and additional information about where the data should land, such as data set ID. Multiple jobs can share a single destination, in which case requests will be combined, reducing the number of requests made to CDF APIs. Metrics are still reported for each individual job.
openapi: 3.1.0
info:
title: Cognite 3D Asset Mapping Destinations API
description: "# Introduction\nThis is the reference documentation for the Cognite API with\nan overview of all the available methods.\n\n# Postman\nSelect the **Download** button to download our OpenAPI specification to get started.\n\nTo import your data into Postman, select **Import**, and the Import modal opens.\nYou can import items by dragging or dropping files or folders. You can choose how to import your API and manage the import settings in **View Import Settings**.\n\nIn the Import Settings, set the **Folder organization** to **Tags**, select\n**Enable optional parameters** to turn off the settings, and select **Always inherit authentication** to turn on the settings. Select **Import**.\n\nSet the Authorization to **Oauth2.0**. By default, the settings are for Open Industrial Data. Navigate to [Cognite Hub](https://hub.cognite.com/open-industrial-data-211) to understand how to get the credentials for use in Postman.\n\nFor more information, see [Getting Started with Postman](https://developer.cognite.com/dev/guides/postman/).\n\n# Pagination\nMost resource types can be paginated, indicated by the field `nextCursor` in the response.\nBy passing the value of `nextCursor` as the cursor you will get the next page of `limit` results.\nNote that all parameters except `cursor` has to stay the same.\n\n# Parallel retrieval\nAs general guidance, Parallel Retrieval is a technique that should be used when due to query complexity, retrieval of data in a single request is significantly slower than it would otherwise be for a simple request. Parallel retrieval does not act as a speed multiplier on optimally running queries. By parallelizing such requests, data retrieval performance can be tuned to meet the client application needs. \n\nCDF supports parallel retrieval through the `partition` parameter, which has the format `m/n` where `n` is the amount of partitions you would like to split the entire data set into.\nIf you want to download the entire data set by splitting it into 10 partitions, do the following in parallel with `m` running from 1 to 10:\n - Make a request to `/events` with `partition=m/10`.\n - Paginate through the response by following the cursor as explained above. Note that the `partition` parameter needs to be passed to all subqueries.\n\nProcessing of parallel retrieval requests is subject to concurrency quota availability. The request returns the `429` response upon exceeding concurrency limits. See the Request throttling chapter below.\n\nTo prevent unexpected problems and to maximize read throughput, you should at most use 10 partitions. \nSome CDF resources will automatically enforce a maximum of 10 partitions.\nFor more specific and detailed information, please read the ```partition``` attribute documentation for the CDF resource you're using. \n\n# Requests throttling\nCognite Data Fusion (CDF) returns the HTTP `429` (too many requests) response status code when project capacity exceeds the limit.\n\nThe throttling can happen:\n - If a user or a project sends too many (more than allocated) concurrent requests.\n - If a user or a project sends a too high (more than allocated) rate of requests in a given amount of time.\n\nCognite recommends using a retry strategy based on truncated exponential backoff to handle sessions with HTTP response codes 429.\n\nCognite recommends using a reasonable number (up to 10) of `Parallel retrieval` partitions.\n\nFollowing these strategies lets you slow down the request frequency to maximize productivity without having to re-submit/retry failing requests.\n\nSee more [here](https://docs.cognite.com/dev/concepts/resource_throttling).\n\n# API versions\n## Version headers\nThis API uses calendar versioning, and version names follow the `YYYYMMDD` format.\nYou can find the versions currently available by using the version selector at the top of this page.\n\nTo use a specific API version, you can pass the `cdf-version: $version` header along with your requests to the API.\n\n## Beta versions\nThe beta versions provide a preview of what the stable version will look like in the future.\nBeta versions contain functionality that is reasonably mature, and highly likely to become a part of the stable API.\n\nBeta versions are indicated by a `-beta` suffix after the version name. For example, the beta version header for the\n2023-01-01 version is then `cdf-version: 20230101-beta`.\n\n## Alpha versions\nAlpha versions contain functionality that is new and experimental, and not guaranteed to ever become a part of the stable API.\nThis functionality presents no guarantee of service, so its use is subject to caution.\n\nAlpha versions are indicated by an `-alpha` suffix after the version name. For example, the alpha version header for\nthe 2023-01-01 version is then `cdf-version: 20230101-alpha`."
version: v1
contact:
name: Cognite Support
url: https://support.cognite.com
email: support@cognite.com
servers:
- url: https://{cluster}.cognitedata.com/api/v1/projects/{project}
description: The URL for the CDF cluster to connect to
variables:
cluster:
enum:
- api
- az-tyo-gp-001
- az-eastus-1
- az-power-no-northeurope
- westeurope-1
- asia-northeast1-1
- gc-dsm-gp-001
default: api
description: The CDF cluster to connect to
project:
default: publicdata
description: The CDF project name.
security:
- oidc-token:
- https://{cluster}.cognitedata.com/.default
- oauth2-client-credentials:
- https://{cluster}.cognitedata.com/.default
- oauth2-open-industrial-data:
- https://api.cognitedata.com/.default
- oauth2-auth-code:
- https://{cluster}.cognitedata.com/.default
tags:
- name: Destinations
description: 'A hosted extractor writes to a **destination**. The destination contains credentials for CDF, and additional information about where the data should land, such as data set ID.
Multiple jobs can share a single destination, in which case requests will be combined, reducing the number of requests made to CDF APIs. Metrics are still reported for each individual job.'
paths:
/hostedextractors/destinations:
get:
tags:
- Destinations
operationId: list_destinations
x-capability:
- hostedExtractors:READ
parameters:
- $ref: '#/components/parameters/100Limit'
- $ref: '#/components/parameters/Cursor'
description: '
> **Required capabilities:** `hostedExtractors:READ`
List all destinations in a given project. If more than `limit` destinations exist, a cursor for pagination will be returned with the response.'
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithCursor_Destination_'
description: List of destinations and an optional cursor.
'400':
$ref: '#/components/responses/400ErrorResponse'
'422':
$ref: '#/components/responses/ErrorResponse'
description: Validation Error
summary: List Destinations
x-code-samples:
- lang: Python
label: Python SDK
source: "destination_list = client.hosted_extractors.destinations.list(limit=5)\n\nfor destination in client.hosted_extractors.destinations:\n destination # do something with the destination\n\nfor destination_list in client.hosted_extractors.destinations(chunk_size=25):\n destination_list # do something with the destinationss\n"
post:
tags:
- Destinations
operationId: create_destinations
x-capability:
- hostedExtractors:WRITE
description: '
> **Required capabilities:** `hostedExtractors:WRITE`
Create up to 100 destinations.'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/Items_CreateDestination_'
required: true
description: Destinations to create.
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/Items_Destination_'
description: List of created destinations
'400':
$ref: '#/components/responses/400ErrorResponse'
description: Response for a failed request
'422':
$ref: '#/components/responses/ValidationError'
summary: Create Destinations
x-code-samples:
- lang: Python
label: Python SDK
source: 'from cognite.client.data_classes.hosted_extractors import DestinationWrite, SessionWrite
destination = DestinationWrite(external_id=''my_dest'', credentials=SessionWrite("my_nonce"), target_data_set_id=123)
res = client.hosted_extractors.destinations.create(destination)
'
/hostedextractors/destinations/byIds:
post:
tags:
- Destinations
operationId: retrieve_destinations
x-capability:
- hostedExtractors:READ
description: '
> **Required capabilities:** `hostedExtractors:READ`
Retrieve a list of up to 100 destinations by their external ID, optionally ignoring unknown IDs.'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithIgnoreUnknownIds_ExternalId_'
required: true
description: List of external IDs of destinations to retrieve.
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/Items_Destination_'
description: List of retrieved destinations.
'400':
$ref: '#/components/responses/400ErrorResponse'
'422':
$ref: '#/components/responses/ValidationError'
summary: Retrieve Destinations
x-code-samples:
- lang: Python
label: Python SDK
source: 'res = client.hosted_extractors.destinations.retrieve(''myDestination'')
res = client.hosted_extractors.destinations.retrieve(["myDestination", "myDestination2"], ignore_unknown_ids=True)
'
/hostedextractors/destinations/delete:
post:
tags:
- Destinations
operationId: delete_destinations
x-capability:
- hostedExtractors:WRITE
description: '
> **Required capabilities:** `hostedExtractors:WRITE`
Delete a list of destinations by their external ID. This operation will fail if any destination in the request is associated with a job, unless the `force` query parameter is set to `true`.'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ItemsWithIgnoreUnknownIdsAndForce_ExternalId_'
required: true
description: List of external IDs of destinations to delete.
responses:
'200':
$ref: '#/components/responses/EmptyResponse'
description: Empty response
'400':
$ref: '#/components/responses/400ErrorResponse'
'422':
$ref: '#/components/responses/ValidationError'
summary: Delete Destinations
x-code-samples:
- lang: Python
label: Python SDK
source: 'client.hosted_extractors.destinations.delete(["myDest", "MyDest2"])
'
/hostedextractors/destinations/update:
post:
tags:
- Destinations
operationId: update_destinations
x-capability:
- hostedExtractors:WRITE
description: '
> **Required capabilities:** `hostedExtractors:WRITE`
Update up to 100 destinations.'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/Items_DestinationUpdateItem_'
required: true
description: Destinations to update.
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/Items_Destination_'
description: List of updated destinations.
'400':
$ref: '#/components/responses/400ErrorResponse'
'422':
$ref: '#/components/responses/ValidationError'
summary: Update Destinations
x-code-samples:
- lang: Python
label: Python SDK
source: 'from cognite.client.data_classes.hosted_extractors import DestinationUpdate
destination = DestinationUpdate(''my_dest'').target_data_set_id.set(123)
res = client.hosted_extractors.destinations.update(destination)
'
components:
schemas:
UpdateItem_SessionCredentials_:
additionalProperties: false
properties:
set:
$ref: '#/components/schemas/SessionCredentials'
required:
- set
title: UpdateItem_SessionCredentials
type: object
description: Set a new set of session credentials for writing to CDF.
Destination:
additionalProperties: false
properties:
externalId:
$ref: '#/components/schemas/CogniteExternalId'
sessionId:
type: integer
title: Session ID
minimum: 1
maximum: 9007199254740991
format: int64
description: ID of the session tied to this destination.
targetDataSetId:
type: integer
title: Data Set ID
minimum: 1
maximum: 9007199254740991
format: int64
description: Data set ID the created items are inserted into, if applicable.
createdTime:
$ref: '#/components/schemas/EpochTimestamp'
lastUpdatedTime:
$ref: '#/components/schemas/EpochTimestamp'
required:
- externalId
- lastUpdatedTime
- createdTime
title: Destination
type: object
CogniteExternalId:
description: The external ID provided by the client. Must be unique for the resource type.
type: string
maxLength: 255
example: my.known.id
ItemsWithIgnoreUnknownIdsAndForce_ExternalId_:
additionalProperties: false
properties:
items:
items:
$ref: '#/components/schemas/ExternalIdWrapper'
title: Items
type: array
maxItems: 100
minItems: 1
ignoreUnknownIds:
type: boolean
title: IgnoreUnknownIds
description: Ignore IDs and external IDs that are not found.
force:
type: boolean
title: Force
description: Delete any jobs associated with each item.
required:
- items
title: ItemsWithIgnoreUnknownIdsAndForce_ExternalId
type: object
ExternalIdWrapper:
additionalProperties: false
properties:
externalId:
$ref: '#/components/schemas/CogniteExternalId'
required:
- externalId
title: ExternalIdWrapper
type: object
Error:
type: object
required:
- code
- message
description: Cognite API error.
properties:
code:
type: integer
description: HTTP status code.
format: int32
example: 401
message:
type: string
description: Error message.
example: Could not authenticate.
missing:
type: array
description: List of lookup objects that do not match any results.
items:
type: object
additionalProperties: true
duplicated:
type: array
description: List of objects that are not unique.
items:
type: object
additionalProperties: true
UpdateDestination:
additionalProperties: false
properties:
credentials:
anyOf:
- $ref: '#/components/schemas/UpdateSetNull'
- $ref: '#/components/schemas/UpdateItem_SessionCredentials_'
title: Credentials
description: Set a new session, or remove the value.
targetDataSetId:
anyOf:
- $ref: '#/components/schemas/UpdateSetNull'
- $ref: '#/components/schemas/UpdateItem_DataSetId_'
title: Dataset Id
description: Set a new value for the dataset ID, or remove the value.
title: UpdateDestination
type: object
Items_DestinationUpdateItem_:
additionalProperties: false
properties:
items:
items:
$ref: '#/components/schemas/DestinationUpdateItem'
title: Items
type: array
maxItems: 10
minItems: 1
required:
- items
title: Items[DestinationUpdateItem]
type: object
EpochTimestamp:
description: The number of milliseconds since 00:00:00 Thursday, 1 January 1970, Coordinated Universal Time (UTC), minus leap seconds.
type: integer
minimum: 0
format: int64
example: 1730204346000
Items_Destination_:
additionalProperties: false
properties:
items:
items:
$ref: '#/components/schemas/Destination'
title: Items
type: array
maxItems: 100
minItems: 0
required:
- items
title: DestinationList
type: object
UpdateSetNull:
additionalProperties: false
properties:
setNull:
title: SetNull
type: boolean
description: Clear this value
required:
- setNull
title: UpdateSetNull
type: object
ItemsWithCursor_Destination_:
additionalProperties: false
properties:
items:
items:
$ref: '#/components/schemas/Destination'
title: Items
type: array
maxItems: 100
minItems: 0
nextCursor:
type: string
title: Next cursor
description: Cursor for pagination
required:
- items
title: DestinationListWithCursor
type: object
Items_CreateDestination_:
additionalProperties: false
properties:
items:
items:
$ref: '#/components/schemas/CreateDestination'
title: Items
type: array
minLength: 1
maxLength: 10
required:
- items
title: CreateDestination list
type: object
CreateDestination:
additionalProperties: false
properties:
externalId:
title: External ID
$ref: '#/components/schemas/CogniteExternalId'
credentials:
$ref: '#/components/schemas/SessionCredentials'
targetDataSetId:
type: integer
title: Data Set ID
minimum: 1
maximum: 9007199254740991
format: int64
description: Data set ID the created items are inserted into, if applicable.
required:
- externalId
title: CreateDestination
type: object
ItemsWithIgnoreUnknownIds_ExternalId_:
additionalProperties: false
properties:
items:
items:
$ref: '#/components/schemas/ExternalIdWrapper'
title: Items
type: array
maxItems: 100
minItems: 1
ignoreUnknownIds:
type: boolean
title: IgnoreUnknownIds
description: Ignore IDs and external IDs that are not found
required:
- items
title: ItemsWithIgnoreUnknownIds_ExternalId
type: object
ValidationErrorContent:
type: object
required:
- code
- message
description: Cognite API error.
properties:
code:
type: integer
description: HTTP status code.
format: int32
example: 422
message:
type: string
description: Error message.
example: 'Extra inputs are not permitted: type'
UpdateItem_DataSetId_:
additionalProperties: false
properties:
set:
title: Set
type: integer
minimum: 1
maximum: 9007199254740991
format: int64
required:
- set
title: UpdateItem_Port
type: object
description: Set a new dataset ID.
SessionCredentials:
additionalProperties: false
properties:
nonce:
title: Nonce
type: string
description: Session nonce for a recently created CDF Session.
required:
- nonce
title: SessionCredentials
type: object
description: Credentials for authenticating towards CDF using a CDF session.
DestinationUpdateItem:
additionalProperties: false
properties:
externalId:
$ref: '#/components/schemas/CogniteExternalId'
update:
$ref: '#/components/schemas/UpdateDestination'
required:
- externalId
- update
title: DestinationUpdateItem
type: object
responses:
ValidationError:
description: Validation Error
content:
application/json:
schema:
type: object
required:
- error
properties:
error:
$ref: '#/components/schemas/ValidationErrorContent'
ErrorResponse:
description: The response for a failed request.
content:
application/json:
schema:
type: object
required:
- error
properties:
error:
$ref: '#/components/schemas/Error'
400ErrorResponse:
description: The response for a bad request.
content:
application/json:
schema:
type: object
required:
- error
properties:
error:
$ref: '#/components/schemas/Error'
example:
error:
code: 400
message: '`null` values aren''t allowed.'
EmptyResponse:
description: Empty response.
content:
application/json:
schema:
type: object
parameters:
100Limit:
name: limit
description: Limits the number of results to be returned. The maximum results returned by the server is 100 even if you specify a higher limit.
in: query
schema:
type: integer
default: 100
minimum: 1
maximum: 100
Cursor:
name: cursor
description: 'Cursor for paging through results. In general, if a response contains a `nextCursor`
property, it means that there may be more results, and you should pass that value as the
`cursor` parameter in the next request.
Note that the cursor may or may not be encrypted, but either way, it is not intended to be
decoded. Its internal structure is not a part of the public API, and may change without
notice. You should treat it as an opaque string and not attempt to craft your own cursors.
'
in: query
schema:
type: string
example: 4zj0Vy2fo0NtNMb229mI9r1V3YG5NBL752kQz1cKtwo
securitySchemes:
oidc-token:
type: http
scheme: bearer
bearerFormat: OpenID Connect or OAuth2 token
description: Access token issued by the CDF project's configured identity provider. Access token must be an OpenID Connect token, and the project must be configured to accept OpenID Connect tokens. Use a header key of 'Authorization' with a value of 'Bearer $accesstoken'. The token can be obtained through any flow supported by the identity provider.
oauth2-client-credentials:
type: oauth2
description: Access token issued by the CDF project's configured identity provider. Access token must be an OpenID Connect token, and the project must be configured to accept OpenID Connect tokens. Use a header key of 'Authorization' with a value of 'Bearer $accesstoken'. The token can be obtained through any flow supported by the identity provider.
flows:
clientCredentials:
tokenUrl: https://your-idps.token.url/
scopes:
default: https://{cluster}.cognitedata.com/.default
oauth2-auth-code:
type: oauth2
description: Access token issued by the CDF project's configured identity provider. Access token must be an OpenID Connect token, and the project must be configured to accept OpenID Connect tokens. Use a header key of 'Authorization' with a value of 'Bearer $accesstoken'. The token can be obtained through any flow supported by the identity provider.
flows:
authorizationCode:
authorizationUrl: https://your-idps.authorization.url/
tokenUrl: https://your-idps.token.url/
scopes:
default: https://{cluster}.cognitedata.com/.default
oauth2-open-industrial-data:
type: oauth2
description: Auth flow for Open Industrial Data. Get your client secret from https://hub.cognite.com/open-industrial-data-211.
flows:
clientCredentials:
tokenUrl: https://login.microsoftonline.com/48d5043c-cf70-4c49-881c-c638f5796997/oauth2/v2.0/token
scopes:
default: https://api.cognitedata.com/.default
org-oidc-token:
type: openIdConnect
openIdConnectUrl: https://auth.cognite.com/.well-known/openid-configuration
description: 'Access token issued by the Cognite authorization server, and valid for the target organization. The token must
be an OpenID Connect token, and it can be obtained by performing an OIDC login flow toward `auth.cognite.com`.
This is a single URL for all CDF organizations.'
x-tagGroups:
- name: Changelog
tags:
- Changelog
- name: Organizations and projects
tags:
- Organizations
- Projects
- name: Identity and access management
tags:
- Principals
- Groups
- Security categories
- Sessions
- Token
- User profiles
- Project Deletion Reporting
- name: Data modeling
tags:
- Data Modeling
- Data models
- Spaces
- Views
- Containers
- Nodes
- Instances
- Statistics
- Streams
- Records
- name: Asset-centric data model
tags:
- Assets
- Time series
- Synthetic Time Series
- Data point subscriptions
- Events
- Files
- Sequences
- Geospatial
- Seismic
- name: 3D
tags:
- 3D Models
- 3D Model Revisions
- 3D Files
- 3D Asset Mapping
- 3D Contextualization
- 3D Jobs
- 3D Migration
- 3D Scenes
- name: Contextualization
tags:
- Entity matching
- Entity matching pipelines
- Engineering diagrams
- Vision
- Advanced joins
- name: Cognite AI
tags:
- Agents
- Skills
- Chat Completions
- Document AI
- Models
- name: Documents
tags:
- Documents
- Document preview
- name: Data ingestion
tags:
- Raw
- Extraction Pipelines
- Extraction Pipelines Runs
- Extraction Pipelines Config
- Extractors
- name: Data organization
tags:
- Data sets
- Data domains
- Data products
- Rule sets
- Labels
- Relationships
- Annotations
- name: Transformations
tags:
- Transformations
- Transformation Jobs
- Transformation Schedules
- Transformation Notifications
- Query
- Schema
- name: Functions
tags:
- Functions
- Function calls
- Function schedules
- name: Hosted Extractors
tags:
- Sources
- Jobs
- Destinations
- Mappings
- name: PostgreSQL Gateway
tags:
- Postgres Gateway Users
- Postgres Gateway Tables
- name: SAP Writeback
tags:
- SAP Instances
- SAP Endpoints
- Schema Mappings
- Writeback Requests
- name: Data workflows
tags:
- Workflows
- Workflow versions
- Workflow executions
- Workflow triggers
- Tasks
- Workers
- name: Simulators
tags:
- Simulators
- Simulator Integrations
- Simulator Models
- Simulator Routines
- Simulation Runs
- Simulator Logs
- name: Units
tags:
- Units
- Unit Systems
- name: ''
tags:
- ''