openapi: 3.1.0
info:
title: Cognite 3D Asset Mapping Documents API
description: "# Introduction\nThis is the reference documentation for the Cognite API with\nan overview of all the available methods.\n\n# Postman\nSelect the **Download** button to download our OpenAPI specification to get started.\n\nTo import your data into Postman, select **Import**, and the Import modal opens.\nYou can import items by dragging or dropping files or folders. You can choose how to import your API and manage the import settings in **View Import Settings**.\n\nIn the Import Settings, set the **Folder organization** to **Tags**, select\n**Enable optional parameters** to turn off the settings, and select **Always inherit authentication** to turn on the settings. Select **Import**.\n\nSet the Authorization to **Oauth2.0**. By default, the settings are for Open Industrial Data. Navigate to [Cognite Hub](https://hub.cognite.com/open-industrial-data-211) to understand how to get the credentials for use in Postman.\n\nFor more information, see [Getting Started with Postman](https://developer.cognite.com/dev/guides/postman/).\n\n# Pagination\nMost resource types can be paginated, indicated by the field `nextCursor` in the response.\nBy passing the value of `nextCursor` as the cursor you will get the next page of `limit` results.\nNote that all parameters except `cursor` has to stay the same.\n\n# Parallel retrieval\nAs general guidance, Parallel Retrieval is a technique that should be used when due to query complexity, retrieval of data in a single request is significantly slower than it would otherwise be for a simple request. Parallel retrieval does not act as a speed multiplier on optimally running queries. By parallelizing such requests, data retrieval performance can be tuned to meet the client application needs. \n\nCDF supports parallel retrieval through the `partition` parameter, which has the format `m/n` where `n` is the amount of partitions you would like to split the entire data set into.\nIf you want to download the entire data set by splitting it into 10 partitions, do the following in parallel with `m` running from 1 to 10:\n - Make a request to `/events` with `partition=m/10`.\n - Paginate through the response by following the cursor as explained above. Note that the `partition` parameter needs to be passed to all subqueries.\n\nProcessing of parallel retrieval requests is subject to concurrency quota availability. The request returns the `429` response upon exceeding concurrency limits. See the Request throttling chapter below.\n\nTo prevent unexpected problems and to maximize read throughput, you should at most use 10 partitions. \nSome CDF resources will automatically enforce a maximum of 10 partitions.\nFor more specific and detailed information, please read the ```partition``` attribute documentation for the CDF resource you're using. \n\n# Requests throttling\nCognite Data Fusion (CDF) returns the HTTP `429` (too many requests) response status code when project capacity exceeds the limit.\n\nThe throttling can happen:\n - If a user or a project sends too many (more than allocated) concurrent requests.\n - If a user or a project sends a too high (more than allocated) rate of requests in a given amount of time.\n\nCognite recommends using a retry strategy based on truncated exponential backoff to handle sessions with HTTP response codes 429.\n\nCognite recommends using a reasonable number (up to 10) of `Parallel retrieval` partitions.\n\nFollowing these strategies lets you slow down the request frequency to maximize productivity without having to re-submit/retry failing requests.\n\nSee more [here](https://docs.cognite.com/dev/concepts/resource_throttling).\n\n# API versions\n## Version headers\nThis API uses calendar versioning, and version names follow the `YYYYMMDD` format.\nYou can find the versions currently available by using the version selector at the top of this page.\n\nTo use a specific API version, you can pass the `cdf-version: $version` header along with your requests to the API.\n\n## Beta versions\nThe beta versions provide a preview of what the stable version will look like in the future.\nBeta versions contain functionality that is reasonably mature, and highly likely to become a part of the stable API.\n\nBeta versions are indicated by a `-beta` suffix after the version name. For example, the beta version header for the\n2023-01-01 version is then `cdf-version: 20230101-beta`.\n\n## Alpha versions\nAlpha versions contain functionality that is new and experimental, and not guaranteed to ever become a part of the stable API.\nThis functionality presents no guarantee of service, so its use is subject to caution.\n\nAlpha versions are indicated by an `-alpha` suffix after the version name. For example, the alpha version header for\nthe 2023-01-01 version is then `cdf-version: 20230101-alpha`."
version: v1
contact:
name: Cognite Support
url: https://support.cognite.com
email: support@cognite.com
servers:
- url: https://{cluster}.cognitedata.com/api/v1/projects/{project}
description: The URL for the CDF cluster to connect to
variables:
cluster:
enum:
- api
- az-tyo-gp-001
- az-eastus-1
- az-power-no-northeurope
- westeurope-1
- asia-northeast1-1
- gc-dsm-gp-001
default: api
description: The CDF cluster to connect to
project:
default: publicdata
description: The CDF project name.
security:
- oidc-token:
- https://{cluster}.cognitedata.com/.default
- oauth2-client-credentials:
- https://{cluster}.cognitedata.com/.default
- oauth2-open-industrial-data:
- https://api.cognitedata.com/.default
- oauth2-auth-code:
- https://{cluster}.cognitedata.com/.default
tags:
- name: Documents
description: "A document is a file that has been indexed by the document search engine.\nEvery time a file is uploaded, updated or deleted in the Files API, it will also\nbe scheduled for processing by the document search engine. After some processing,\nit will be possible to search for the file in the document search API.\n\nThe document search engine is able to extract content from a variety of document\ntypes, and perform classification, contextualization and other operations on the\nfile. This extracted and derived information is made available in the form of a\n`Document` object.\n\nThe document structure consists of a selection of derived fields, such as the\n`title`, `author` and `language` of the document, plus some of the original fields\nfrom the raw file. The fields from the raw file can be found in the\n`sourceFile` structure. The derived fields are described in more detail below.\n\n### Derived fields\n\n#### title\nSome document types (such as PDFs) contain additional metadata fields. If the\ndocument contains its title as part of this metadata, this field will be populated\nwith that title.\n\nNote that we do not currently extract the title from the document content itself.\nIf there is a need for this, we may consider adding such functionality in the future.\n\n#### author\nSimilar to the `title` field, the author field is another field that can often be\nextracted from the document's metadata.\n\n#### producer\nThe `producer` field also exists in the document metadata. It contains information\nabout the software or the system that was used to create the document.\n\n#### createdTime\nThe `createdTime` we assign to the document is not exactly the same as the one found\nin the Files API. We first try to extract the created time from the document metadata.\nIf the document does not contain such a timestamp, we fall back to the time set in\nthe Files API.\n\n#### mimeType\nIf there is a mime type set in on the file in the Files API, this field will be set\nto the same mime type. If there is no mime type set on the file, we will try to\nauto-detect it.\n\n#### extension\nThis field contains the extension of the file, derived from the file name. For\ninstance, if the file name is `My Document.docx`, the `extension` field will contain\n`docx`.\n\n#### pageCount\nContains the number of pages in the document, if possible to determine.\n\n#### type\nThe `type` field contains a high level file type, derived from the mime type. Mime\ntypes are not that pleasant to look at, and not always easy to understand. That is\nwhy we map the mime types into more user-friendly types. Below is the list of types\ncurrently returned, but be aware that this list may be extended in the future.\n\n- `Document`: Document files from Microsoft Word or similar word processing software.\n- `PDF`: PDF files.\n- `Spreadsheet`: Files from Microsoft Excel or similar spreadsheet software.\n- `Presentation`: Slides from Microsoft Powerpoint or similar.\n- `Image`: Any kind of image such as PNG or JPG files.\n- `Video`: Any kind of video such as MOV or MP4 files.\n- `Tabular data`: Csv, tsv and other kinds of tabular data files.\n- `Plain text`: Plain text files.\n- `Compressed`: ZIP files and other kinds of compressed archive files.\n- `Script`: Program code such as python or matlab.\n- `Other`: Anything that doesn't fit in any of the above types.\n\n#### geoLocation\nIf there is a geolocation set on the file in the Files API, then this field will contain\nthe same geolocation. If there is no explicitly assigned geolocation, the document\nprocessing system will try to detect a location using two different techniques;\n\n1. We will extract locations from files that contain embedded GPS locations. Photos and\n videos often have this kind of metadata.\n2. We will look at related assets that have locations, and assign the same location(s) to\n the document.\n\n### File type support\n\nWe create a document for each uploaded file, but only derive data from certain files.\n\nThe following file types are eligible for further data extraction & enrichment:\n- PDF files\n- Spreadsheets, documents, and presentations from the Microsoft, Libre Office and macOS office suites\n- Plain text files\n- Images"
paths:
/documents/search:
post:
tags:
- Documents
summary: Search for documents
description: "\n\n> **Required capabilities:** `filesAcl:READ`\n\nThis endpoint lets you search for documents by using advanced filters and free text queries.\nFree text queries are matched against the documents' filenames and contents.\n\n### Free text search\n\n#### Boolean operators\n\nThe `+` symbol represents an AND operation, and the `|` symbol represents an OR.\nSearching for `lorem + ipsum` will match documents containing both \"lorem\" AND \"ipsum\" in the filename or content.\nSimilarly, searching for `lorem | ipsum` will match documents containing either \"lorem\" OR \"ipsum\" in the filename or\ncontent.\n\nThe default operator between the search keywords is AND.\nThat means that searching for two terms without any operator, for example, `lorem ipsum`, will\nmatch documents containing both the words \"lorem\" and \"ipsum\" in the filename or content.\n\nYou can use the operator `-` to exclude documents containing a specific word.\nFor instance, the search `lorem -ipsum` will match documents that contain the word \"lorem\", but does NOT contain the\nword \"ipsum\".\n\n#### Phrases\n\nEnclose multiple words inside double quotes `\"` to group these words together.\nNormally, the search query `lorem ipsum` will match not only \"lorem ipsum\" but also \"lorem cognite ipsum\",\nand in general, there can be any number of words between the two words in the query.\nThe search query `\"lorem ipsum\"`, however, will match only exactly \"lorem ipsum\" and not \"lorem cognite ipsum\".\n\n#### Escape\n\nTo search for the special characters (`+`, `|`, `-`, `\"`. `\\`), escape with a preceding backslash `\\`.\n\n#### Ordering\n\nWhen you search for a term, the endpoint tries to return the most relevant documents first, with less relevant documents further down\nthe list.\nThere are a few factors that determine the relevance of a document:\n\n- If the search terms are found multiple times within a document, the relevance of that document is higher.\n- For searches with multiple terms, documents containing all the terms are considered more relevant than documents\n containing only some.\n- Matches of the terms in the filename field of the document are more relevant than matches in the document's content.\n\n#### Examples\n\nThe following request will return documents matching the specified search query.\n\n```json\n{\n \"search\": {\n \"query\": \"cognite \\\"lorem ipsum\\\"\"\n }\n}\n ```\n\nThe following example combines a query with a filter.\nThe search request will return documents matching the search query, where `externalId` starts with \"1\".\nThe results will be ordered by how well they match the query string.\n\n```json\n{\n \"search\":{\n \"query\":\"cognite \\\"lorem ipsum\\\"\"\n },\n \"filter\":{\n \"prefix\":{\n \"property\":[\n \"externalId\"\n ],\n \"value\":\"1\"\n }\n }\n}\n```\n\n### Highlights\n\nWhen you enable highlights for your search query, the response contains an additional highlight field for each\nsearch hit, including the highlighted fragments for your query matches. However, the result limit is 20 documents due to the operation costs.\n\n### Filtering\n\nFiltering uses a special JSON filtering language.\nIt's quite flexible and consists of a number of different \"leaf\" filters, which can be combined arbitrarily using the boolean clauses `and`, `or`, and `not`.\n\n#### Supported leaf filters\n\n<table>\n<tr>\n<th> <div style=\"width:150px\"> Leaf filter </div> </th> \n<th> <div style=\"width:200px\"> Supported fields </div> </th> \n<th> Description </th>\n</tr>\n\n<tr>\n<td> <code>equals</code> </td> <td> Non-array type fields </td> <td> Only includes results that are equal to the specified value. \n\n```json\n{\n \"equals\":{\n \"property\":[\n \"property\"\n ],\n \"value\":\"example\"\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>in</code> </td> <td> Non-array type fields </td> <td> Only includes results that are equal to one of the specified values. \n\n```json\n{\n \"in\":{\n \"property\":[\n \"property\"\n ],\n \"values\":[\n 1,\n 2,\n 3\n ]\n }\n}\n```\n</td>\n</tr>\n\n<td> <code>containsAll</code> </td> <td> Array type fields </td> <td> Only includes results which contain all of the specified values. \n\n```json\n{\n \"containsAll\":{\n \"property\":[\n \"property\"\n ],\n \"values\":[\n 1,\n 2,\n 3\n ]\n }\n}\n```\n</td>\n<tr>\n\n<td> <code>containsAny</code> </td> <td> Array type fields </td> <td> Only includes results which contain all of the specified values. \n\n```json\n{\n \"containsAny\":{\n \"property\":[\n \"property\"\n ],\n \"values\":[\n 1,\n 2,\n 3\n ]\n }\n}\n```\n</td>\n\n<tr>\n<td> <code>exists</code> </td> <td> All fields </td> <td> Only includes results where the specified property exists (has value). \n\n```json\n{\n \"exists\":{\n \"property\":[\n \"property\"\n ]\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>prefix</code> </td> <td> String type fields </td> <td> Only includes results which start with the specified value. \n\n```json\n{\n \"prefix\":{\n \"property\":[\n \"property\"\n ],\n \"value\":\"example\"\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>range</code> </td> <td> Non-array type fields </td> <td> Only includes results that fall within the specified range. \n\nSupported operators: <code>gt</code>, <code>lt</code>, <code>gte</code>, <code>lte</code>\n\n```json\n{\n \"range\":{\n \"property\":[\n \"property\"\n ],\n \"gt\":1,\n \"lte\":5\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>geojsonIntersects</code> </td> <td> <code>geoLocation</code> </td> <td> Only includes results where the geoshape intersects with the specified geometry.\n\n```json\n{\n \"geojsonIntersects\":{\n \"property\":[\n \"sourceFile\",\n \"geoLocation\"\n ],\n \"geometry\":{\n \"type\":\"Polygon\",\n \"coordinates\": [[[30, 10], [40, 40], [20, 40], [10, 20], [30, 10]]],\n }\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>geojsonDisjoint</code> </td> <td> <code>geoLocation</code> </td> <td> Only includes results where the geoshape has nothing in common with the specified geometry.\n\n```json\n{\n \"geojsonDisjoint\":{\n \"property\":[\n \"sourceFile\",\n \"geoLocation\"\n ],\n \"geometry\":{\n \"type\":\"Polygon\",\n \"coordinates\": [[[30, 10], [40, 40], [20, 40], [10, 20], [30, 10]]],\n }\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>geojsonWithin</code> </td> <td> <code>geoLocation</code> </td> <td> Only includes results where the geoshape falls within the specified geometry.\n\n```json\n{\n \"geojsonWithin\":{\n \"property\":[\n \"sourceFile\",\n \"geoLocation\"\n ],\n \"geometry\":{\n \"type\":\"Polygon\",\n \"coordinates\": [[[30, 10], [40, 40], [20, 40], [10, 20], [30, 10]]],\n }\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>inAssetSubtree</code> </td> <td> <code>assetIds</code>, <code>assetExternalIds</code> </td> <td> Only includes results with a related asset in a subtree rooted at any specified IDs.\n\n```json\n{\n \"filter\":{\n \"inAssetSubtree\":{\n \"property\":[\n \"sourceFile\",\n \"assetIds\"\n ],\n \"values\":[\n 1,\n 2,\n 3\n ]\n }\n }\n}\n```\n</td>\n</tr>\n\n<tr>\n<td> <code>search</code> </td> <td> <code>name</code>, <code>content</code> </td> <td>\n\n```json\n{\n \"search\":{\n \"property\":[\n \"property\"\n ],\n \"value\":\"example\"\n }\n}\n```\n</td>\n</tr>\n\n</table>\n\n#### Properties\n\nThe following overview shows the properties you can filter and which filter applies to which property.\n\n| Property | Type | Applicable filters |\n|------------------------------------------|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `[\"id\"]` | integer | equals, in, range, exists |\n| `[\"externalId\"]` | string | equals, in, prefix, exists |\n| `[\"instanceId\"]` | string | equals, in |\n| `[\"instanceId\", \"space\"]` | string | equals, in, prefix, exists |\n| `[\"instanceId\", \"externalId\"]` | string | equals, in, prefix, exists |\n| `[\"title\"]` | string | equals, in, prefix, exists |\n| `[\"author\"]` | string | equals, in, prefix, exists |\n| `[\"createdTime\"]` | integer | equals, in, range, exists |\n| `[\"modifiedTime\"]` | integer | equals, in, range, exists |\n| `[\"lastIndexedTime\"]` | integer | equals, in, range, exists |\n| `[\"mimeType\"]` | string | equals, in, prefix, exists |\n| `[\"extension\"]` | string | equals, in, prefix, exists |\n| `[\"pageCount\"]` | integer | equals, in, range, exists |\n| `[\"type\"]` | string | equals, in, prefix, exists |\n| `[\"geoLocation\"]` | geometry object | geojsonIntersects, geojsonDisjoint, geojsonWithin, exists |\n| `[\"language\"]` | string | equals, in, prefix, exists |\n| `[\"assetIds\"]` | array of integers | containsAny, containsAll, exists, inAssetSubtree |\n| `[\"assetExternalIds\"]` | array of strings | containsAny, containsAll, exists, inAssetSubtree |\n| `[\"labels\"]` | array of Labels | containsAny, containsAll, exists |\n| `[\"content\"]` | string | search |\n| `[\"sourceFile\", \"name\"]` | string | equals, in, prefix, exists, search |\n| `[\"sourceFile\", \"mimeType\"]` | string | equals, in, prefix, exists |\n| `[\"sourceFile\", \"size\"]` | integer | equals, in, range, exists |\n| `[\"sourceFile\", \"source\"]` | string | equals, in, prefix, exists |\n| `[\"sourceFile\", \"directory\"]` | string | equals, in, prefix, exists |\n| `[\"sourceFile\", \"assetIds\"]` | array of integers | containsAny, containsAll, exists, inAssetSubtree |\n| `[\"sourceFile\", \"assetExternalIds\"]` | array of strings | containsAny, containsAll, exists, inAssetSubtree |\n| `[\"sourceFile\", \"datasetId\"]` | integer | equals, in, range, exists |\n| `[\"sourceFile\", \"securityCategories\"]` | array of integers | containsAny, containsAll, exists |\n| `[\"sourceFile\", \"geoLocation\"]` | geometry object | geojsonIntersects, geojsonDisjoint, geojsonWithin, exists |\n| `[\"sourceFile\", \"labels\"]` | array of Labels | containsAny, containsAll, exists |\n| `[\"sourceFile\", \"metadata\", <key>]` | string | equals, in, prefix, exists |\n| `[\"sourceFile\", \"metadata\"]` | string | equals, in, prefix, exists<br><br>This is a special filter field that targets all metadata values. <br>An alternative to creating a filter for each key in the metadata field. |\n\n#### Full example\n\n```json\n{\n \"filter\": {\n \"and\": [\n {\n \"or\": [\n {\n \"equals\": {\n \"property\": [\n \"type\"\n ],\n \"value\": \"PDF\"\n }\n },\n {\n \"prefix\": {\n \"property\": [\n \"externalId\"\n ],\n \"value\": \"hello\"\n }\n }\n ]\n },\n {\n \"range\": {\n \"property\": [\n \"createdTime\"\n ],\n \"lte\": 1519862400000\n }\n },\n {\n \"not\": {\n \"in\": {\n \"property\": [\n \"sourceFile\",\n \"name\"\n ],\n \"values\": [\n \"My Document.doc\",\n \"My Other Document.docx\"\n ]\n }\n }\n }\n ]\n }\n}\n ```\n\n### Sorting\n\nBy default, search results are ordered by relevance, meaning how well they match the given query string.\nHowever, it's possible to specify a different property to sort by.\nSorting can be ascending or descending. The sort order is ascending if none is specified.\n\n#### Properties\n\nThe following overview shows all properties that can be sorted on.\n\n| Property |\n|---------------------------------|\n| `[\"id\"]` |\n| `[\"externalId\"]` |\n| `[\"mimeType\"]` |\n| `[\"extension\"]` |\n| `[\"pageCount\"]` |\n| `[\"author\"]` |\n| `[\"title\"]` |\n| `[\"language\"]` |\n| `[\"type\"]` |\n| `[\"createdTime\"]` |\n| `[\"modifiedTime\"]` |\n| `[\"lastIndexedTime\"]` |\n| `[\"sourceFile\", \"name\"]` |\n| `[\"sourceFile\", \"mimeType\"]` |\n| `[\"sourceFile\", \"size\"]` |\n| `[\"sourceFile\", \"source\"]` |\n| `[\"sourceFile\", \"datasetId\"]` |\n| `[\"sourceFile\", \"metadata\", *]` |\n\n#### Example\n\n```json\n{\n \"sort\":[\n {\n \"property\":[\n \"createdTime\"\n ],\n \"order\":\"asc\",\n \n }\n ]\n}\n ```\n"
operationId: documentsSearch
requestBody:
description: Fields to be set for the search request.
content:
application/json:
schema:
$ref: '#/components/schemas/DocumentSearchRequest'
required: true
responses:
'200':
$ref: '#/components/responses/DocumentSearchResponse'
'400':
$ref: '#/components/responses/ErrorResponse'
x-capability:
- filesAcl:READ
- null
x-code-samples:
- lang: JavaScript
label: JavaScript SDK
source: "const documents = await client.documents.search({\n search: {\n query: 'Stuck pipe'\n },\n filter: {\n in: {\n property: ['type'],\n values: ['Document', 'PDF']\n }\n },\n limit: 5\n});"
- lang: Python
label: Python SDK
source: "from cognite.client.data_classes import filters\nfrom cognite.client.data_classes.documents import DocumentProperty\nis_pdf = filters.Equals(DocumentProperty.mime_type, \"application/pdf\")\ndocuments = client.documents.search(\"pump 123\", filter=is_pdf)\n\nfrom datetime import datetime, timedelta\nfrom cognite.client.data_classes import filters\nfrom cognite.client.data_classes.documents import DocumentProperty\nfrom cognite.client.utils import timestamp_to_ms\nis_plain_text = filters.Equals(DocumentProperty.mime_type, \"text/plain\")\nlast_week = filters.Range(DocumentProperty.created_time,\n gt=timestamp_to_ms(datetime.now() - timedelta(days=7)))\ndocuments = client.documents.search('\"CPLEX Error 1217: No Solution exists.\"',\n highlight=True,\n filter=filters.And(is_plain_text, last_week))\n"
/documents/passages/search:
post:
tags:
- Documents
summary: Semantic search for passages
description: "\n\n> **Required capabilities:** `filesAcl:READ`\n\nThis endpoint lets you search for relevant passages of up to 100 PDF documents by using advanced filters and semantic search queries. \nWhere documents search results in a list of relevant documents, this endpoint gives you a result of the most relevant parts of a set of documents.\nFor each query you have to specify a list of document ids using the `in` filter (see exmple).\n\nSemantic search works by comparing the semantic meaning of the search query to the semantic meaning of the document passages. \nThe document passages are automatically derived and indexed upon file uploads (PDF only).\n\nA natural flow would be:\n 1. Upload a document\n 2. Query `/documents/status` to check if the document is ready (see beta documentation)\n 3. Query `/documents/passages/search` to find relevant passages in the document(s) \n\n### Passages search offers semantic search filters\n\n> This endpoint requires a list of document ids that you want to do semantic search on. \n> You can not currently search through all passages for the entire project in a single query.\n\n#### Ordering\n\nSearch results are ranked by relevance, with the most relevant appearing first. How relevance is determined depends on the fil
# --- truncated at 32 KB (107 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/cognite/refs/heads/main/openapi/cognite-documents-api-openapi.yml