Microsoft Azure ImageVisualSearch API

openapi: 3.2.0
info:
  version: 2020-08-01-preview
  title: Microsoft Azure AccessControlClient AccessConnector Image Visual Search API
tags:
- name: ImageVisualSearch
paths:
  /images/visualsearch:
    post:
      summary: 'Microsoft Azure Visual Search Api Lets You Discover Insights About An Image Such As Visually Similar Images, Shopping Sources, And Related Searches The Api Can Also Perform Text Recognition, Identify Entities People, Places, Things , Return Other Topical Content For The User To Explore, And More For More Information, See [visual Search Overview] Https: Docs Microsoft Com Azure Cognitive Services Bing Visual Search Overview'
      operationId: microsoftAzureImagesVisualsearch
      tags:
      - ImageVisualSearch
      parameters:
      - $ref: '#/components/parameters/x-bingapis-sdk'
      - name: Accept
        in: header
        description: The default media type is application/json. To specify that the response use [JSON-LD](http://json-ld.org/), set the Accept header to application/ld+json.
        required: false
        schema:
          type: string
      - name: Accept-Language
        x-ms-client-name: AcceptLanguage
        in: header
        description: A comma-delimited list of one or more languages to use for user interface strings. The list is in decreasing order of preference. For additional information, including expected format, see [RFC2616](http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html). This header and the [setLang](https://docs.microsoft.com/en-us/rest/api/cognitiveservices/bing-visual-search-api-v7-reference#setlang) query parameter are mutually exclusive; do not specify both. If you set this header, you must also specify the [cc](https://docs.microsoft.com/en-us/rest/api/cognitiveservices/bing-visual-search-api-v7-reference#cc) query parameter. To determine the market to return results for, Bing uses the first supported language it finds from the list and combines it with the cc parameter value. If the list does not include a supported language, Bing finds the closest language and market that supports the request or it uses an aggregated or default market for the results. To determine the market that Bing used, see the BingAPIs-Market header. Use this header and the cc query parameter only if you specify multiple languages. Otherwise, use the [mkt](https://docs.microsoft.com/en-us/rest/api/cognitiveservices/bing-visual-search-api-v7-reference#mkt) and [setLang](https://docs.microsoft.com/en-us/rest/api/cognitiveservices/bing-visual-search-api-v7-reference#setlang) query parameters. A user interface string is a string that's used as a label in a user interface. There are few user interface strings in the JSON response objects. Any links to Bing.com properties in the response objects apply the specified language.
        required: false
        schema:
          type: string
      - name: Content-Type
        x-ms-client-name: ContentType
        in: header
        description: Must be set to multipart/form-data and include a boundary parameter (for example, multipart/form-data; boundary=<boundary string>). For more details, see [Content form types]( https://docs.microsoft.com/en-us/azure/cognitive-services/bing-visual-search/overview#content-form-types).
        required: false
        schema:
          type: string
      - name: User-Agent
        x-ms-client-name: UserAgent
        in: header
        description: 'The user agent originating the request. Bing uses the user agent to provide mobile users with an optimized experience. Although optional, you are encouraged to always specify this header. The user-agent should be the same string that any commonly used browser sends. For information about user agents, see [RFC 2616](http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html). The following are examples of user-agent strings. Windows Phone: Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0; ARM; Touch; NOKIA; Lumia 822). Android: Mozilla / 5.0 (Linux; U; Android 2.3.5; en - us; SCH - I500 Build / GINGERBREAD) AppleWebKit / 533.1 (KHTML; like Gecko) Version / 4.0 Mobile Safari / 533.1. iPhone: Mozilla / 5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit / 536.26 (KHTML; like Gecko) Mobile / 10B142 iPhone4; 1 BingWeb / 3.03.1428.20120423. PC: Mozilla / 5.0 (Windows NT 6.3; WOW64; Trident / 7.0; Touch; rv:11.0) like Gecko. iPad: Mozilla / 5.0 (iPad; CPU OS 7_0 like Mac OS X) AppleWebKit / 537.51.1 (KHTML, like Gecko) Version / 7.0 Mobile / 11A465 Safari / 9537.53.'
        required: false
        schema:
          type: string
      - name: X-MSEdge-ClientID
        x-ms-client-name: ClientId
        in: header
        description: 'Bing uses this header to provide users with consistent behavior across Bing API calls. Bing often flights new features and improvements, and it uses the client ID as a key for assigning traffic on different flights. If you do not use the same client ID for a user across multiple requests, then Bing may assign the user to multiple conflicting flights. Being assigned to multiple conflicting flights can lead to an inconsistent user experience. For example, if the second request has a different flight assignment than the first, the experience may be unexpected. Also, Bing can use the client ID to tailor web results to that client ID’s search history, providing a richer experience for the user. Bing also uses this header to help improve result rankings by analyzing the activity generated by a client ID. The relevance improvements help with better quality of results delivered by Bing APIs and in turn enables higher click-through rates for the API consumer. IMPORTANT: Although optional, you should consider this header required. Persisting the client ID across multiple requests for the same end user and device combination enables 1) the API consumer to receive a consistent user experience, and 2) higher click-through rates via better quality of results from the Bing APIs. Each user that uses your application on the device must have a unique, Bing generated client ID. If you do not include this header in the request, Bing generates an ID and returns it in the X-MSEdge-ClientID response header. The only time that you should NOT include this header in a request is the first time the user uses your app on that device. Use the client ID for each Bing API request that your app makes for this user on the device. Persist the client ID. To persist the ID in a browser app, use a persistent HTTP cookie to ensure the ID is used across all sessions. Do not use a session cookie. For other apps such as mobile apps, use the device''s persistent storage to persist the ID. The next time the user uses your app on that device, get the client ID that you persisted. Bing responses may or may not include this header. If the response includes this header, capture the client ID and use it for all subsequent Bing requests for the user on that device. ATTENTION: You must ensure that this Client ID is not linkable to any authenticatable user account information. If you include the X-MSEdge-ClientID, you must not include cookies in the request.'
        required: false
        schema:
          type: string
      - name: X-MSEdge-ClientIP
        x-ms-client-name: ClientIp
        in: header
        description: The IPv4 or IPv6 address of the client device. The IP address is used to discover the user's location. Bing uses the location information to determine safe search behavior. Although optional, you are encouraged to always specify this header and the X-Search-Location header. Do not obfuscate the address (for example, by changing the last octet to 0). Obfuscating the address results in the location not being anywhere near the device's actual location, which may result in Bing serving erroneous results.
        required: false
        schema:
          type: string
      - name: X-Search-Location
        x-ms-client-name: Location
        in: header
        description: 'A semicolon-delimited list of key/value pairs that describe the client''s geographical location. Bing uses the location information to determine safe search behavior and to return relevant local content. Specify the key/value pair as <key>:<value>. The following are the keys that you use to specify the user''s location. lat (required): The latitude of the client''s location, in degrees. The latitude must be greater than or equal to -90.0 and less than or equal to +90.0. Negative values indicate southern latitudes and positive values indicate northern latitudes. long (required): The longitude of the client''s location, in degrees. The longitude must be greater than or equal to -180.0 and less than or equal to +180.0. Negative values indicate western longitudes and positive values indicate eastern longitudes. re (required): The radius, in meters, which specifies the horizontal accuracy of the coordinates. Pass the value returned by the device''s location service. Typical values might be 22m for GPS/Wi-Fi, 380m for cell tower triangulation, and 18,000m for reverse IP lookup. ts (optional): The UTC UNIX timestamp of when the client was at the location. (The UNIX timestamp is the number of seconds since January 1, 1970.) head (optional): The client''s relative heading or direction of travel. Specify the direction of travel as degrees from 0 through 360, counting clockwise relative to true north. Specify this key only if the sp key is nonzero. sp (optional): The horizontal velocity (speed), in meters per second, that the client device is traveling. alt (optional): The altitude of the client device, in meters. are (optional): The radius, in meters, that specifies the vertical accuracy of the coordinates. Specify this key only if you specify the alt key. Although many of the keys are optional, the more information that you provide, the more accurate the location results are. Although optional, you are encouraged to always specify the user''s geographical location. Providing the location is especially important if the client''s IP address does not accurately reflect the user''s physical location (for example, if the client uses VPN). For optimal results, you should include this header and the X-MSEdge-ClientIP header, but at a minimum, you should include this header.'
        required: false
        schema:
          type: string
      - name: mkt
        x-ms-client-name: Market
        in: query
        description: 'The market where the results come from. Typically, mkt is the country where the user is making the request from. However, it could be a different country if the user is not located in a country where Bing delivers results. The market must be in the form <language code>-<country code>. For example, en-US. The string is case insensitive. For a list of possible market values, see [Market Codes](https://docs.microsoft.com/en-us/azure/cognitive-services/bing-visual-search/supported-countries-markets). NOTE: If known, you are encouraged to always specify the market. Specifying the market helps Bing route the request and return an appropriate and optimal response. If you specify a market that is not listed in [Market Codes](https://docs.microsoft.com/en-us/azure/cognitive-services/bing-visual-search/supported-countries-markets), Bing uses a best fit market code based on an internal mapping that is subject to change.'
        required: false
        schema:
          type: string
      - name: safeSearch
        x-ms-client-name: SafeSearch
        in: query
        description: 'Filter the image results in actions with type ''VisualSearch'' for adult content. The following are the possible filter values. Off: May return images with adult content. Moderate: Do not return images with adult content. Strict: Do not return images with adult content. The default is Moderate. If the request comes from a market that Bing''s adult policy requires that safeSearch is set to Strict, Bing ignores the safeSearch value and uses Strict. If you use the site: filter in the knowledge request, there is the chance that the response may contain adult content regardless of what the safeSearch query parameter is set to. Use site: only if you are aware of the content on the site and your scenario supports the possibility of adult content.'
        required: false
        x-ms-enum:
          name: SafeSearch
          modelAsString: true
        schema:
          type: string
          enum:
          - 'Off'
          - Moderate
          - Strict
      - name: setLang
        x-ms-client-name: SetLang
        in: query
        description: The language to use for user interface strings. Specify the language using the ISO 639-1 2-letter language code. For example, the language code for English is EN. The default is EN (English). Although optional, you should always specify the language. Typically, you set setLang to the same language specified by mkt unless the user wants the user interface strings displayed in a different language. A user interface string is a string that's used as a label in a user interface. There are few user interface strings in the JSON response objects. Also, any links to Bing.com properties in the response objects apply the specified language.
        required: false
        schema:
          type: string
      responses:
        '200':
          description: Success.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ImageKnowledge'
        default:
          description: An error has occurred. Check the response type and/or status code for more details.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
      x-ms-examples:
        Successful visual search:
          $ref: ./examples/SuccessfulVisualSearchRequest.json
      description: Needs a more full description created.
      requestBody:
        content:
          multipart/form-data:
            schema:
              type: object
              properties:
                knowledgeRequest:
                  type: string
                  description: The form data is a JSON object that identifies the image using an insights token or URL to the image. The object may also include an optional crop area that identifies an area of interest in the image. The insights token and URL are mutually exclusive – do not specify both. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only (it must not include an insights token or URL).
                image:
                  type: string
                  description: The form data is an image binary. The Content-Disposition header's name parameter must be set to "image". You must specify an image binary if you do not use knowledgeRequest form data to specify the image; you may not use both forms to specify an image. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only  (it must not include an insights token or URL).
                  format: binary
          application/json:
            schema:
              type: object
              properties:
                knowledgeRequest:
                  type: string
                  description: The form data is a JSON object that identifies the image using an insights token or URL to the image. The object may also include an optional crop area that identifies an area of interest in the image. The insights token and URL are mutually exclusive – do not specify both. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only (it must not include an insights token or URL).
                image:
                  type: string
                  description: The form data is an image binary. The Content-Disposition header's name parameter must be set to "image". You must specify an image binary if you do not use knowledgeRequest form data to specify the image; you may not use both forms to specify an image. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only  (it must not include an insights token or URL).
                  format: binary
          text/json:
            schema:
              type: object
              properties:
                knowledgeRequest:
                  type: string
                  description: The form data is a JSON object that identifies the image using an insights token or URL to the image. The object may also include an optional crop area that identifies an area of interest in the image. The insights token and URL are mutually exclusive – do not specify both. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only (it must not include an insights token or URL).
                image:
                  type: string
                  description: The form data is an image binary. The Content-Disposition header's name parameter must be set to "image". You must specify an image binary if you do not use knowledgeRequest form data to specify the image; you may not use both forms to specify an image. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only  (it must not include an insights token or URL).
                  format: binary
          application/xml:
            schema:
              type: object
              properties:
                knowledgeRequest:
                  type: string
                  description: The form data is a JSON object that identifies the image using an insights token or URL to the image. The object may also include an optional crop area that identifies an area of interest in the image. The insights token and URL are mutually exclusive – do not specify both. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only (it must not include an insights token or URL).
                image:
                  type: string
                  description: The form data is an image binary. The Content-Disposition header's name parameter must be set to "image". You must specify an image binary if you do not use knowledgeRequest form data to specify the image; you may not use both forms to specify an image. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only  (it must not include an insights token or URL).
                  format: binary
          text/xml:
            schema:
              type: object
              properties:
                knowledgeRequest:
                  type: string
                  description: The form data is a JSON object that identifies the image using an insights token or URL to the image. The object may also include an optional crop area that identifies an area of interest in the image. The insights token and URL are mutually exclusive – do not specify both. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only (it must not include an insights token or URL).
                image:
                  type: string
                  description: The form data is an image binary. The Content-Disposition header's name parameter must be set to "image". You must specify an image binary if you do not use knowledgeRequest form data to specify the image; you may not use both forms to specify an image. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only  (it must not include an insights token or URL).
                  format: binary
          application/x-www-form-urlencoded:
            schema:
              type: object
              properties:
                knowledgeRequest:
                  type: string
                  description: The form data is a JSON object that identifies the image using an insights token or URL to the image. The object may also include an optional crop area that identifies an area of interest in the image. The insights token and URL are mutually exclusive – do not specify both. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only (it must not include an insights token or URL).
                image:
                  type: string
                  description: The form data is an image binary. The Content-Disposition header's name parameter must be set to "image". You must specify an image binary if you do not use knowledgeRequest form data to specify the image; you may not use both forms to specify an image. You may specify knowledgeRequest form data and image form data in the same request only if knowledgeRequest form data specifies the cropArea field only  (it must not include an insights token or URL).
                  format: binary
components:
  schemas:
    StructuredValue:
      allOf:
      - $ref: '#/components/schemas/Intangible'
      type: object
    ImageTagRegion:
      description: Defines an image region relevant to the ImageTag.
      type: object
      required:
      - queryRectangle
      - displayRectangle
      properties:
        queryRectangle:
          description: A rectangle that outlines the area of interest for this tag.
          $ref: '#/components/schemas/NormalizedQuadrilateral'
        displayRectangle:
          description: A recommended rectangle to show to the user.
          $ref: '#/components/schemas/NormalizedQuadrilateral'
    Error:
      description: Defines the error that occurred.
      type: object
      required:
      - code
      - message
      properties:
        code:
          description: The error code that identifies the category of error.
          type: string
          default: None
          enum:
          - None
          - ServerError
          - InvalidRequest
          - RateLimitExceeded
          - InvalidAuthorization
          - InsufficientAuthorization
          x-ms-enum:
            name: ErrorCode
            modelAsString: true
        subCode:
          description: The error code that further helps to identify the error.
          readOnly: true
          type: string
          enum:
          - UnexpectedError
          - ResourceError
          - NotImplemented
          - ParameterMissing
          - ParameterInvalidValue
          - HttpNotAllowed
          - Blocked
          - AuthorizationMissing
          - AuthorizationRedundancy
          - AuthorizationDisabled
          - AuthorizationExpired
          x-ms-enum:
            name: ErrorSubCode
            modelAsString: true
        message:
          description: A description of the error.
          type: string
        moreDetails:
          description: A description that provides additional information about the error.
          readOnly: true
          type: string
        parameter:
          description: The parameter in the request that caused the error.
          readOnly: true
          type: string
        value:
          description: The parameter's value in the request that was not valid.
          readOnly: true
          type: string
    Response:
      description: Defines a response. All schemas that return at the root of the response must inherit from this object.
      allOf:
      - $ref: '#/components/schemas/Identifiable'
      type: object
      properties:
        readLink:
          description: The URL that returns this resource. To use the URL, append query parameters as appropriate and include the Ocp-Apim-Subscription-Key header.
          readOnly: true
          type: string
        webSearchUrl:
          description: The URL to Bing's search result for this item.
          readOnly: true
          type: string
    Organization:
      description: Defines an organization.
      allOf:
      - $ref: '#/components/schemas/Thing'
      type: object
      properties: {}
    Point2D:
      description: Defines a 2D point with X and Y coordinates.
      allOf:
      - $ref: '#/components/schemas/StructuredValue'
      type: object
      required:
      - x
      - y
      properties:
        x:
          description: The x-coordinate of the point.
          type: number
          format: float
        y:
          description: The y-coordinate of the point.
          type: number
          format: float
    ImageObject:
      description: Defines an image.
      allOf:
      - $ref: '#/components/schemas/MediaObject'
      type: object
      properties:
        thumbnail:
          description: The URL to a thumbnail of the image.
          $ref: '#/components/schemas/ImageObject'
          readOnly: true
        imageInsightsToken:
          description: The token that you use in a subsequent call to Visual Search API to get additional information about the image. For information about using this token, see the imageInsightsToken field inside the knowledgeRequest request parameter.
          readOnly: true
          type: string
        insightsMetadata:
          description: A count of the number of websites where you can shop or perform other actions related to the image. For example, if the image is of an apple pie, this object includes a count of the number of websites where you can buy an apple pie. To indicate the number of offers in your UX, include badging such as a shopping cart icon that contains the count. When the user clicks on the icon, use imageInsightsToken in a subsequent Visual Search API call to get the list of shopping websites.
          $ref: '#/components/schemas/ImagesImageMetadata'
          readOnly: true
        imageId:
          description: Unique Id for the image.
          readOnly: true
          type: string
        accentColor:
          description: A three-byte hexadecimal number that represents the color that dominates the image. Use the color as the temporary background in your client until the image is loaded.
          readOnly: true
          type: string
        visualWords:
          description: For internal use only.
          readOnly: true
          type: string
    PropertiesItem:
      description: Defines an item.
      x-ms-discriminator-value: Properties/Item
      discriminator:
        propertyName: _type
      type: object
      required:
      - _type
      properties:
        text:
          description: Text representation of an item.
          readOnly: true
          type: string
        _type:
          type: string
    ImageTag:
      description: A visual search tag.
      allOf:
      - $ref: '#/components/schemas/Thing'
      type: object
      properties:
        displayName:
          description: Display name for this tag. For the default tag, the display name is empty.
          readOnly: true
          type: string
        boundingBox:
          description: The bounding box for this tag. For the default tag, there is no bounding box.
          $ref: '#/components/schemas/ImageTagRegion'
          readOnly: true
        actions:
          description: Actions within this tag. The order of the items denotes the default ranking order of these actions, with the first action being the most likely user intent.
          readOnly: true
          type: array
          items:
            $ref: '#/components/schemas/ImageAction'
    MediaObject:
      description: Defines a media object.
      allOf:
      - $ref: '#/components/schemas/CreativeWork'
      type: object
      properties:
        contentUrl:
          description: Original URL to retrieve the source (file) for the media object (e.g., the source URL for the image).
          readOnly: true
          type: string
        hostPageUrl:
          description: URL of the page that hosts the media object.
          readOnly: true
          type: string
        contentSize:
          description: Size of the media object content. Use format "value unit" (e.g., "1024 B").
          readOnly: true
          type: string
        encodingFormat:
          description: Encoding format (e.g., png, gif, jpeg, etc).
          readOnly: true
          type: string
        hostPageDisplayUrl:
          description: Display URL of the page that hosts the media object.
          readOnly: true
          type: string
        width:
          description: The width of the media object, in pixels.
          readOnly: true
          type: integer
          format: int32
        height:
          description: The height of the media object, in pixels.
          readOnly: true
          type: integer
          format: int32
    Intangible:
      description: A utility class that serves as the umbrella for a number of 'intangible' things such as quantities, structured values, etc.
      allOf:
      - $ref: '#/components/schemas/Thing'
      type: object
    ImageAction:
      description: Defines an image action.
      allOf:
      - $ref: '#/components/schemas/Action'
      type: object
      discriminator:
        propertyName: _type
      properties:
        actionType:
          description: A string representing the type of action.
          readOnly: true
          type: string
        _type:
          type: string
          description: Specifies the sub-class of the action.
          enum:
          - ImageEntityAction
          - ImageModuleAction
          - ImageRecipesAction
          - ImageRelatedSearchesAction
          - ImageShoppingSourcesAction
    ImageKnowledge:
      description: Defines a visual search API response.
      allOf:
      - $ref: '#/components/schemas/Response'
      type: object
      properties:
        tags:
          description: A list of visual search tags.
          readOnly: true
          type: array
          items:
            $ref: '#/components/schemas/ImageTag'
        image:
          description: Image object containing metadata about the requested image.
          $ref: '#/components/schemas/ImageObject'
          readOnly: true
    Identifiable:
      description: Defines the identity of a resource.
      allOf:
      - $ref: '#/components/schemas/ResponseBase'
      type: object
      properties:
        id:
          description: A String identifier.
          readOnly: true
          type: string
    ResponseBase:
      description: Response base.
      discriminator:
        propertyName: _type
      type: object
      properties:
        _type:
          type: string
    ImagesImageMetadata:
      description: Defines a count of the number of websites where you can shop or perform other actions related to the image.
      x-ms-discriminator-value: Images/ImageMetadata
      type: object
      properties:
        shoppingSourcesCount:
          description: The number of websites that sell the products seen in the image.
          readOnly: true
          type: integer
          format: int32
        recipeSourcesCount:
          description: The number of websites that offer recipes of the food seen in the image.
          readOnly: true
          type: integer
          format: int32
        aggregateOffer:
          description: 'A summary of the online offers of products found in the image. For example, if the image is of a dress, the offer might identify the lowest price and the number of offers found. Only visually similar products insights include this field. The offer includes the following fields: Name, AggregateRating, OfferCount, and LowPrice.'
          $ref: '#/components/schemas/AggregateOffer'
          readOnly: true
    NormalizedQuadrilateral:
      description: Defines a region of an image. The region is a convex quadrilateral defined by coordinates of its top left, top right, bottom left, and bottom right points. The coordinates are fractional values of the original image's width and height in the range 0.0 through 1.0.
      allOf:
      - $ref: '#/components/schemas/StructuredValue'
      type: object
      required:
      - topLeft
      - topRight
      - bottomRight
      - bottomLeft
      properties:
        topLeft:
          description: The top left corner coordinate.
          $ref: '#/components/schemas/Point2D'
        topRight:
          description: The to

# --- truncated at 32 KB (41 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/microsoft-azure/refs/heads/main/openapi/microsoft-azure-imagevisualsearch-api-openapi.yml