Mobilerun Tools/Navigation API

The Tools/Navigation API from Mobilerun — 6 operation(s) for tools/navigation.

GitHub OpenAPI

Operations 8

POST /devices/{deviceId}/global Perform a global action #
DELETE /devices/{deviceId}/keyboard Clear input #
POST /devices/{deviceId}/keyboard Input text #
PUT /devices/{deviceId}/keyboard Input key #
GET /devices/{deviceId}/screenshot Take screenshot #
POST /devices/{deviceId}/swipe Swipe #
POST /devices/{deviceId}/tap Tap by coordinates #
GET /devices/{deviceId}/ui-state UI state #

Work with this as data

Every API here is available over the APIs.io API and to AI agents over MCP.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for apis

7 MCP tools reach this
  • find_apisBrowse and filter every API in the catalog.
  • get_api_artifactsOne API's artifacts, grouped by type.
  • get_openapiThe primary OpenAPI for this API.
  • find_similar_apisAPIs that look like this one.
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
This API
curl "https://apis.io/api/v1/apis/mobilerun-ai:mobilerun-ai-tools-navigation-api"
All apis
curl "https://apis.io/api/v1/apis?limit=25"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. By signing up you are subscribing to the APIs.io and API Evangelist newsletters — you can unsubscribe from any issue. See our Privacy Policy and Terms.

A second provider on the same verified email joins the account you already have.

OpenAPI Specification

mobilerun-ai-tools-navigation-api-openapi.yml Raw ↑
openapi: 3.2.0
info:
  title: Mobilerun Ai Tools/Navigation API
  version: v1
  contact:
    name: Droidrun
    email: support@droidrun.ai
  description: 'Operations tagged Tools/Navigation across 2 of this provider''s published API definitions: openapi.json, mobilerun-ai-openapi.json. Each path carries the servers of the definition it was published in.'
servers:
- url: https://api.mobilerun.ai
  description: Droidrun Cloud API
security:
- bearerAuth: []
tags:
- name: Tools/Navigation
paths:
  /devices/{deviceId}/global:
    post:
      description: Performs a global system action on the device, such as navigating back or going to the home screen, identified by an action code.
      operationId: global
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/GlobalArgs'
        required: true
      responses:
        '204':
          description: No Content
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Perform a global action
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Action codes follow Android''s accessibility global actions:


          - `1` — back

          - `2` — home

          - `3` — recents (app switcher)


          These three work on every Android device. Higher codes — `4` (notifications), `5` (quick settings), `6` (power dialog), `8` (lock screen) — are device-dependent and rejected with an error where unsupported.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
    servers:
    - url: https://api.mobilerun.ai
      description: Droidrun Cloud API
  /devices/{deviceId}/keyboard:
    delete:
      description: Clears the contents of the currently focused text input field.
      operationId: clear-input
      responses:
        '204':
          description: No Content
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Clear input
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Clears whatever text the currently focused input field contains. Make sure an editable field has focus first — check `phone_state.isEditable` and `focusedElement` in the UI state response.


          When you''re replacing a field''s contents, the input text endpoint''s `clear` flag does the same thing in one call.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
    post:
      description: Types text into the focused input field. The optional completionMode defaults to accepted for backwards-compatible low latency; committed additionally waits for the complete text or a quiescent UI state.
      operationId: input-text
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/KeyboardInputArgs'
        required: true
      responses:
        '204':
          description: No Content
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Input text
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Text goes to the currently focused input field — tap a field first, and confirm focus via the UI state response (`phone_state.focusedElement` and `keyboardVisible`).


          `wpm` and `errorRate` are humanize tuning knobs, and each applies to a different class of device: devices with human-like input support honor `errorRate` and ignore `wpm`; all other devices honor `wpm` and ignore `errorRate`.


          `completionMode` is optional and defaults to `accepted`, which preserves the existing behavior and returns after the device input provider accepts the operation. Use `committed` when the next action depends on the complete value being visible: the request then waits until the focused field contains the full expected text or the focused UI state remains unchanged for a bounded stability window. Devices without accessibility read-back reject `committed` before typing.


          Set `humanize: true` for human-like typing where the device supports it. Deprecated alias: `stealth`. If both are sent they must agree, otherwise the request is rejected with 422.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
    put:
      description: Sends a single Android key event to the device, identified by its key code.
      operationId: input-key
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/KeyboardKeyArgs'
        required: true
      responses:
        '204':
          description: No Content
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Input key
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Key codes are standard Android `KeyEvent` codes — for example `66` (Enter) or `4` (Back).


          A common pattern: type into a field with the input text endpoint, then send `66` to submit it.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
    servers:
    - url: https://api.mobilerun.ai
      description: Droidrun Cloud API
  /devices/{deviceId}/screenshot:
    get:
      deprecated: true
      description: 'Deprecated: use POST /devices/{deviceId}/screenshots. Captures the device screen and returns it as a PNG image. An optional hideOverlay query parameter excludes the accessibility overlay from the capture.'
      operationId: take-screenshot
      responses:
        '200':
          content:
            application/json:
              schema:
                contentEncoding: base64
                type: string
          description: OK
          headers:
            Content-Type:
              schema:
                type: string
            X-Screenshot-Saved:
              schema:
                description: 'Set when save=true: "true" if the screenshot was stored, otherwise "false".'
                type: string
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Take screenshot
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Deprecated: use `POST /devices/{deviceId}/screenshots` to capture a screenshot. Existing callers can continue using this endpoint and migrate in a follow-up.


          The response body is the PNG image itself (`Content-Type: image/png`), not a JSON wrapper.


          The accessibility overlay is the on-screen element-highlight layer controlled by the set overlay visibility endpoint. If you''ve enabled it, pass `hideOverlay=true` to capture a clean frame; on devices whose capture path sits below the overlay, the flag has no effect either way.


          Pass `save=true` to also store the captured PNG as device media. If the device has an active recording owned by the device owner, the screenshot is saved into that recording and inherits its retention. Otherwise it is saved standalone for seven days. Without `save=true`, no screenshot is stored.


          The PNG response is unchanged; `X-Screenshot-Saved` is `true` when both the object and its metadata were stored, and `false` if storage limits or a save failure prevented persistence. A failed save does not fail screenshot capture. The device screenshots endpoints list saved screenshots, open their images, and delete individual screenshots. Recording screenshots remain available through the existing list recording screenshots endpoint and are also deleted with the recording.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
      - explode: false
        in: query
        name: hideOverlay
        schema:
          default: false
          type: boolean
      - description: 'Store the screenshot as device media: in the owner''s active recording, or standalone for 7 days. Without save=true nothing is stored. The PNG response is unchanged; X-Screenshot-Saved reports whether it was stored.'
        explode: false
        in: query
        name: save
        schema:
          default: false
          description: 'Store the screenshot as device media: in the owner''s active recording, or standalone for 7 days. Without save=true nothing is stored. The PNG response is unchanged; X-Screenshot-Saved reports whether it was stored.'
          type: boolean
    servers:
    - url: https://api.mobilerun.ai
      description: Droidrun Cloud API
  /devices/{deviceId}/swipe:
    post:
      description: Swipes from a start coordinate to an end coordinate over the given duration in milliseconds. The optional humanize flag (best-effort, also with holdMs) adds a human-like path and timing. With holdMs > 0 the request returns 422 when duration + holdMs exceeds 9000 ms, or on devices that do not support it.
      operationId: swipe
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/SwipeArgs'
        required: true
      responses:
        '204':
          description: No Content
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Swipe
      tags:
      - Tools/Navigation
      x-mint:
        content: 'With `humanize: true`, the gesture is humanized: start and end points shift randomly (up to ±5 and ±15 pixels), the duration gets ±10% jitter, and supported devices trace a curved path with micro-jitter instead of a straight line. Devices with human-like input support inject the whole gesture with human motor timing instead.


          A humanized swipe covering less than ~5 pixels is treated as a long press with slight drift.


          Deprecated alias: `stealth`. If both are sent they must agree, otherwise the request is rejected with 422. With `holdMs`, a humanized swipe keeps `duration` (±10%, up to 9000 ms), ends within 1% of the screen of the end point without overshoot, and holds for about `holdMs` before lifting, so it does not fling.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
    servers:
    - url: https://api.mobilerun.ai
      description: Droidrun Cloud API
  /devices/{deviceId}/tap:
    post:
      description: Taps the device screen at the given x/y coordinates. An optional humanize flag routes the tap through human-like input on devices that support it.
      operationId: tap
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/TapByCoordinatesArgs'
        required: true
      responses:
        '204':
          description: No Content
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: Tap by coordinates
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Coordinates are absolute screen pixels — the same space as `boundsInScreen` in the UI state response, so you can read an element''s bounds and tap the center of its box.


          With `humanize: true`, devices that support human-like input inject the tap with human motor timing rather than a plain synthetic event.


          Deprecated alias: `stealth`. If both are sent they must agree, otherwise the request is rejected with 422.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
    servers:
    - url: https://api.mobilerun.ai
      description: Droidrun Cloud API
  /devices/{deviceId}/ui-state:
    get:
      description: Returns the current accessibility UI state of the device as a structured tree of on-screen elements. An optional filter query reduces the result to interactive elements.
      operationId: ui-state
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/AndroidState'
          description: OK
        default:
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorModel'
          description: Error
      summary: UI state
      tags:
      - Tools/Navigation
      x-mint:
        content: 'Alongside the main `a11y_tree`, the response carries a separate `ime_tree` for the on-screen keyboard window, `phone_state` (foreground app, focused element, keyboard visibility), and `device_context` (screen bounds, display density).


          <Tip>

          Element `boundsInScreen` values are in the same pixel space the tap and swipe endpoints use — read an element''s bounds and tap the center of its box for element-driven automation.

          </Tip>


          ## iOS phones


          iOS phones (`ios_phone`) return the same response: the same top-level keys (`a11y_tree`, `ime_tree`, `phone_state`, `device_context`), the same node fields and the same `filter` semantics. The device''s capabilities report `accessibility: true` when it serves the UI state; otherwise this endpoint answers 400 `DEVICE_TOOL_UNSUPPORTED`.


          - Coordinates are iOS logical points, the space the tap and swipe endpoints use on iOS, so the tap-the-center rule above holds unchanged. `device_context.screen_bounds` is in points, and `display_metrics` gives the scale (`density` 3 means `widthPixels` is three times the width in points).

          - `className` uses iOS element types (`Application`, `Button`, `TextField`, `SearchField`, `SecureTextField`, `Switch`, `StaticText`, `Key`, ...), and `resourceId` is the accessibility identifier.

          - `ime_tree` is the software keyboard when it is up, otherwise empty. On phones that cannot read the keyboard separately it stays empty and `keyboardVisible` still reports it.

          - `phone_state.packageName` is the bundle ID and `currentApp` the app''s name. `activityName` is absent: iOS has no activities.

          - `isVisibleToUser` is absent and `isLongClickable` is always `false`: iOS exposes no such signal.

          '
      parameters:
      - in: path
        name: deviceId
        required: true
        schema:
          type: string
      - in: header
        name: X-Device-Display-ID
        schema:
          default: 0
          format: int64
          minimum: 0
          type: integer
      - explode: false
        in: query
        name: filter
        schema:
          default: false
          type: boolean
    servers:
    - url: https://api.mobilerun.ai
      description: Droidrun Cloud API
components:
  schemas:
    AndroidState:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/AndroidState.json
          format: uri
          readOnly: true
          type: string
        a11y_tree:
          $ref: '#/components/schemas/A11YNode'
        device_context:
          $ref: '#/components/schemas/DeviceContext'
        ime_tree:
          $ref: '#/components/schemas/A11YNode'
        phone_state:
          $ref: '#/components/schemas/PhoneState'
      required:
      - a11y_tree
      - ime_tree
      - phone_state
      - device_context
      type: object
    DeviceContext:
      additionalProperties: false
      properties:
        display_metrics:
          $ref: '#/components/schemas/DisplayMetrics'
        filtering_params:
          $ref: '#/components/schemas/FilteringParams'
        screen_bounds:
          $ref: '#/components/schemas/Rect'
      required:
      - screen_bounds
      - filtering_params
      - display_metrics
      type: object
    Rect:
      additionalProperties: false
      properties:
        height:
          format: int64
          type: integer
        width:
          format: int64
          type: integer
      required:
      - width
      - height
      type: object
    A11YBounds:
      additionalProperties: false
      properties:
        bottom:
          format: int64
          type: integer
        left:
          format: int64
          type: integer
        right:
          format: int64
          type: integer
        top:
          format: int64
          type: integer
      required:
      - left
      - top
      - right
      - bottom
      type: object
    TapByCoordinatesArgs:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/TapByCoordinatesArgs.json
          format: uri
          readOnly: true
          type: string
        humanize:
          description: Request human-like input where supported by the device. Defaults to false.
          type: boolean
        stealth:
          deprecated: true
          description: 'Deprecated: use humanize. Both names must agree when supplied together.'
          type: boolean
        x:
          format: int64
          type: integer
        y:
          format: int64
          type: integer
      required:
      - x
      - y
      type: object
    PhoneState:
      additionalProperties: false
      properties:
        activityName:
          type: string
        currentApp:
          type: string
        focusedElement:
          $ref: '#/components/schemas/PhoneStateFocusedElementStruct'
        isEditable:
          type: boolean
        keyboardVisible:
          type: boolean
        packageName:
          type: string
      required:
      - keyboardVisible
      - isEditable
      type: object
    KeyboardKeyArgs:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/KeyboardKeyArgs.json
          format: uri
          readOnly: true
          type: string
        key:
          format: int64
          type: integer
      required:
      - key
      type: object
    A11YNode:
      additionalProperties: false
      properties:
        boundsInScreen:
          $ref: '#/components/schemas/A11YBounds'
        children:
          items:
            $ref: '#/components/schemas/A11YNode'
          type:
          - array
          - 'null'
        className:
          type: string
        contentDescription:
          type: string
        hint:
          type: string
        isCheckable:
          type: boolean
        isChecked:
          type: boolean
        isClickable:
          type: boolean
        isEnabled:
          type: boolean
        isFocusable:
          type: boolean
        isFocused:
          type: boolean
        isLongClickable:
          type: boolean
        isPassword:
          type: boolean
        isScrollable:
          type: boolean
        isSelected:
          type: boolean
        isVisibleToUser:
          type: boolean
        packageName:
          type: string
        resourceId:
          type: string
        text:
          type: string
      required:
      - boundsInScreen
      - className
      - resourceId
      - text
      - contentDescription
      - packageName
      - isCheckable
      - isChecked
      - isClickable
      - isEnabled
      - isFocusable
      - isFocused
      - isScrollable
      - isLongClickable
      - isPassword
      - isSelected
      - children
      type: object
    DisplayMetrics:
      additionalProperties: false
      properties:
        density:
          format: double
          type: number
        densityDpi:
          format: int64
          type: integer
        heightPixels:
          format: int64
          type: integer
        scaledDensity:
          format: double
          type: number
        widthPixels:
          format: int64
          type: integer
      required:
      - density
      - densityDpi
      - scaledDensity
      - widthPixels
      - heightPixels
      type: object
    ErrorModel:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/ErrorModel.json
          format: uri
          readOnly: true
          type: string
        detail:
          description: A human-readable explanation specific to this occurrence of the problem.
          examples:
          - Property foo is required but is missing.
          type: string
        errors:
          description: Optional list of individual error details
          items:
            $ref: '#/components/schemas/ErrorDetail'
          type:
          - array
          - 'null'
        instance:
          description: A URI reference that identifies the specific occurrence of the problem.
          examples:
          - https://example.com/error-log/abc123
          format: uri
          type: string
        status:
          description: HTTP status code
          examples:
          - 400
          format: int64
          type: integer
        title:
          description: A short, human-readable summary of the problem type. This value should not change between occurrences of the error.
          examples:
          - Bad Request
          type: string
        type:
          default: about:blank
          description: A URI reference to human-readable documentation for the error.
          examples:
          - https://example.com/errors/example
          format: uri
          type: string
      type: object
    ErrorDetail:
      additionalProperties: false
      properties:
        location:
          description: Where the error occurred, e.g. 'body.items[3].tags' or 'path.thing-id'
          type: string
        message:
          description: Error message text
          type: string
        value:
          description: The value at the given location
      type: object
    FilteringParams:
      additionalProperties: false
      properties:
        min_element_size:
          format: int64
          type: integer
        overlay_offset:
          format: int64
          type: integer
      required:
      - min_element_size
      - overlay_offset
      type: object
    GlobalArgs:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/GlobalArgs.json
          format: uri
          readOnly: true
          type: string
        action:
          format: int64
          type: integer
      required:
      - action
      type: object
    KeyboardInputArgs:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/KeyboardInputArgs.json
          format: uri
          readOnly: true
          type: string
        clear:
          default: false
          type: boolean
        completionMode:
          default: accepted
          description: Completion guarantee. accepted returns after the input provider accepts the operation; committed additionally waits for the focused UI state to contain the complete text or become quiescent.
          enum:
          - accepted
          - committed
          type: string
        errorRate:
          default: -1
          description: Per-character mistake rate for humanized typing. -1 uses server default.
          format: double
          type: number
        humanize:
          description: Request human-like input where supported by the device. Defaults to false.
          type: boolean
        stealth:
          deprecated: true
          description: 'Deprecated: use humanize. Both names must agree when supplied together.'
          type: boolean
        text:
          type: string
        wpm:
          default: 0
          description: Words per minute for humanized typing. 0 uses the device default.
          format: int64
          type: integer
      required:
      - text
      type: object
    SwipeArgs:
      additionalProperties: false
      properties:
        $schema:
          description: A URL to the JSON Schema for this object.
          examples:
          - https://example.com/schemas/SwipeArgs.json
          format: uri
          readOnly: true
          type: string
        duration:
          description: Swipe duration in milliseconds
          format: int64
          minimum: 10
          type: integer
        endX:
          format: int64
          type: integer
        endY:
          format: int64
          type: integer
        holdMs:
          default: 0
          description: Keep the finger at the end point for holdMs and suppress fling before lifting, so content moves by about the finger travel. Supported on Android cloud phones.
          format: int64
          maximum: 5000
          minimum: 0
          type: integer
        humanize:
          description: Request human-like input where supported by the device. Defaults to false.
          type: boolean
        startX:
          format: int64
          type: integer
        startY:
          format: int64
          type: integer
        stealth:
          deprecated: true
          description: 'Deprecated: use humanize. Both names must agree when supplied together.'
          type: boolean
      required:
      - startX
      - startY
      - endX
      - endY
      - duration
      type: object
    PhoneStateFocusedElementStruct:
      additionalProperties: false
      properties:
        className:
          type: string
        resourceId:
          type: string
        text:
          type: string
      type: object
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: Opaque
      description: Bearer token via Authorization header
x-refined-from:
- openapi.json
- mobilerun-ai-openapi.json
x-provenance:
  method: harvested
  publisher: Mobilerun
  source: https://api.mobilerun.ai/v1/openapi.json
  note: split by refine-openapis from the provider's own published document, verified by fetch (openapi/_original/*.fetch.json);
    stamped by stamp-openapi-provenance.py (roadmap#793)