Mage

Mage is an open-source data pipeline tool for building, running, and managing data pipelines from transformer, data loader, and data exporter blocks. The self-hosted Mage app exposes a REST API for triggering pipeline runs and managing pipelines, blocks, pipeline runs, and schedules; Mage Pro is the managed cloud edition.

4 APIs 0 Features

Data PipelinesOrchestrationETLData EngineeringOpen Source

APIs

Mage Pipeline Triggers & Runs API

Trigger a configured API trigger by POSTing to its pipeline schedule to create a pipeline run, optionally passing runtime variables, and read the status of resulting pipeline runs.

Mage Pipelines API

List, create, read, update, and delete pipelines in a Mage project using the standard /api/pipelines REST resource with api_key and oauth_token authentication.

Mage Blocks API

Create, read, update, and delete the data loader, transformer, and data exporter blocks that make up a pipeline, nested under the parent pipeline resource.

Mage Pipeline Schedules API

Create, read, and update pipeline schedules (triggers) - including schedule, event, and API trigger types - that govern when and how pipelines run.

Collections

Mage API

OPEN

Pricing Plans

Mage Ai Plans Pricing

5 plans

PLANS

Rate Limits

Mage Ai Rate Limits

4 limits

RATE LIMITS

FinOps

Mage Ai Finops

FINOPS

Resources

👥

GitHubOrganization

Sources

opencollection: 1.0.0
info:
  name: Mage API
  version: '1.0'
request:
  auth:
    type: apikey
    key: api_key
    value: '{{apiKey}}'
    in: query
items:
- info:
    name: Pipeline Triggers & Runs
    type: folder
  items:
  - info:
      name: Trigger a pipeline run via an API trigger
      type: http
    http:
      method: POST
      url: http://localhost:6789/api/pipeline_schedules/:schedule_id/pipeline_runs/:token
      params:
      - name: schedule_id
        value: '3'
        type: path
        description: The unique ID of the pipeline schedule (API trigger).
      - name: token
        value: abc123
        type: path
        description: The unique token associated with the API trigger.
      body:
        type: json
        data: "{\n  \"pipeline_run\": {\n    \"variables\": {\n      \"env\": \"staging\",\n      \"schema\": \"public\"\n\
          \    }\n  }\n}"
    docs: Creates a pipeline run for the API trigger identified by schedule_id and token. Runtime variables are optional.
  - info:
      name: List pipeline runs
      type: http
    http:
      method: GET
      url: http://localhost:6789/api/pipeline_runs
    docs: Lists pipeline runs.
  - info:
      name: Read a pipeline run
      type: http
    http:
      method: GET
      url: http://localhost:6789/api/pipeline_runs/:id
      params:
      - name: id
        value: '1'
        type: path
        description: The pipeline run ID.
    docs: Reads a single pipeline run.
- info:
    name: Pipelines
    type: folder
  items:
  - info:
      name: List pipelines
      type: http
    http:
      method: GET
      url: http://localhost:6789/api/pipelines
    docs: Lists pipelines.
  - info:
      name: Create a pipeline
      type: http
    http:
      method: POST
      url: http://localhost:6789/api/pipelines
      body:
        type: json
        data: "{\n  \"pipeline\": {\n    \"name\": \"arwen-starlight\",\n    \"type\": \"python\"\n  }\n}"
    docs: Creates a pipeline.
  - info:
      name: Read a pipeline
      type: http
    http:
      method: GET
      url: http://localhost:6789/api/pipelines/:uuid
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
    docs: Reads a single pipeline.
  - info:
      name: Update a pipeline
      type: http
    http:
      method: PUT
      url: http://localhost:6789/api/pipelines/:uuid
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
      body:
        type: json
        data: "{\n  \"pipeline\": {\n    \"name\": \"arwen-starlight\"\n  }\n}"
    docs: Updates a pipeline.
  - info:
      name: Delete a pipeline
      type: http
    http:
      method: DELETE
      url: http://localhost:6789/api/pipelines/:uuid
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
    docs: Deletes a pipeline.
- info:
    name: Blocks
    type: folder
  items:
  - info:
      name: Create a block in a pipeline
      type: http
    http:
      method: POST
      url: http://localhost:6789/api/pipelines/:uuid/blocks
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
      body:
        type: json
        data: "{\n  \"block\": {\n    \"name\": \"load_data\",\n    \"type\": \"data_loader\",\n    \"language\": \"python\"\
          \n  }\n}"
    docs: Creates a block in a pipeline.
  - info:
      name: Read a block in a pipeline
      type: http
    http:
      method: GET
      url: http://localhost:6789/api/pipelines/:uuid/blocks/:block_uuid
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
      - name: block_uuid
        value: load_data
        type: path
        description: The block UUID.
    docs: Reads a single block in a pipeline.
  - info:
      name: Update a block in a pipeline
      type: http
    http:
      method: PUT
      url: http://localhost:6789/api/pipelines/:uuid/blocks/:block_uuid
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
      - name: block_uuid
        value: load_data
        type: path
        description: The block UUID.
      body:
        type: json
        data: "{\n  \"block\": {\n    \"content\": \"# updated block code\"\n  }\n}"
    docs: Updates a block in a pipeline.
  - info:
      name: Delete a block from a pipeline
      type: http
    http:
      method: DELETE
      url: http://localhost:6789/api/pipelines/:uuid/blocks/:block_uuid
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
      - name: block_uuid
        value: load_data
        type: path
        description: The block UUID.
    docs: Deletes a block from a pipeline.
- info:
    name: Pipeline Schedules
    type: folder
  items:
  - info:
      name: List pipeline schedules
      type: http
    http:
      method: GET
      url: http://localhost:6789/api/pipeline_schedules
    docs: Lists pipeline schedules (triggers).
  - info:
      name: Create a pipeline schedule for a pipeline
      type: http
    http:
      method: POST
      url: http://localhost:6789/api/pipelines/:uuid/pipeline_schedules
      params:
      - name: uuid
        value: arwen_starlight
        type: path
        description: The pipeline UUID.
      body:
        type: json
        data: "{\n  \"pipeline_schedule\": {\n    \"name\": \"daily run\",\n    \"schedule_type\": \"time\",\n    \"schedule_interval\"\
          : \"@daily\"\n  }\n}"
    docs: Creates a pipeline schedule (trigger) for a pipeline.
  - info:
      name: Update a pipeline schedule
      type: http
    http:
      method: PUT
      url: http://localhost:6789/api/pipeline_schedules/:id
      params:
      - name: id
        value: '3'
        type: path
        description: The pipeline schedule ID.
      body:
        type: json
        data: "{\n  \"pipeline_schedule\": {\n    \"status\": \"active\"\n  }\n}"
    docs: Updates a pipeline schedule.
bundled: true