Apache Livy

Apache Livy is a service that enables easy interaction with a Spark cluster over a REST interface. It allows submitting Spark jobs or snippets of Spark code, retrieving results synchronously or asynchronously, and managing Spark contexts across multiple users. Licensed under Apache 2.0.

Apache Livy publishes 3 APIs on the APIs.io network: Batches API, Sessions API, and Statements API. Tagged areas include Big Data, Interactive Computing, Open Source, REST, and Spark.

The Apache Livy catalog on APIs.io includes 1 JSON-LD context and 2 Spectral governance rulesets.

Apache Livy’s developer surface includes documentation, getting-started guide, and 9 more developer resources.

🌐 Visit website 📡 Source on GitHub

49.7/100 developing ▼ -5.0 Agent 28/100 agent aware Full breakdown ↓
scored 2026-07-28 · rubric v0.6

AccessFreemium

3 APIs 7 Features 4 Use Cases

Big DataInteractive ComputingOpen SourceRESTSpark

Kin Score

Kin Score How this is scored →
scored 2026-07-28 · rubric v0.6

Composite quality — 49.7/100 · developing

Contract Quality 14.2 / 25

Developer Ergonomics 3.9 / 20

Commercial Clarity 10.0 / 20

Operational Transparency 6.8 / 13

Governance 8.3 / 12

Discoverability 6.5 / 10

Agent readiness — 28/100 · agent aware

Machine-Readable Contract 18 / 18

Agentic Access Contract 10 / 10

MCP Server 0 / 12

Machine-Readable Auth 0 / 10

Idempotency 0 / 9

Stable Error Semantics 0 / 8

Request/Response Examples 7 / 7

Rate-Limit Signaling 7 / 7

Typed Event Surface 0 / 6

Agent Skills 0 / 5

Well-Known Catalog 0 / 4

Consent & Bot Identity 0 / 3

A2A Agent Card 0 / 8

Dry-Run / Simulate Mode 0 / 4

Improve this rating by publishing the missing artifacts — every area above can be raised, and the full rubric is at apis.io/rating/. This rating is computed from github.com/api-evangelist/apache-livy: open an issue to ask a question, or submit a pull request to add artifacts. Want it done for you? Prioritized profiling — $2,500 →

APIs 3

Individual APIs this provider publishes, each with its own machine-readable definition.

Rate Limits 1

Documented rate limits and quota policies.

Apache Livy Rate Limits

5 limits

RATE LIMITS

FinOps 1

Cost, billing, and metering signals for API financial operations.

Apache Livy Finops

FINOPS

Features 7

Notable capabilities this provider offers.

Interactive Spark Sessions

Create persistent Spark contexts for interactive code execution in Python, Scala, R, and SQL.

Batch Job Submission

Submit batch Spark jobs without creating an interactive session.

Multi-Language Support

Execute code in PySpark, Spark (Scala), SparkR, and SQL.

Multi-User Impersonation

Proxy user support for multi-tenant Spark cluster access.

Asynchronous Execution

Submit jobs and poll for results asynchronously.

Log Access

Retrieve driver and executor logs for debugging.

REST Interface

Simple HTTP REST API for Spark cluster interaction without native clients.

Scroll for all 7

JSON Schema 12

Standalone JSON Schema definitions for this provider's data models.

Scroll for all 12

JSON Structure 12

JSON Structure definitions describing this provider's data shapes.

Scroll for all 12

Examples 12

Example request and response payloads for these APIs.

Scroll for all 12

Use Cases 4

What developers build with this provider.

Notebook Integration

Power Jupyter, Zeppelin, and other notebooks with Spark backends via Livy.

Data Engineering Pipelines

Submit Spark batch jobs from orchestration tools like Airflow and Oozie.

Interactive Data Exploration

Execute ad-hoc Spark code for exploratory data analysis.

Multi-Tenant Spark Access

Enable multiple users to share a Spark cluster with isolation via Livy sessions.

Integrations 5

Pre-built integrations with other platforms and tools.

Apache Spark

Livy requires a Spark cluster and acts as the REST gateway to Spark.

Apache Zeppelin

Zeppelin notebook backend using Livy for distributed Spark execution.

Jupyter Notebook

Jupyter sparkmagic extension uses Livy for remote Spark kernel access.

Apache Airflow

Airflow LivyOperator for submitting Spark batch jobs from DAGs.

Amazon EMR

Livy is available as an EMR application for REST-based Spark access.

Resources

Get Started 1

Portal, sign-up, and the first successful call

GettingStarted

GettingStarted

Documentation 1

Reference material describing how the API behaves

Documentation

Documentation

Agent Surfaces 1

MCP servers, agent skills, and machine-readable catalogs

AgenticAccess

AgenticAccess

Design & Contract 3

Pagination, idempotency, versioning, errors, and events

Versioning

Versioning

SpectralRules

SpectralRules

Vocabulary

Vocabulary

Build 2

SDKs, sample code, and the tooling you integrate with

GitHubOrganization

GitHubOrganization

GitHubRepository

GitHubRepository

Access & Security 2

Authentication, authorization, and security posture

VulnerabilityDisclosure

VulnerabilityDisclosure

DomainSecurity

DomainSecurity

Commercial 1

Pricing, plans, and the legal terms of use

TermsOfService

TermsOfService

Source (apis.yml)

aid: apache-livy
name: Apache Livy
description: Apache Livy is a service that enables easy interaction with a Spark cluster over a REST interface. It allows
  submitting Spark jobs or snippets of Spark code, retrieving results synchronously or asynchronously, and managing Spark
  contexts across multiple users. Licensed under Apache 2.0.
type: Index
accessModel:
  pricing: freemium
  onboarding: unknown
  trial: false
  try_now: false
  public: false
  label: Freemium
  confidence: medium
  source:
  - plans
  generated: '2026-07-22'
  method: derived
position: Consumer
access: 3rd-Party
image: https://kinlane-images.s3.amazonaws.com/shared/apis-json/icons/apache-livy.png
tags:
- Big Data
- Interactive Computing
- Open Source
- REST
- Spark
created: '2026-03-16'
modified: '2026-05-19'
url: https://raw.githubusercontent.com/api-evangelist/apache-livy/refs/heads/main/apis.yml
specificationVersion: '0.19'
apis:
- aid: apache-livy:apache-livy-batches-api
  name: Apache Livy Batches API
  description: Batch Spark job submission
  humanURL: https://livy.apache.org/docs/latest/rest-api.html
  tags:
  - Batches
  properties:
  - type: OpenAPI
    url: openapi/apache-livy-batches-api-openapi.yml
  - type: Documentation
    url: https://livy.apache.org/docs/latest/rest-api.html
- aid: apache-livy:apache-livy-sessions-api
  name: Apache Livy Sessions API
  description: Interactive Spark session management
  humanURL: https://livy.apache.org/docs/latest/rest-api.html
  tags:
  - Sessions
  properties:
  - type: OpenAPI
    url: openapi/apache-livy-sessions-api-openapi.yml
  - type: Documentation
    url: https://livy.apache.org/docs/latest/rest-api.html
- aid: apache-livy:apache-livy-statements-api
  name: Apache Livy Statements API
  description: Code statement execution within sessions
  humanURL: https://livy.apache.org/docs/latest/rest-api.html
  tags:
  - Statements
  properties:
  - type: OpenAPI
    url: openapi/apache-livy-statements-api-openapi.yml
  - type: Documentation
    url: https://livy.apache.org/docs/latest/rest-api.html
common:
- type: AgenticAccess
  url: agentic-access/apache-livy-agentic-access.yml
- type: VulnerabilityDisclosure
  url: security/apache-livy-vulnerability-disclosure.yml
- type: DomainSecurity
  url: security/apache-livy-domain-security.yml
- type: GitHubOrganization
  url: https://github.com/apache
- type: GitHubRepository
  url: https://github.com/apache/incubator-livy
- type: Documentation
  url: https://livy.apache.org/docs/latest/
- type: GettingStarted
  url: https://livy.apache.org/get-started/
- type: TermsOfService
  url: https://www.apache.org/licenses/LICENSE-2.0
- type: Versioning
  url: https://livy.apache.org/download/
- type: SpectralRules
  url: rules/apache-livy-spectral-rules.yml
- type: Vocabulary
  url: vocabulary/apache-livy-vocabulary.yaml
- type: Features
  data:
  - name: Interactive Spark Sessions
    description: Create persistent Spark contexts for interactive code execution in Python, Scala, R, and SQL.
  - name: Batch Job Submission
    description: Submit batch Spark jobs without creating an interactive session.
  - name: Multi-Language Support
    description: Execute code in PySpark, Spark (Scala), SparkR, and SQL.
  - name: Multi-User Impersonation
    description: Proxy user support for multi-tenant Spark cluster access.
  - name: Asynchronous Execution
    description: Submit jobs and poll for results asynchronously.
  - name: Log Access
    description: Retrieve driver and executor logs for debugging.
  - name: REST Interface
    description: Simple HTTP REST API for Spark cluster interaction without native clients.
- type: UseCases
  data:
  - name: Notebook Integration
    description: Power Jupyter, Zeppelin, and other notebooks with Spark backends via Livy.
  - name: Data Engineering Pipelines
    description: Submit Spark batch jobs from orchestration tools like Airflow and Oozie.
  - name: Interactive Data Exploration
    description: Execute ad-hoc Spark code for exploratory data analysis.
  - name: Multi-Tenant Spark Access
    description: Enable multiple users to share a Spark cluster with isolation via Livy sessions.
- type: Integrations
  data:
  - name: Apache Spark
    description: Livy requires a Spark cluster and acts as the REST gateway to Spark.
  - name: Apache Zeppelin
    description: Zeppelin notebook backend using Livy for distributed Spark execution.
  - name: Jupyter Notebook
    description: Jupyter sparkmagic extension uses Livy for remote Spark kernel access.
  - name: Apache Airflow
    description: Airflow LivyOperator for submitting Spark batch jobs from DAGs.
  - name: Amazon EMR
    description: Livy is available as an EMR application for REST-based Spark access.
maintainers:
- FN: Kin Lane
  email: info@apievangelist.com

Apache Livy

APIs 3

Pricing Plans 1

Rate Limits 1

FinOps 1

Features 7

Interactive Spark Sessions

Batch Job Submission

Multi-Language Support

Multi-User Impersonation

Asynchronous Execution

Log Access

REST Interface

Semantic Vocabularies 1

Spectral Rules 2

JSON Schema 12

JSON Structure 12

Examples 12

Security Posture 2

Agentic Access 1

Use Cases 4

Notebook Integration

Data Engineering Pipelines

Interactive Data Exploration

Multi-Tenant Spark Access

Integrations 5

Apache Spark

Apache Zeppelin

Jupyter Notebook

Apache Airflow

Amazon EMR

Get Started 1

Documentation 1

Agent Surfaces 1

Design & Contract 3

Build 2

Access & Security 2

Commercial 1

Source (apis.yml)