Apache Helix website screenshot

Apache Helix

Apache Helix is a generic cluster management framework for partitioned and replicated distributed resources. It automates partition management, replication, fault tolerance, and cluster expansion for distributed systems, providing a REST API for cluster administration and a Java API for participant, spectator, and controller roles.

2 APIs 8 Features
ApacheCluster ManagementDistributed SystemsOpen SourcePartitioningReplication

APIs

Apache Helix REST API

REST API for managing Apache Helix clusters, instances, resources, and partition state assignments, including ideal state queries and external view inspection.

Apache Helix Java API

Java API for implementing Helix participant, spectator, and controller roles, with APIs for resource management, task execution, and state machine definitions.

Collections

Apache Helix REST API

OPEN

Pricing Plans

Apache Helix Plans Pricing

3 plans

PLANS

Rate Limits

Apache Helix Rate Limits

5 limits

RATE LIMITS

FinOps

Apache Helix Finops

FINOPS

Features

Automatic Partition Management

Automatically assign and balance partitions across cluster nodes using pluggable rebalancer algorithms.

State Machine Framework

Define custom resource state machines (e.g., Master-Slave, Leader-Standby) for any distributed service.

Fault Tolerance

Detect node failures and automatically reassign partitions to maintain replication targets.

REST API

HTTP REST API for cluster administration, resource management, and state inspection.

Task Framework

Distributed task scheduling framework for batch jobs and recurring workflows with failure handling.

ZooKeeper Integration

Uses Apache ZooKeeper as the distributed coordination backend for cluster state storage.

Spectator API

Read-only API for external services to observe resource state and routing decisions.

Cloud-Aware Rebalancing

Rack and zone-aware partition placement for fault-domain isolation in cloud environments.

Use Cases

Distributed Database Cluster Management

Manage shard assignment and replication for distributed databases like DistributedLog or Espresso.

Search Index Partition Management

Automatically balance and assign search index shards across a cluster of query servers.

Distributed Task Scheduling

Schedule and execute distributed batch tasks with automatic retry and failure recovery.

Microservices Load Balancing

Use Helix spectator API to implement client-side load balancing based on partition state.

Stateful Service Migration

Perform rolling upgrades and partition migrations without service downtime.

Integrations

Apache ZooKeeper

ZooKeeper is the required coordination backend for Helix cluster state management.

Apache Kafka

Helix is used internally by some Kafka ecosystem projects for partition management.

LinkedIn Pinot

Apache Pinot uses Helix for real-time OLAP cluster partition and segment management.

LinkedIn Venice

Venice feature store uses Helix for managing data store partition assignments.

Semantic Vocabularies

Apache Helix Rest Context

16 classes · 0 properties

JSON-LD

API Governance Rules

Apache Helix API Rules

8 rules · 2 errors 5 warnings 1 info

SPECTRAL

JSON Structure

Helix Rest Cluster Structure

5 properties

JSON STRUCTURE

Helix Rest Externalview Structure

2 properties

JSON STRUCTURE

Helix Rest Idealstate Structure

3 properties

JSON STRUCTURE

Helix Rest Instance Structure

5 properties

JSON STRUCTURE

Helix Rest Partition Structure

2 properties

JSON STRUCTURE

Helix Rest Resource Structure

5 properties

JSON STRUCTURE

Example Payloads

Helix Rest Cluster Example

5 fields

EXAMPLE

Helix Rest Externalview Example

2 fields

EXAMPLE

Helix Rest Idealstate Example

3 fields

EXAMPLE

Helix Rest Instance Example

5 fields

EXAMPLE

Helix Rest Partition Example

2 fields

EXAMPLE

Helix Rest Resource Example

5 fields

EXAMPLE

Resources

🔗
Documentation
Documentation
🚀
GettingStarted
GettingStarted
👥
GitHubOrganization
GitHubOrganization
👥
GitHubRepository
GitHubRepository
🔗
SpectralRules
SpectralRules
🔗
Vocabulary
Vocabulary

Sources

Raw ↑ 
opencollection: 1.0.0
info:
  name: Apache Helix REST API
  version: 1.0.0
items:
- info:
    name: Clusters
    type: folder
  items:
  - info:
      name: Apache Helix List Clusters
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters
    docs: List all Helix clusters managed by this controller.
  - info:
      name: Apache Helix Create Cluster
      type: http
    http:
      method: POST
      url: http://localhost:9100/clusters
      body:
        type: json
        data: '{}'
    docs: Create a new Helix cluster.
  - info:
      name: Apache Helix Get Cluster
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId
      params:
      - name: clusterId
        value: ''
        type: path
        description: Cluster name
    docs: Get configuration and status of a specific Helix cluster.
  - info:
      name: Apache Helix Delete Cluster
      type: http
    http:
      method: DELETE
      url: http://localhost:9100/clusters/:clusterId
      params:
      - name: clusterId
        value: ''
        type: path
    docs: Delete a Helix cluster and all its associated ZooKeeper data.
- info:
    name: Instances
    type: folder
  items:
  - info:
      name: Apache Helix List Instances
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId/instances
      params:
      - name: clusterId
        value: ''
        type: path
    docs: List all instances (participants) registered in a Helix cluster.
  - info:
      name: Apache Helix Get Instance
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId/instances/:instanceName
      params:
      - name: clusterId
        value: ''
        type: path
      - name: instanceName
        value: ''
        type: path
        description: Instance name (e.g. localhost_12913)
    docs: Get configuration and state of a specific Helix instance.
- info:
    name: Resources
    type: folder
  items:
  - info:
      name: Apache Helix List Resources
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId/resources
      params:
      - name: clusterId
        value: ''
        type: path
    docs: List all resources managed in a Helix cluster.
  - info:
      name: Apache Helix Get Resource
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId/resources/:resourceName
      params:
      - name: clusterId
        value: ''
        type: path
      - name: resourceName
        value: ''
        type: path
    docs: Get configuration and ideal state of a specific Helix resource.
- info:
    name: State
    type: folder
  items:
  - info:
      name: Apache Helix Get External View
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId/resources/:resourceName/externalView
      params:
      - name: clusterId
        value: ''
        type: path
      - name: resourceName
        value: ''
        type: path
    docs: Get the current external view showing actual partition state assignments across instances.
  - info:
      name: Apache Helix Get Ideal State
      type: http
    http:
      method: GET
      url: http://localhost:9100/clusters/:clusterId/resources/:resourceName/idealState
      params:
      - name: clusterId
        value: ''
        type: path
      - name: resourceName
        value: ''
        type: path
    docs: Get the ideal state (desired partition placement) for a Helix resource.
bundled: true