Apache Kafka

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. It provides a REST Proxy API, Kafka Connect REST API, and AsyncAPI for event streaming.

3 APIs 7 Features

Distributed SystemsEvent StreamingMessagingOpen SourcePub-Sub

APIs

Kafka REST Proxy API

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster for producing and consuming messages, managing topics, partitions, consumer groups, and viewing cluster stat...

Kafka Connect REST API

Kafka Connect REST API for managing connectors, their configurations, tasks, and offsets for integrating Kafka with external data systems including databases, object stores, and...

Apache Kafka Messaging API

The core Kafka messaging protocol for producing and consuming records to/from topics using the native Kafka binary protocol, supporting exactly-once semantics, compaction, and p...

Collections

Kafka Connect REST API

OPEN

Confluent Kafka REST Proxy API

OPEN

Pricing Plans

Apache Kafka Plans Pricing

3 plans

PLANS

Rate Limits

Apache Kafka Rate Limits

5 limits

RATE LIMITS

FinOps

Apache Kafka Finops

FINOPS

Features

High Throughput

Handle millions of messages per second with low latency at massive scale.

Exactly-Once Semantics

Guarantee exactly-once message delivery with idempotent producers and transactional APIs.

Distributed Replication

Automatic replication across brokers for fault tolerance and high availability.

Stream Processing

Real-time stream processing via Kafka Streams library and KSQL.

Connector Ecosystem

200+ pre-built Kafka Connect connectors for databases, clouds, and SaaS.

Log Compaction

Retain the latest value for each key with topic log compaction.

Consumer Groups

Horizontally scalable consumers with automatic partition rebalancing.

Use Cases

Event-Driven Architecture

Build event-driven microservices with reliable message delivery.

Data Pipeline

Move data between systems at scale with exactly-once delivery guarantees.

Real-Time Analytics

Process and analyze event streams in real time with Kafka Streams.

Log Aggregation

Centralize application and infrastructure logs for analysis and alerting.

CDC (Change Data Capture)

Capture database changes and stream them to data warehouses and caches.

Integrations

Apache Spark

Spark Structured Streaming integration for batch and streaming analytics.

Apache Flink

Native Flink Kafka connector for low-latency stream processing.

Debezium

CDC platform using Kafka Connect to capture database change events.

Elasticsearch

Kafka Connect Elasticsearch sink for indexing event data.

Amazon S3

Kafka Connect S3 sink for archiving event streams to object storage.

Apache Hadoop

HDFS sink connector for streaming data into Hadoop data lake.

Event Specifications

Apache Kafka Messaging API

Apache Kafka is a distributed event streaming platform capable of handling trillions of events a day. This spec describes the core messaging protocol for producing and consuming...

Apache Kafka API Rules

16 rules · 9 errors 3 warnings 4 info

👥

GitHubOrganization

Sources

opencollection: 1.0.0
info:
  name: Confluent Kafka REST Proxy API
  version: 3.0.0
items:
- info:
    name: Cluster
    type: folder
  items:
  - info:
      name: Apache kafka List Clusters
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters
    docs: Apache kafka List Clusters
  - info:
      name: Apache kafka Get Cluster
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id
      params:
      - name: cluster_id
        value: item-6938
        type: path
    docs: Apache kafka Get Cluster
- info:
    name: Topic
    type: folder
  items:
  - info:
      name: Apache kafka List Topics
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/topics
      params:
      - name: cluster_id
        value: item-8966
        type: path
    docs: Apache kafka List Topics
  - info:
      name: Apache kafka Create Topic
      type: http
    http:
      method: POST
      url: http://localhost:8082/clusters/:cluster_id/topics
      params:
      - name: cluster_id
        value: item-1233
        type: path
      body:
        type: json
        data: '{}'
    docs: Apache kafka Create Topic
  - info:
      name: Apache kafka Get Topic
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/topics/:topic_name
      params:
      - name: cluster_id
        value: item-8264
        type: path
      - name: topic_name
        value: example-name
        type: path
    docs: Apache kafka Get Topic
  - info:
      name: Apache kafka Delete Topic
      type: http
    http:
      method: DELETE
      url: http://localhost:8082/clusters/:cluster_id/topics/:topic_name
      params:
      - name: cluster_id
        value: item-9796
        type: path
      - name: topic_name
        value: example-name
        type: path
    docs: Apache kafka Delete Topic
- info:
    name: Partition
    type: folder
  items:
  - info:
      name: Apache kafka List Partitions
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/topics/:topic_name/partitions
      params:
      - name: cluster_id
        value: item-7679
        type: path
      - name: topic_name
        value: example-name
        type: path
    docs: Apache kafka List Partitions
- info:
    name: Consumer Group
    type: folder
  items:
  - info:
      name: Apache kafka List Consumer Groups
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/consumer-groups
      params:
      - name: cluster_id
        value: item-6701
        type: path
    docs: Apache kafka List Consumer Groups
  - info:
      name: Apache kafka Get Consumer Group
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/consumer-groups/:consumer_group_id
      params:
      - name: cluster_id
        value: item-8506
        type: path
      - name: consumer_group_id
        value: item-4788
        type: path
    docs: Apache kafka Get Consumer Group
- info:
    name: Broker
    type: folder
  items:
  - info:
      name: Apache kafka List Brokers
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/brokers
      params:
      - name: cluster_id
        value: item-2365
        type: path
    docs: Apache kafka List Brokers
- info:
    name: ACL
    type: folder
  items:
  - info:
      name: Apache kafka Search ACLs
      type: http
    http:
      method: GET
      url: http://localhost:8082/clusters/:cluster_id/acls
      params:
      - name: cluster_id
        value: item-5726
        type: path
      - name: resource_type
        value: UNKNOWN
        type: query
      - name: pattern_type
        value: UNKNOWN
        type: query
    docs: Apache kafka Search ACLs
  - info:
      name: Apache kafka Create ACLs
      type: http
    http:
      method: POST
      url: http://localhost:8082/clusters/:cluster_id/acls
      params:
      - name: cluster_id
        value: item-7456
        type: path
      body:
        type: json
        data: '{}'
    docs: Apache kafka Create ACLs
  - info:
      name: Apache kafka Delete ACLs
      type: http
    http:
      method: DELETE
      url: http://localhost:8082/clusters/:cluster_id/acls
      params:
      - name: cluster_id
        value: item-2190
        type: path
    docs: Apache kafka Delete ACLs
- info:
    name: Records
    type: folder
  items:
  - info:
      name: Apache kafka Produce Records to a Topic
      type: http
    http:
      method: POST
      url: http://localhost:8082/topics/:topic_name/records
      params:
      - name: topic_name
        value: example-name
        type: path
      body:
        type: json
        data: '{}'
    docs: Apache kafka Produce Records to a Topic
bundled: true