Table Format · Controlled Vocabulary

Table Format Vocabulary

Domain vocabulary and taxonomy for open table formats used in data lakehouses, covering Apache Iceberg, Delta Lake, Apache Hudi, and related catalog and storage concepts.

19 Terms Updated 2026-05-03
View Vocabulary File View on GitHub

Table Format Vocabulary is a controlled vocabulary published by Table Format on the APIs.io network, defining 19 terms.

Defined terms include Open Table Format, Apache Iceberg, Delta Lake, Apache Hudi, and Snapshot.

A controlled vocabulary pins the words an API’s schemas, parameters and documentation use to one agreed definition each — the shared language a human or an agent needs to read the API the same way its authors do.

Terms

Open Table Format
A specification that defines how tabular data is organized, tracked, and accessed in object storage. Open table formats add ACID transactions, schema evolution, time travel, and efficient query planning to data lakes without requiring a dedicated storage engine. The three dominant formats are Apache Iceberg, Delta Lake, and Apache Hudi.
Apache Iceberg
An open table format specification originally developed at Netflix, now an Apache Software Foundation top-level project. Iceberg uses a hierarchical tree of metadata files (metadata.json → manifests → data files) for snapshot isolation, schema evolution, partition evolution, and hidden partitioning. It has become the dominant open table format.
Delta Lake
An open-source storage layer developed by Databricks that adds ACID transactions to data lakes. Delta Lake maintains a delta log (JSON-based transaction log) recording all changes. It is deeply integrated with Apache Spark and supports both batch and streaming workloads.
Apache Hudi
An open table format (Hadoop Upserts Deletes and Incrementals) optimized for record-level upserts and deletes. Hudi supports two table types: Copy-on-Write (COW) for read-optimized workloads and Merge-on-Read (MOR) for write-optimized workloads. Popular for CDC-based data pipelines.
Snapshot
An immutable, point-in-time view of an Iceberg table's state. Each write operation creates a new snapshot. Snapshots enable time travel queries and concurrent reads without locking. Each snapshot references a manifest list file that points to all data files in that version of the table.
Manifest File
An Avro file that tracks a subset of data files in an Iceberg table snapshot. Manifests record file-level statistics (row counts, min/max values per column) enabling partition pruning and predicate pushdown without reading data files.
Manifest List
An Avro file referenced by each Iceberg snapshot that lists all manifest files for that snapshot. The manifest list enables efficient snapshot management and incremental metadata operations.
Catalog
A service that tracks table locations, schema versions, and metadata file locations. Catalogs provide the mapping from table name to metadata file. Common Iceberg catalog implementations include Apache Polaris, Project Nessie, AWS Glue, Hive Metastore, and JDBC catalogs.
REST Catalog
A standard HTTP/REST API specification for Iceberg catalog operations, defined as an OpenAPI spec in the Apache Iceberg project. REST catalog allows any catalog service (Polaris, Nessie, BigLake, Glue) to expose a common interface, enabling catalog interoperability.
Schema Evolution
The ability to add, drop, rename, or reorder columns in a table without rewriting all data files. Iceberg, Delta Lake, and Hudi all support schema evolution with backward and forward compatibility guarantees.
Partition Evolution
The ability to change a table's partitioning strategy without rewriting existing data. Apache Iceberg supports partition evolution, allowing old data to use the original partitioning while new data uses the updated partitioning scheme.
Hidden Partitioning
Apache Iceberg's approach where partition transforms (e.g., day(timestamp), bucket(user_id, 16)) are applied automatically by the engine, without requiring users to add separate partition columns. This eliminates common data engineering mistakes.
Time Travel
The ability to query historical versions of a table by specifying a snapshot ID or timestamp. All three major table formats support time travel, enabling auditing, rollback, and reproducible analytics.
Copy-on-Write (COW)
A table storage strategy where updates and deletes rewrite entire data files, producing clean Parquet files optimized for read performance. Used in Hudi and Iceberg for read-heavy workloads.
Merge-on-Read (MOR)
A table storage strategy where updates and deletes are recorded as delta files (log files or delete files) that are merged with base files during reads. Optimizes write performance at the cost of additional read-time merging.
Delta Log
The transaction log used by Delta Lake, stored as a sequence of JSON files in the _delta_log/ directory. Each log entry records the files added, removed, and schema changes in a single transaction.
Z-Order
A data layout optimization technique (used in Delta Lake and Iceberg) that co-locates related data within the same set of files to improve query performance for multi-dimensional filters.
Lakehouse
An architecture that combines the low-cost scalable storage of data lakes with the transactional reliability and governance features of data warehouses. Open table formats are the foundational layer enabling the lakehouse paradigm.
ACID Transactions
Atomicity, Consistency, Isolation, and Durability guarantees applied to table operations. Open table formats bring ACID transactions to object storage by using metadata-level coordination rather than row-level locking.

Controlled Vocabulary

Raw ↑
name: Table Format Vocabulary
description: >-
  Domain vocabulary and taxonomy for open table formats used in data lakehouses,
  covering Apache Iceberg, Delta Lake, Apache Hudi, and related catalog and
  storage concepts.
created: '2026-05-03'
modified: '2026-05-03'
tags:
  - Data Lakehouse
  - Open Table Format
  - Apache Iceberg
  - Delta Lake
  - Apache Hudi

terms:
  - term: Open Table Format
    definition: >-
      A specification that defines how tabular data is organized, tracked, and
      accessed in object storage. Open table formats add ACID transactions,
      schema evolution, time travel, and efficient query planning to data lakes
      without requiring a dedicated storage engine. The three dominant formats
      are Apache Iceberg, Delta Lake, and Apache Hudi.
    category: Core Concept
    tags:
      - Data Lakehouse
      - ACID

  - term: Apache Iceberg
    definition: >-
      An open table format specification originally developed at Netflix, now
      an Apache Software Foundation top-level project. Iceberg uses a
      hierarchical tree of metadata files (metadata.json → manifests → data files)
      for snapshot isolation, schema evolution, partition evolution, and
      hidden partitioning. It has become the dominant open table format.
    category: Table Format
    related:
      - Delta Lake
      - Apache Hudi
    tags:
      - Apache Iceberg

  - term: Delta Lake
    definition: >-
      An open-source storage layer developed by Databricks that adds ACID
      transactions to data lakes. Delta Lake maintains a delta log (JSON-based
      transaction log) recording all changes. It is deeply integrated with Apache
      Spark and supports both batch and streaming workloads.
    category: Table Format
    related:
      - Apache Iceberg
      - Apache Hudi
    tags:
      - Delta Lake
      - Databricks

  - term: Apache Hudi
    definition: >-
      An open table format (Hadoop Upserts Deletes and Incrementals) optimized
      for record-level upserts and deletes. Hudi supports two table types:
      Copy-on-Write (COW) for read-optimized workloads and Merge-on-Read (MOR)
      for write-optimized workloads. Popular for CDC-based data pipelines.
    category: Table Format
    related:
      - Apache Iceberg
      - Delta Lake
    tags:
      - Apache Hudi
      - CDC

  - term: Snapshot
    definition: >-
      An immutable, point-in-time view of an Iceberg table's state. Each write
      operation creates a new snapshot. Snapshots enable time travel queries and
      concurrent reads without locking. Each snapshot references a manifest list
      file that points to all data files in that version of the table.
    category: Apache Iceberg
    tags:
      - Time Travel
      - ACID

  - term: Manifest File
    definition: >-
      An Avro file that tracks a subset of data files in an Iceberg table snapshot.
      Manifests record file-level statistics (row counts, min/max values per
      column) enabling partition pruning and predicate pushdown without reading
      data files.
    category: Apache Iceberg
    tags:
      - Apache Iceberg
      - Metadata

  - term: Manifest List
    definition: >-
      An Avro file referenced by each Iceberg snapshot that lists all manifest
      files for that snapshot. The manifest list enables efficient snapshot
      management and incremental metadata operations.
    category: Apache Iceberg
    tags:
      - Apache Iceberg
      - Metadata

  - term: Catalog
    definition: >-
      A service that tracks table locations, schema versions, and metadata file
      locations. Catalogs provide the mapping from table name to metadata file.
      Common Iceberg catalog implementations include Apache Polaris, Project
      Nessie, AWS Glue, Hive Metastore, and JDBC catalogs.
    category: Architecture
    tags:
      - Catalog
      - Metadata

  - term: REST Catalog
    definition: >-
      A standard HTTP/REST API specification for Iceberg catalog operations,
      defined as an OpenAPI spec in the Apache Iceberg project. REST catalog
      allows any catalog service (Polaris, Nessie, BigLake, Glue) to expose a
      common interface, enabling catalog interoperability.
    category: Apache Iceberg
    tags:
      - REST API
      - Catalog
      - Interoperability

  - term: Schema Evolution
    definition: >-
      The ability to add, drop, rename, or reorder columns in a table without
      rewriting all data files. Iceberg, Delta Lake, and Hudi all support
      schema evolution with backward and forward compatibility guarantees.
    category: Core Concept
    tags:
      - Schema Management

  - term: Partition Evolution
    definition: >-
      The ability to change a table's partitioning strategy without rewriting
      existing data. Apache Iceberg supports partition evolution, allowing old
      data to use the original partitioning while new data uses the updated
      partitioning scheme.
    category: Apache Iceberg
    tags:
      - Partitioning
      - Apache Iceberg

  - term: Hidden Partitioning
    definition: >-
      Apache Iceberg's approach where partition transforms (e.g., day(timestamp),
      bucket(user_id, 16)) are applied automatically by the engine, without
      requiring users to add separate partition columns. This eliminates common
      data engineering mistakes.
    category: Apache Iceberg
    tags:
      - Partitioning
      - Apache Iceberg

  - term: Time Travel
    definition: >-
      The ability to query historical versions of a table by specifying a
      snapshot ID or timestamp. All three major table formats support time travel,
      enabling auditing, rollback, and reproducible analytics.
    category: Core Concept
    tags:
      - Time Travel
      - ACID

  - term: Copy-on-Write (COW)
    definition: >-
      A table storage strategy where updates and deletes rewrite entire data files,
      producing clean Parquet files optimized for read performance. Used in Hudi
      and Iceberg for read-heavy workloads.
    category: Storage Strategy
    tags:
      - Apache Hudi
      - Performance

  - term: Merge-on-Read (MOR)
    definition: >-
      A table storage strategy where updates and deletes are recorded as delta
      files (log files or delete files) that are merged with base files during
      reads. Optimizes write performance at the cost of additional read-time merging.
    category: Storage Strategy
    tags:
      - Apache Hudi
      - Apache Iceberg
      - Performance

  - term: Delta Log
    definition: >-
      The transaction log used by Delta Lake, stored as a sequence of JSON files
      in the _delta_log/ directory. Each log entry records the files added,
      removed, and schema changes in a single transaction.
    category: Delta Lake
    tags:
      - Delta Lake
      - Transaction Log

  - term: Z-Order
    definition: >-
      A data layout optimization technique (used in Delta Lake and Iceberg)
      that co-locates related data within the same set of files to improve
      query performance for multi-dimensional filters.
    category: Performance
    tags:
      - Performance
      - Data Layout

  - term: Lakehouse
    definition: >-
      An architecture that combines the low-cost scalable storage of data lakes
      with the transactional reliability and governance features of data warehouses.
      Open table formats are the foundational layer enabling the lakehouse paradigm.
    category: Architecture
    tags:
      - Data Architecture
      - Data Lakehouse

  - term: ACID Transactions
    definition: >-
      Atomicity, Consistency, Isolation, and Durability guarantees applied to
      table operations. Open table formats bring ACID transactions to object storage
      by using metadata-level coordination rather than row-level locking.
    category: Core Concept
    tags:
      - ACID
      - Data Integrity

Work with this as data

Every vocabulary here is available over the APIs.io API and to AI agents over MCP. Vocabularies is not yet its own endpoint on the v1 API. Reach it through catalog search and the tag graph, or the MCP server.

MCP server

One button, every client — Claude, Cursor, VS Code and the rest.

https://apis.io/mcp

Tools for vocabularies

3 MCP tools reach this
  • apis_io_searchSTART HERE — APIs, providers and tags for one query, each with its total.
  • resolveTurn a domain, URL or GitHub org into the provider it belongs to.
  • find_cohortsEvery scored population of providers in the catalog.
All 92 tools →

Call it yourself

curl for this page
Search the catalog
curl "https://apis.io/api/v1/search?q=table-format-vocabulary&limit=10"
Everything under a tag
curl "https://apis.io/api/v1/tags/table-format-vocabulary"

Calls without X-API-Key need no key. Calls with it need a paid plan — get a key.

Get an API key

Free tier, no form to fill in. Signing in shares your email address with us — we store it to create your key and to recognise you if you sign in with another provider. See our Privacy Policy and Terms.

A second provider on the same verified email joins the account you already have.