Iceberg Table Spec
The specification defining how a logical table is represented as metadata files, manifest lists, manifests, and data files on object storage. Provides ACID, schema evolution, hidden partitioning, and time-travel.
Summary
The specification defining how a logical table is represented as metadata files, manifest lists, manifests, and data files on object storage. Provides ACID, schema evolution, hidden partitioning, and time-travel.
The Iceberg spec is the blueprint that Apache Iceberg implements. It defines the metadata tree structure that turns a collection of Parquet files on S3 into a reliable, evolvable table — and enables any engine to read the same table consistently.
- The spec defines behavior, not implementation. Different engines (Spark, Flink, Trino) may implement the spec at different levels of completeness.
- Manifest files accumulate with every write. Without regular metadata cleanup (expire snapshots, remove orphan files), metadata overhead grows.
enablesLakehouse Architecture — the specification that makes Iceberg-based lakehouses possiblesolvesSchema Evolution (column-ID-based evolution), Partition Pruning Complexity (partition specs in metadata)scoped_toTable Formats, Lakehouse
Definition
A specification defining how a logical table is represented as a set of data files, metadata files, manifest lists, and snapshots on object storage. Provides ACID semantics, schema evolution, hidden partitioning, and time-travel.
Files on S3 have no inherent table structure. The Iceberg spec adds a metadata layer that turns a collection of Parquet files into a reliable, evolvable table — without requiring a database server.
Defining lakehouse tables on S3, multi-engine table access (Spark, Trino, Flink can all read the same Iceberg table), schema evolution without rewriting data.
Recent developments
- V3 GA (May 7 2026) — deletion vectors, row lineage, VARIANT, geospatial, default values. V3 mandates row-lineage tracking (
_row_id+ last-modified sequence number per row); adds deletion vectors as compact Roaring bitmaps in Puffin files; introduces VARIANT semi-structured type (Parquet Variant encoding); native geometry + geography + nanosecond-timestamp types; multi-argument transforms. Per Apache Iceberg Spec and AWS Prescriptive Guidance — Working with v3. - V4 in active development — typed metrics + relative paths. V4 stores metrics as typed values (vs v3's map-of-strings representation); adds relative-path support within table metadata so Iceberg tables can be moved or copied across object stores without rewriting metadata. Per Cloudera — Apache Iceberg Specs Explained: v1, v2, v3, what's coming in v4.
- Spec is now the multi-engine interoperability standard. Iceberg's V3 spec is implemented across the major commercial engines (Snowflake, Databricks, AWS) — multi-engine writes + reads against the same physical tables are the assumed pattern, not the aspirational one. Per DeepWiki — Apache Iceberg Table and View Specifications.
- Manifest immutability is structural, not advisory. A manifest is an immutable Avro file listing data + delete files with their per-file partition tuples, metrics, and tracking info — one or more manifests compose a snapshot. Engines rely on this immutability for correctness during concurrent reads. Per Iceberg Spec — Table Metadata.
- Position deletes encoding split by version. Position deletes are encoded in a position-delete file (V2) or a deletion vector (V3+) — the V3 deletion-vector encoding is materially more efficient under high-churn CDC workloads where V2's positional-delete-file accumulation was the dominant overhead. Per GitHub — apache/iceberg format/spec.md.
Connections 11
Outbound 5
scoped_to2enables1Inbound 6
extends1Resources 3
The authoritative Iceberg Table Specification defining the metadata tree, manifest files, snapshot structure, schema evolution rules, and partitioning model.
Canonical repository containing the reference Java implementation and the specification source documents.
Official documentation covering API usage, configuration, integrations with Spark/Flink/Trino, and migration guides.