Deletion Vector
A metadata pattern that tracks which rows in a data file have been logically deleted or updated, using a compact bitmap instead of rewriting the entire file.
Summary
A metadata pattern that tracks which rows in a data file have been logically deleted or updated, using a compact bitmap instead of rewriting the entire file.
Deletion vectors are the key mechanism that makes merge-on-read (MoR) practical for lakehouse formats on S3. Instead of the expensive copy-on-write approach (rewriting a 128MB Parquet file to delete one row), a tiny deletion vector file marks the invalidated rows. Query engines skip those rows at read time, and periodic compaction reconciles the deletes.
- Deletion vectors improve write performance at the cost of read performance. Queries must check deletion vectors for every data file, adding overhead until compaction runs.
- Not all engines support deletion vectors equally. Check your query engine's support before depending on this pattern for high-throughput reads.
enablesApache Iceberg, Delta Lake — efficient row-level operationssolvesSmall Files Problem — reduces write amplificationscoped_toTable Formats, S3
Definition
A metadata pattern used by lakehouse table formats to track which rows in a data file have been deleted or updated, without rewriting the entire data file. Instead of copy-on-write, a compact bitmap or vector records the positions of invalidated rows.
Copy-on-write (CoW) updates in lakehouse formats require rewriting entire Parquet files to delete or update a single row, causing massive write amplification. Deletion vectors enable merge-on-read (MoR) by recording row-level deletions in lightweight metadata files, dramatically reducing write costs for high-frequency update workloads.
Efficient row-level deletes and updates in Iceberg and Delta Lake, high-frequency CDC ingestion with low write amplification, streaming update workloads on S3.
Recent developments
- Deletion vectors are now production-ready as of Apache Iceberg 1.11.0 (May 19, 2026). Each deletion vector is a Roaring bitmap stored in the Puffin file format, maintaining a 1:1 relationship between a data file and its deletion file — the V3 mechanism that replaces V2 positional delete files. The 1.11.0 GA moves deletion vectors from experimental, flag-gated status to the stable supported path, enabling broad engine rollout (Spark 4.1, Flink 2.1, and the wider multi-engine ecosystem). Per What's New in Apache Iceberg 1.11.0 (Dremio).
- Apache Iceberg 1.11.0 (May 2026) ships V3 deletion vectors as production-ready. Apache Iceberg 1.11.0 release marked V3 maturity — deletion vectors, Variant type, and the rest of V3 features now production-grade across the major engines. Per Dataverses — Apache Iceberg 1.11.0 Release: Deletion Vectors, Variant Type, V3 Maturity.
- Roaring bitmaps in Puffin sidecar files — one DV per data file maximum. Each
file_A.parquetis paired with onefile_A.puffincontaining the Roaring bitmap of deleted row positions. 64-bit-position-capable but optimized for the common case of positions fitting in 32 bits. Per Apache Iceberg — Deletion Vectors spec. - AWS EMR adds first-class deletion-vector support. EMR Iceberg integration uses V3 deletion vectors automatically — production-grade Spark + EMR pipelines get the V3 benefits with no config changes. Per AWS Big Data Blog — Unlock the Power of Apache Iceberg V3 Deletion Vectors on EMR.
- CDC pipeline perf: up to 10× faster MERGE/UPDATE vs V2 position-delete files. Removes the read-amplification of V2 where queries had to join data files against delete files at query time — DV-aware engines read the bitmap once at scan start, no per-row join. Per DataLakehouseHub — What Iceberg V3 Advances Mean for CDC Pipelines.
- MinIO AIStor Tables ships native V3 deletion-vector support on-prem. First on-prem object storage to ship Iceberg V3 deletion-vector compaction natively in the bucket — no separate compute layer needed. Per MinIO Blog — AIStor Tables: Native Iceberg V3 for On-Prem Object Storage.
- Cross-format consensus: Iceberg V3 + Delta Lake + Hudi 1.x all support deletion vectors as soft deletes. All three major table formats now ship deletion-vector support; the architectural argument that "MoR with DVs beats CoW for high-frequency updates" is now uncontested across the table-format ecosystem. Per Olake — Comparing Delete Methods in Iceberg & Delta Lake.
Connections5
Outbound5
Resources2
Databricks documentation explaining deletion vector mechanics, performance benefits, and configuration for Delta Lake.
Comparison of delete strategies across Iceberg, Delta, and Hudi including deletion vector approaches and their performance trade-offs.