Deletion vectors

Deletion vectors is a Databricks data engineering capability, introduced October 2023.

A merge-on-read trick for Delta Lake and Iceberg tables - a delete marks rows in metadata rather than rewriting whole data files, and reads subtract the marked rows at query time.

• The speed comes from not doing the work. A delete that would have rewritten a gigabyte file instead writes down which rows to skip, and the file is left exactly where it was. 🔗
• Enabling them on a materialized view or streaming table is a one-way door - you can switch the feature back off, but the table protocol can never be downgraded again. 🔗
• On Delta they are something you turn on, but every Apache Iceberg v3 table gets them whether you asked or not. 🔗

⚠ Limitations: Reading requires Databricks Runtime 12.2 LTS or above. Iceberg v2 reads don't support deletion vectors (Iceberg v3 does), you cannot run GENERATE to build a manifest for a table with deletion-vector files (REORG TABLE ... APPLY (PURGE) first) or generate manifests incrementally, and on a materialized view or streaming table existing deletion vectors remain after you turn the feature off and the table protocol cannot be downgraded. 🔗

Category

Data engineering

Introduced

October 2023🔗

Also known as

deletion vector, merge-on-read deletes, delta.enableDeletionVectors

Verified

2026-09-20