New feature
GA
Auto Optimize
Auto Optimize is a Databricks data engineering capability, introduced June 2019.
Delta Lake's umbrella name for two table settings, autoOptimize.optimizeWrite and autoOptimize.autoCompact, that shape data files as they are written (shuffling before the write for fewer, larger files) and compact small files right after a write succeeds.
- The 2019 headline Delta Lake feature has shrunk to a one-sentence glossary entry - the current docs say only that Auto optimize describes the settings autoOptimize.autoCompact and autoOptimize.optimizeWrite, then send you to its two halves.
- The autoCompact setting accepts a value literally spelled legacy, which the docs define as an alias for true - so legacy mode is just the original fixed 128 MB target with no dynamic sizing.
- Auto compaction leaves a fingerprint - in the table history it appears as an OPTIMIZE operation with operationParameters.auto set to true, which is how you tell it apart from an OPTIMIZE a human ran.
Limitations: Auto compaction and optimized writes are always enabled for MERGE, UPDATE and DELETE operations and cannot be turned off there; the manual settings do not apply to Unity Catalog managed tables, which tune file size automatically, and the docs treat the autoOptimize.autoCompact table property as a legacy setting to unset when moving such tables to background auto compaction.
Open in REbricked →- Category
- Data engineering
- Also known as
- Auto optimize, autoOptimize, delta.autoOptimize.autoCompact, delta.autoOptimize.optimizeWrite, Optimized writes, Auto compaction
- Verified
- 2026-09-11