Lakeflow Declarative Pipelines

Lakeflow Declarative Pipelines is the current Databricks name, previously Delta Live Tables (since June 2025).

Delta Live Tables → Lakeflow Declarative Pipelines

Declarative framework for batch and streaming ETL pipelines.

• You describe the tables you want and it works out the execution order and dependencies for you - and the DLT code written back in 2021 still runs unchanged under the new name. 🔗
• The open-source engine underneath is a separate project called Spark Declarative Pipelines, and Databricks docs increasingly shorten the product to just 'Lakeflow pipelines.' 🔗

⚠ Limitations: A workspace is limited to 1000 concurrent pipeline updates; a pipeline allows at most 100 source files (or 50 folder entries referencing up to 1000 files); pivot() is unsupported, Delta time travel works only on streaming tables not materialized views, and identity columns are incompatible with AUTO CDC processing. 🔗

Category

Data engineering

In use from

June 2025🔗

Announced at

Data + AI Summit 2025🔗

Also known as

Lakeflow Spark Declarative Pipelines, Lakeflow pipelines, DLT

Verified

2026-07-23