New feature
GA
Query-based connectors
Query-based connectors is a Databricks data engineering capability, introduced April 2026.
Lakeflow Connect connectors that ingest from databases (Oracle, Teradata, SQL Server, MySQL, MariaDB, PostgreSQL and any Lakehouse Federation source) by querying the source on a schedule with a cursor column, with no CDC setup and no ingestion gateway.
- To catch rows committed late by a long-running source transaction, each run quietly re-reads a short window just below the previous high-water mark.
- It only sees the latest state of each changed row - if a row changes five times between runs, the four in-between versions never existed as far as it knows.
- Through foreign catalog ingestion it can read any Lakehouse Federation source, so anything Databricks can federate it can also ingest.
Limitations: Queries run directly against source tables, so they can be slower and add load compared with CDC connectors; runs are scheduled rather than continuous, deletion tracking needs API configuration (hard-delete tracking is in Beta), and the cursor column must be monotonically increasing.
Open in REbricked →- Category
- Data engineering
- Also known as
- Query-based ingestion, Lakeflow Connect query-based connector
- Verified
- 2026-10-20