Microsoft SQL Server connector

Microsoft SQL Server connector is a Databricks data engineering capability, introduced July 2024.

A managed Lakeflow Connect database connector that incrementally ingests SQL Server tables (Azure SQL, Amazon RDS, VMs, on-premises) into Databricks using Microsoft change tracking or change data capture.

• It was in the very first batch of Lakeflow Connect sources in July 2024, next to Salesforce and Workday. 🔗
• It comes in two shapes - the standard gateway-based setup with two pipelines, or integrated CDC that folds extraction into one pipeline. 🔗
• Using Microsoft CDC needs SQL Server 2012 SP1 CU3 or later, because that patch added the command id column the gateway uses to tell an UPDATE from a DELETE-INSERT pair. 🔗

⚠ Limitations: Primary instances only (change tracking and CDC do not work on read replicas), the gateway must run continuously on classic compute and must not be stopped manually, 250 or fewer tables per pipeline is recommended, source names are case-sensitive, and column renames or type changes need a full refresh. 🔗

Category

Data engineering

Introduced

July 2024🔗

Also known as

SQL Server connector, Microsoft SQL Server ingestion connector, Lakeflow Connect SQL Server

Verified

2026-10-20