/
New feature Beta

AI Function tasks for jobs

AI Function tasks for jobs is a Databricks ai / ml capability, introduced November 2026.

A Lakeflow Jobs task type that runs ai_parse_document, ai_extract or ai_classify on every file in a volume or every row of a table, incrementally and without writing SQL, and writes the results to a Unity Catalog table it manages.

  • Skip the Table name and it names its own output table, starting with parsed, extracted or classified - and keeps that name across runs.
  • Strictly one at a time: start a second run while the first is still going and the newcomer fails.
  • Change the labels or the fields to extract and only new inputs see it; to redo everything, you give it a new table name.

Limitations: Beta, turned on from the Previews page. Only one active run per task, and all runs in a workspace share its AI Functions capacity (Databricks suggests about 20 or fewer active at once). Input tables need change data feed (and row tracking, unless you set an ID column or use a FILE column), the output schema must already exist, and changing the input or output table reprocesses every input.

Open in REbricked →
Category
AI / ML
Introduced
November 2026
Also known as
AI Function task, ai_function_task, Lakeflow AI Function Task
Verified
2026-11-05

Sources

Related in AI / ML