Model Serving

Model Serving is the current Databricks name, previously Serverless Real-Time Inference (since March 2023).

Serverless Real-Time Inference → Model Serving

Highly available, low-latency service for deploying ML and GenAI models behind autoscaling REST endpoints.

• It shed the tongue-twister name 'Serverless Real-Time Inference' when it reached general availability in March 2023. 🔗
• Some surfaces later slapped a 'Mosaic AI' prefix on it. 🔗

⚠ Limitations: Documented limits include up to 1000 endpoints per workspace (raisable on request), a 16 MB request payload (4 MB for AI agent endpoints), a 597-second model execution duration per request, 50 create/update operations per 5-minute window, provisioned concurrency of up to 1024 per model and 4096 per workspace (raisable on request), a queries-per-second cap of 300,000 with route optimization (200 without, recommended only for small dev use), and no logging for requests or responses over 1 MB. 🔗

Category

AI / ML

In use from

March 2023🔗

Announced at

Model Serving GA, March 7, 2023🔗

Also known as

Serverless Real-Time Inference, Mosaic AI Model Serving, Model Serving

Verified

2026-07-23