deprecated
HorovodRunner
HorovodRunner is deprecated, replaced by TorchDistributor (PyTorch) or tf.distribute.Strategy (TensorFlow) (deprecated November 2024).
Databricks API for running distributed deep learning with the Horovod framework on Databricks Runtime ML, embedding a Horovod MPI job inside a Spark job via barrier execution mode so TensorFlow, Keras, and PyTorch training could span a cluster.
- Under the hood it was MPI in a Spark costume: the training function was pickled on the driver, the first executor gathered every worker's IP address over BarrierTaskContext, and then it simply called mpirun.
- A negative process count was a feature, not a bug - HorovodRunner(np=-n) meant run n subprocesses on the driver only, the documented way to test on a single multi-GPU node before going wide.
- Its built-in profiler shipped with a health warning: enabling Horovod Timeline could cut Inception3 throughput by roughly 40 percent, so the docs told you to leave it off if you wanted the job to finish.
Limitations: With workspace files, HorovodRunner does not work when np is greater than 1 and the notebook imports from other relative files (the docs point to horovod.spark instead), and Open MPI TCP connection warnings mean pinning training to the primary network interface.
Open in REbricked →- Category
- AI / ML
- Also known as
- Horovod, horovod.spark, sparkdl.HorovodRunner, Horovod on Databricks
- Verified
- 2026-09-11
Sources
- Official Official Databricks / Microsoft docs
- Official Databricks Runtime 15.0 ML release notes: deprecation of Horovod and HorovodRunner announced
- Official Databricks Runtime 16.0 ML release notes: Horovod, HorovodRunner, Petastorm, spark-tensorflow-distributor removed
- Official Databricks release notes: Databricks Runtime 15.0 and 15.0 ML GA (March 22, 2024)
- Official Databricks release notes: Databricks Runtime 16.0 and 16.0 ML GA (November 11, 2024)
- Official Microsoft Learn (archived): HorovodRunner on Azure Databricks