Google DeepMindAnalysisDevelopersNovember 19, 2026

Google's Streaming DiLoCo in MaxText cuts cross-datacenter training traffic 60x

Read original source →developers.googleblog.com

Streaming DiLoCo in MaxText trains LLMs across geographically distributed TPU slices by streaming parameter fragments instead of syncing full models each step, cutting cross-slice communication volume over 60x. DiLoCo, from Google DeepMind and Google Research, splits training into independent learner groups that take many local steps before an outer SGD-with-Nesterov-momentum update.

1 source

More stories today

Open the live feed