AnalysisAI ModelsNovember 21, 2026

Reddit post questions why Shampoo, SOAP, and Adam don't measure per-sample second moments

Read original source →reddit.com

The r/MachineLearning discussion argues Shampoo itself doesn't measure true second moments, noting Muon matched Shampoo's performance despite dropping the actual second moments L and R. The poster contrasts these optimizers with the Fisher Information Matrix's per-sample approach.

1 source

More stories today

Open the live feed