Spectral Asymptotics of Neural Network Loss Landscapes

Arxiv CS.LG (machine learning)

Paper presents exact decomposition of the curvature exponent α, finding α ≈ 2 for convolutions and ≈ 1 for transformers. The analysis provides a theoretical framework for Hessian eigenvalue scaling in deep learning.

neural-networksloss-landscapes

Arxiv CS.LG (machine learning)·Anherutowa Calvo

8 days ago