New papers examine weight norm's role in grokking delay

Arxiv CS.LG (machine learning)

Three new theoretical papers investigate how weight norm controls grokking's delayed generalization. One paper identifies a critical norm threshold, while another shows that weight norm alone sets the grokking timescale via a causal delay law.

weight-normNeural networks

See 1 more source

The Weight Norm Sets the Grokking Timescale: A Causal Delay Law3 days agoTruong Xuan Khanh, Doan Hoang Viet, Luu Duc Trung, Phan Thanh Duc

Arxiv CS.LG (machine learning)·Truong Xuan Khanh

Jun 15, 4:00 AM