MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers

Paper constructs efficient fact-storing MLPs via Hebbian-like updates, providing a mechanistic interpretability model for how LLMs encode knowledge in parameters. The approach matches observed properties of real MLP layers in transformers.