Language Re-generation: An investigation into information locality effects on…

The paper explores how information locality, the tendency for related words to appear close together, affects language models' ability to reconstruct language. It builds on prior work examining whether models can learn from non-local patterns.