Paper examines geometric limits of feature representation in transformers

Arxiv CS.LG (machine learning)

The study analyzes how model dimension (d_model) sets geometric limits on feature representation in transformer language models, grounded in linear representation and superposition. It provides a theoretical framework for understanding capacity constraints.

transformerllm

Arxiv CS.LG (machine learning)·Alexander Guha

8 days ago