New arxiv papers propose efficient attention for long-context LLMs

Four arxiv papers explore methods to overcome the quadratic cost of standard attention: uncertainty-gated block-sparse selection, dynamic bifocal RoPE, comparative study of linear attention, and hierarchical sparse attention for infinite context. Each addresses limitations in scaling LLMs to longer sequences.