AnalysisAI ModelsSeptember 30, 2026

Rank-8 LoRA at one early layer extends transformer reference-following

Read original source →arxiv.org

Thirteen pretrained base models reliably follow only 1.4-3.6 lines of in-context references, and extra pretrained loops add little. A task-trained rank-8 LoRA at a single early layer extends this computation with all model weights frozen; Qwen3-8B improves.

1 source

More stories today

Open the live feed