AnalysisAI ModelsNovember 3, 2026

Zyphra releases ZHA, a tree-indexed sparse attention method

Read original source →marktechpost.com

ZHA adds 81,920 trainable parameters (~0.00024%) to Qwen3.6-35B-A3B and retrofits frozen models after a 4B-token KL warmup at 32K context. LongBench v2 hits 63.4% vs 58.0% dense using ~1.1% of keys, but deep-tree RULER-13 drops 25.3 points on Nemotron-3.5-Lightning-30B-A3B.

1 source

More stories today

Open the live feed