NVIDIAHow-ToAI ModelsNovember 17, 2026

NVIDIA publishes sparse attention co-design guide for long-context inference

Read original source →developer.nvidia.com

At 128K tokens, dense attention can consume most of a layer's prefill time; in DeepSeek V3.2, the indexer and Top-k take 47% of prefill. The primer sweeps indexer knobs — query heads, key heads, head dimension, compression ratio, speculative-decoding depth — on a production FP8 kernel.

1 source

More stories today

Open the live feed