AnalysisDevelopersNovember 3, 2026

Basalt engine hits 300 tok/s on Qwen 3.8 Flash Next locally

Read original source →reddit.com

Basalt 0.2.0, a Blackwell-only inference engine for Qwen 3.8 Flash Next, reports 300 tok/s prose and 600 tok/s structured output with 10K tok/s prefill on a 5090 plus 5060 Ti and 128GB RAM. A separate guide claims ~3.5x llama.cpp's decode without drafting and ~7x faster prompt reads on a single RTX 5080.

How this story unfolded

11 days · 0 reports · 5 community posts · from Oct 25

  1. Oct 25
  2. Oct 29
  3. Nov 2
  4. Nov 3
  5. Nov 5

More stories today

Open the live feed