How-ToDevelopersNovember 3, 2026

Basalt engine hits 300 tok/s on Qwen 3.8 Flash Next locally

Read original source →reddit.com

Basalt 0.2.0, a Blackwell-only inference engine for Qwen 3.8 Flash Next, reports 300 tok/s prose and 600 tok/s structured output with 10K tok/s prefill on a 5090 plus 5060 Ti and 128GB RAM. A separate RTX 5080 + 64GB setup logged >250 tok/s generation and 5,700 tok/s prefill, roughly 3.5x llama.cpp's decode without drafting.

How this story unfolded

11 days · 0 reports · 5 community posts · from Oct 25

  1. Oct 25
  2. Oct 29
  3. Nov 2
  4. Nov 3
  5. Nov 5

More stories today

Open the live feed