AnalysisDevelopersNovember 2, 2026

tandem.cpp patch set speeds MoE models on llama.cpp

Read original source →reddit.com

tandem.cpp keeps hot experts on the GPU and offloads the rest to CPU, hitting 97 tok/s with Qwen3.8-Flash-Next on an RTX 4090, i5-12400F and 64 GB DDR4 — 3.3× llama.cpp. It works for any MoE model llama.cpp supports and targets single-user decode.

1 source

More stories today

Open the live feed