AnalysisDevelopersNovember 6, 2026

llama.cpp PR swaps SLRU for class-based MoE expert cache

Read original source →github.com

Pull request #31233 replaces SLRU with an LRFU/MQ-like hybrid cache that ranks frequently used experts in classes using decaying usage history. With a warm cache, the author measured +6% decode on QFN and +7% on GLM versus SLRU, with 8% fewer uploads.

1 source

More stories today

Open the live feed