How-ToDevelopersJuly 6, 2026

Llama-server's slot save/restore wastes KV caches; fix available

Up to 2.49 GB of KV cache is restored then discarded due to slot save/restore design. A workaround preserves cached state across requests, improving performance on budget hardware.

1 source