AnalysisAI Models
9 days ago
Speculative KV coding compresses KV cache losslessly by up to 4×
The method achieves up to ~4× lossless compression of the KV cache for transformer inference. It uses a speculative encoding approach to reduce memory overhead without sacrificing quality.