TurboQuant: Training-free compression for agent retrieval

TurboQuant is a training-free compression method that reduces the memory footprint of embeddings and KV cache for agent retrieval. It addresses the hidden RAM cost of storing data at 32-bit precision, which requires four times more memory than needed. The method was presented by Shashi Jagtap at AI Engineer.

Featured · Shashi Jagtap