Red Hat engineers discuss KV cache-aware routing and P/D disaggregation on Kubernetes

Yuchen Fama and Ashish Kamra of Red Hat present KV cache-aware routing and P/D disaggregation for Kubernetes. Agentic sessions in Red Hat's traces run from a few turns to 3,000, with cache hit rates routinely above 90% and input-to-output token ratios often exceeding 100:1.

Featured · Yuchen Fama, Ashish Kamra