LaunchDevelopersOctober 21, 2026

Google Cloud ships llm-d 1.0 with cache-aware routing

Read original source →marktechpost.com

llm-d 1.0, the first stable release of the Kubernetes-native distributed inference stack, cuts median TTFT by over 85% and lifts throughput 2.4x on an Ironwood TPU coding workload. It runs vLLM or SGLang servers on NVIDIA, AMD or Intel GPUs or Google TPUs.

1 source

More stories today

Open the live feed