NVIDIAHow-ToDevelopersSeptember 30, 2026

NVIDIA Dynamo-Triton adds HSTU generative recommender inference

Read original source →developer.nvidia.com

Dynamo-Triton now supports an end-to-end HSTU generative recommender workflow via NVIDIA's recsys-examples repo, combining PyTorch AOTI, FlexKV KV caching, and NV embedding cache. At batch size 8 on an RTX PRO 6000 Blackwell Workstation GPU, it hit up to 4.47x speedup for the three-layer HSTU model and 5.93x for the eight-layer model at 100% KV-cache hit rate.

1 source

More stories today

Open the live feed