LaunchDevelopers
Jun 25, 4:43 PM
NVIDIA TensorRT 11.0 adds multi-device inference support
TensorRT 11.0 introduces native multi-GPU inference, enabling deployment of large models across multiple devices without sacrificing optimizations like kernel fusion and quantization. It integrates with NCCL for high-performance collective operations and supports parallelism strategies such as tensor parallelism.
·
Jun 25, 4:43 PM