How Together AI uses NVIDIA's full stack to deliver AI responses under 100ms

VP Dan Fu explains a megakernel that fits an entire model into a single CUDA kernel, achieving sub-100ms response times and low token costs. The approach leverages NVIDIA GPUs and optimizations across the full stack.

Featured · Dan Fu