NVIDIA releases Nemotron 3 Ultra 550B MoE model

Arxiv CS.CL (NLP/language models)

Nemotron 3 Ultra is a 550B total/55B active parameter Mixture-of-Experts hybrid Mamba-Transformer model, pre-trained on 20T tokens with 1M context length. It is already seeing significant adoption, reaching 35B tokens/day on OpenRouter.

NemotronNvidiaMixture of experts

See 1 more source

Nemotron 3 Ultra is taking off on Together AI 🚀 In just days, it's climbed to 35B tokens/day on...3 hours agoTogether AI

Arxiv CS.CL (NLP/language models)·NVIDIA (Allan): (Allan)Aaron Blakeman (Allan)Aaron Thomas (Allan)Aastha Jhunjhunwala (Allan)Abhibha Gupta (Allan)Abhinav Khattar (Allan)Adam Rajfer (Allan)Adi Renduchintala (Allan)Adil Asif (Allan)Aditya Vavre (Allan)Adriana Flores Mi

Jun 16, 4:00 AM