AnalysisAI ModelsNovember 18, 2026

Aurora2-400M-A20M: a sparse MoE model trained on one RTX Pro 6000

Read original source →huggingface.co

Aurora2-400M-A20M was pretrained from scratch on 100B tokens in about 36 hours on a single RTX Pro 6000. The 12-layer model uses 32 experts with 1 active per token, a 2,048-token context, and a 16,384-token vocabulary.

1 source

More stories today

Open the live feed