LaunchDevelopersOctober 1, 2026

Ai2 releases Olmo-core 3 open MoE training stack

Read original source →huggingface.co

Olmo-core 3 scales mixture-of-experts training into the trillion-parameter range; in one benchmark Ai2 grew the expert pool from 8 to 128 while still routing four experts per token, holding active parameters near 3.2B. Total capacity rose from 4.6B to 47B with training throughput down less than 5%.

2 sources

More stories today

Open the live feed