AnalysisAI ModelsOctober 21, 2026

OpenBMB runs 35B MoE at 20.5 tok/s on iPhone without skipping experts

Read original source →reddit.com

OpenBMB's engineering write-up targets the real bottleneck — moving expert weights from flash storage to compute fast enough — rather than shrinking the model or dropping experts. The result is a 35B MoE model running at 20.5 tokens/sec on an iPhone.

1 source

More stories today

Open the live feed