Why not separate small expert models instead of MoE?

A Reddit user questions the MoE architecture, asking why we can't train separate small expert models (3B-9B params) instead of one large MoE model. The discussion explores trade-offs in specialization vs. routing efficiency.