AnalysisAI ModelsJuly 16, 2026

Reddit user explores MoE expert prediction for CPU/GPU offload, claims 5-7x speedup

User reports improving inference speed from ~30 tg/s to 150-200 tg/s on Qwen3 35B-A3B (MXFP4) with a 12GB 3060 by predicting which MoE experts will be used next token. The technique aims to reduce idle GPU time waiting for experts transferred over PCIe, but user questions if it's implementable in practice.

1 source