SkewAdam optimizer cuts MoE state memory by 97%

The SkewAdam tiered optimizer reduces MoE state memory by 97%, enabling a 6.7B MoE model to fit on a single 40GB GPU. The paper and open-source code are available on arXiv and GitHub.