Loopie series: 20B (2B active) and 6B (0.6B active) looped MoE models

The Loopie series consists of two Mixture-of-Experts looped Transformer models: a 20B-parameter model with 2B active parameters and a 6B-parameter model with 0.6B active parameters. Looped transformers address the challenge of limited depth by reusing layers.