Moonshot AI releases Kimi K3, a 2.8T parameter MoE model

Kimi K3 is a 2.8T parameter sparse Mixture-of-Experts model featuring native vision support and a 1M-token context window. It introduces Kimi Delta Attention and Attention Residuals, which Moonshot AI claims deliver 2.5x better scaling efficiency than the previous Kimi K2 model.