AnalysisAI ModelsJune 17, 2026
MiniMax releases MSA sparse attention for long contexts
MSA builds on GQA to reduce quadratic cost of softmax attention. Tested on a 109B-parameter MoE model with a 3T-token budget.
MSA builds on GQA to reduce quadratic cost of softmax attention. Tested on a 109B-parameter MoE model with a 3T-token budget.