AnalysisAI ModelsSeptember 26, 2026

Two papers tackle multi-teacher on-policy distillation routing

Read original source →arxiv.org

Both papers target multi-teacher on-policy distillation (MOPD), which merges specialist models (math, coding, instruction-following) into one student. One proposes domain-normalized distillation; the other, MOPD-Router, drops the hard prompt-level domain-label routing that blocks unlabeled training data.

How this story unfolded

2 days · 3 reports · from Sep 28

  1. Sep 28
  2. Sep 29
  3. Sep 30

More stories today

Open the live feed