On-policy self-distillation papers improve LLM reasoning

Arxiv CS.LG (machine learning)

Multiple recent arxiv papers propose on-policy self-distillation methods, including Self-Distilled Policy Gradient and LARK trajectory selection, to enhance reasoning in large language models. One paper identifies Supervision Fidelity Decay as a key bottleneck in token-level teacher feedback.

LLMself-distillation

See 7 more sources

Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation25 days agoYuying Li, Leqi Zheng, Yongzi Yu, Wenrui Zhou, Xuchang Zhong, Xing Hu, Jing Jin, Huangjie Yuan, Tao Feng

Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation26 days agoRuiqi Zhang, Lingxiang Wang, Hainan Zhang Zhiming Zheng

Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance26 days agoYuxuan Jiang, Francis Ferraro

Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversight26 days agoCan Jin, Jiakang Li, Rui Wu, Eddy Zhang, Dimitris N. Metaxas

LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation27 days agoTianrun Yu, Kaixiang Zhao, Chih-Chun Chen, Amanda Hughes, Taylor W. Killian, Fenglong Ma, Weitong Zhang, Porter Jenkins

Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation27 days agoYanjiang Liu, Jie Lou, Xinyan Guan, Yuqiu Ji, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Xing Yu, Yaojie Lu

OISD: On-Policy Internal Self-Distillation of Language Models31 days agoXinyu Liu, Darryl Cherian Jacob, Yang Zhou, Jindong Wang, Pan He

Arxiv CS.LG (machine learning)·Yifeng LiuShiyuan ZhangYifan ZhangQuanquan Gu

Jun 4, 4:00 AM