New papers advance on-policy distillation for LLM reasoning

Arxiv CS.LG (machine learning)

Multiple Arxiv papers propose methods like near-future guidance, dynamic token selection, and trajectory selection to improve LLM reasoning via on-policy knowledge distillation. Techniques address supervision fidelity decay and cross-tokenizer distillation, showing growing research focus.

llmknowledge-distillationreasoning

See 9 more sources

When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning18 days agoXiaogeng Liu, Xinyan Wang, Yingzi Ma, Yechao Zhang, Chaowei Xiao

LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation9 days agoTianrun Yu, Kaixiang Zhao, Chih-Chun Chen, Amanda Hughes, Taylor W. Killian, Fenglong Ma, Weitong Zhang, Porter Jenkins

Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation9 days agoYanjiang Liu, Jie Lou, Xinyan Guan, Yuqiu Ji, Hongyu Lin, Ben He, Xianpei Han, Le Sun, Xing Yu, Yaojie Lu

Self-Distilled Policy Gradient6 days agoYifeng Liu, Shiyuan Zhang, Yifan Zhang, Quanquan Gu

Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation7 days agoYuying Li, Leqi Zheng, Yongzi Yu, Wenrui Zhou, Xuchang Zhong, Xing Hu, Jing Jin, Huangjie Yuan, Tao Feng

Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance8 days agoYuxuan Jiang, Francis Ferraro

Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation8 days agoRuiqi Zhang, Lingxiang Wang, Hainan Zhang Zhiming Zheng

Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversight8 days agoCan Jin, Jiakang Li, Rui Wu, Eddy Zhang, Dimitris N. Metaxas

The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works14 days agoGuanghui Wang, Kaiwen Lv Kacuila, Zhiyong Yang, Zitai Wang, Jin-Wen Wu, Longtao Huang, Qianqian Xu, Qingming Huang

Arxiv CS.LG (machine learning)·Xinyu LiuDarryl Cherian JacobYang ZhouJindong WangPan He

9 days ago