ReLoRA: Knowledge-Reusing Adaptation for Evolving LLMs

Arxiv CS.LG (machine learning)

Proposes ReLoRA, a method to reuse knowledge from previous LoRA adapters when base LLMs are updated, reducing retraining cost. Evaluated on multiple benchmarks, it achieves comparable performance while significantly speeding up adapter rollout.

llmloraadaptation

Arxiv CS.LG (machine learning)·Yang XuZihuai XuHongli XuYunming LiaoZhiwei YaoXitong Fu

8 days ago