DuDi: Dual-signal distillation for multilingual small language models

Arxiv CS.CL (NLP/language models)

DuDi is a dual-signal distillation framework that improves multilingual performance of sub-billion-scale SLMs, particularly for Southeast Asian languages. It uses a cross-lingual verbalizer to transfer knowledge from a larger teacher model.

llmmultilingualdistillation

Arxiv CS.CL (NLP/language models)·Patomporn PayoungkhamdeeTinnakit UdsaJian Gang NguiSarana NutanongAlham Fikri AjiPeerat Limkonchotiwat

7 days ago