Paper finds KL divergence is a poor fidelity metric for quantized LLMs

Arxiv CS.LG (machine learning)

The paper evaluates per-token KL divergence (KLD) as a fidelity metric for quantized LLMs, using a 28-quant cohort of Qwen3.6-35B-A3B and a 41-quant cohort. It finds KLD does not reliably predict benchmark quality, as displacement in output does not equate to direction of degradation.

LLMkl-divergence

Arxiv CS.LG (machine learning)·Milo\v{s} Nikoli\'cAli Hadi ZadehEnrique Torres SanchezAndreas Moshovos

Jun 19, 4:00 AM