Variable bit-width quantization learns per-group precision for LLMs

VBQ trains each weight group at its own optimal bit-width, achieving higher accuracy than uniform quantization at the same average bits. The method uses Gumbel-Softmax for discrete precision selection during training.