Unsloth releases Gemma 4 QAT GGUF quantized model

Hugging Face Models

Unsloth's Gemma-4-31B-it-qat-GGUF model on HuggingFace has 51k downloads and 56 likes. Reddit user reports QAT model responds significantly better to KV cache quantization, with KL divergence results on wikitext at 16k context.

Gemma

See 1 more source

Gemma 4 QAT seems to respond significantly better to KV cache quantization23 hours agorima_2711Discuss

Hugging Face Models·unsloth

Jun 5, 10:35 AM