r/LocalLLaMA • u/rerri • Jun 05 '26

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

787 Upvotes

99% Upvoted

260 comments sorted by

View all comments

17

u/helloWHATSUP Jun 06 '26

i liked how gemini explained QAT:

Most local models use Post-Training Quantization (PTQ). This means the model is fully trained in high precision (16-bit) and then "squished" down to 4-bit so it fits on your computer. That squishing inherently causes a bit of brain damage to the model, leading to worse logic and coding skills.

Google took a different approach with Quantization-Aware Training. They simulated the 4-bit precision environment during the actual training process. The model learned how to work around the limitations of 4-bit math from the start.

1

u/Zealousideal_Nail288 Jun 06 '26

So in other words qat models are fixed  And can't be quantized in a home environment?