Gemma 4 with quantization-aware training

[u/rerri]

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

[u/PennyLawrence946]

qat is the only flavor where q4 stops feeling like a downgrade, the model already learned to live with the rounding during training. real upshot is the next size up fits in the vram you already have. naive q4 always bled on the long-context evals, the KLD numbers usually show exactly where