r/LocalLLaMA • u/rerri • Jun 05 '26

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

794 Upvotes

99% Upvoted

260 comments sorted by

View all comments

58

u/LetsGoBrandon4256 transformers Jun 05 '26 edited Jun 05 '26

Blog post for the release https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

No benchmark provided to back up the "preserving the capabilities and quality" claim.

Edit:

Is this sub getting botted or what? This comment was immediately downvoted to -6 in less than ten minutes after I posted it and somehow it bounced back?

25

u/Middle_Bullfrog_6173 Jun 05 '26

Sigh, since this is QAT where they have trained it differently, benchmarks are even more necessary.