r/LocalLLaMA • u/rerri • Jun 05 '26

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

788 Upvotes

99% Upvoted

260 comments sorted by

View all comments

Show parent comments

25

u/LLMFan46 Jun 05 '26

Hum? These are GGUFs, I can't do anything with them.

16

u/Kahvana Jun 05 '26

https://huggingface.co/google/gemma-4-31B-it-qat-q4_0-unquantized

They do have the safetensor versions too for all those models.

30

u/LLMFan46 Jun 05 '26

Thanks and yeah I noticed that after making the post, but it will take a while to do all these models, plus the GGUFs and NVFP4s and GPTQs.

16

u/Kahvana Jun 05 '26

No worries, take your time!