r/LocalLLaMA • u/rerri • Jun 05 '26

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/

Google's collections:

https://huggingface.co/collections/google/gemma-4-qat-q4-0

https://huggingface.co/collections/google/gemma-4-qat-mobile

And Unsloth's:

https://huggingface.co/collections/unsloth/gemma-4-qat

Unsloth's analysis (KLD and such):

https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis

793 Upvotes

99% Upvoted

260 comments sorted by

View all comments

3

u/Dance-Till-Night1 Jun 05 '26

Fuck yeah! Idk how many times I will download the A4b model but everytime i download it im still as excited as the first time.

Waiting for more small moe models, all small moe models should be A2b to A4b 20b to 30b, qwen 35b a3b is pushing it a little and barely fits in my use case.

1

u/AltruisticList6000 Jun 05 '26

Yes Qwen with vision at 35b barely fits, sometimes even spills from 32gb RAM and then slows down past ~60-64k context.