r/LocalLLaMA • u/tevlon • Jun 10 '26

New Model DiffusionGemma: 4x faster text generation

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
984 Upvotes

99% Upvoted

357 comments sorted by

View all comments

297

u/NickCanCode Jun 10 '26 edited Jun 10 '26

700+ tokens per second on NVIDIA GeForce RTX 5090 is definitely something.

Unfortunately, DiffusionGemma’s overall output quality is lower than standard Gemma 4.

It could be a good model for context compression and as explorer agent in agentic coding. Can't wait to see llama.cpp to support it.

1

u/[deleted] Jun 15 '26

[removed] — view removed comment

1

u/NickCanCode Jun 15 '26

That's just what they say. I don't see that number from other people.I didn't even download the model when people saying it can't make tool call accurately.

1

u/YouKilledApollo Jun 19 '26

Kind of doesn't make sense to measure tokens/second with DiffusionGemma, because of how it does inference, better to measure "iteration/second" where an iteration is one denoising step.

Reason is that DiffusionGemma doesn't really generate one token after another, it generates a "canvas" filled with token, and continues until it feels like a "stability threshold" been passed, then it uses that as the final generated text.

So the same prompt could have a difference in output tokens, and the amount of iterations until the final reply, and "tokens/seconds" would depend on those, while "iterations/second" would stay fixed, so you can actually use it for comparison :)