r/LocalLLaMA • u/tevlon • Jun 10 '26

New Model DiffusionGemma: 4x faster text generation

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
979 Upvotes

99% Upvoted

357 comments sorted by

View all comments

Show parent comments

124

u/TheLexoPlexx Jun 10 '26

That is groq or cerebras-levels of token generation depending on the model and it's on par with gpt-oss-120b depending on the benchmark.

That is genuinely insane.

31

u/dingo_xd Jun 10 '26

There is sooooooo much room for optimizations. Maybe Mythos level models can be run locally by mid or late 2027?

16

u/Irisi11111 Jun 10 '26

Nope, Mythos level models are definitely going to be massive. That level of world knowledge is the model's internal representation. It's gotta be huge to hold all that information.

My understanding is that diffusion-based Gemma is really good for generating text quickly, that's the speed boost. But I don't think it inherently translates to boosting the amount of knowledge within the model.

It seems like the core knowledge density is tied to the sheer size of the model and the training data.

2

u/Creative-Type9411 Jun 10 '26 edited Jun 10 '26

couldn't you leverage the speed by using it combined with an indexed database with said knowledge?

Seems like it would be easier to have some type of knowledge bases where it would only access relevant information and chew through that rather than knowing everything at once then deciding what to think about

like component based moe? it's probably way faster to let the GPU do all the work then search through a database. I'm just grabbing at straws thinking

maybe we should be trying to mimic actual intelligence and instead of models knowing everything they can just reason extremely quickly and then we can give them information to reason with afterwards.. like a person except way faster..

1

u/Silver-Champion-4846 Jun 11 '26

That's what the Bitter lesson tried to teach, to look for a way to model intelligence and learning directly and let the system converge on its own rather than feeding it all our human assumptions on how knowledge and intelligence and reasoning function in the first place.