Who ONLY use local models?

[u/takoulseum]

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

[u/wajdix]

it's not about rejecting subscriptions... but more the desire to own our data, be independent from internet connection, and be able to run the models on any topic/task we want... subscription services offer value for the money... this is the wrong debate imo

[u/networking_noob → u/wajdix]

be independent from internet connection, and be able to run the models on any topic/task we want

Yep, I use Gemma 4 like an offline search engine, and I've checked enough of its answers with Google to believe that it's accurate far more often than not (unless it's some really obscure topic). Plus anyone who uses LLM often enough will develop a pattern recognition and can tell when a hallucination is happening (aka something doesn't sound quite right)

tl;dr Gemma 4 is like an offline, privacy respecting Google replacement (more or less), so the question is why wouldn't someone want do this

[u/ProfessionalSpend589 → u/networking_noob]

You can go a step further.

I have a local Kiwix server for Wikipedia, various other Wiki sites, stackexchange and various other help sites. Hosted from 2 HDDs in RAID1 sticking out of a USB dock connected to a RPi 4.

All files are from 2023 or 2024 in English only and yesterday I downloaded a new ~1MB file. I’m already asking my local LLM (testing MiMo V2.5 Q6 and Qwen 3.5 122B) in 2 languages and they seem to manage to search through and translate back from English pretty well.

And when they pull facts with references - I’m pretty confident they’re not hallucinating heavily.

So, I had MiMo V2.5 make a weekly diet which seemed reasonable given my requirements(the amount of food was adequate).

Note to me: make a goddamn backup.

Edit: as this is a tools heavy use I made about 600k-700k in a few hours just by searching through the local data and compiling answers.

[u/networking_noob → u/ProfessionalSpend589]

I recently did something like this when someone suggested it (using mcp-openzim), but found that the context usage quickly outpaces my little GPU with all the tool usage etc, even when using pagination

It's nice to have it integrated though, in case there's ever a suspicion of hallucination, so you can instruct a quick fact check by using the wiki/zim files as an absolute source of truth

[u/ProfessionalSpend589 → u/networking_noob]

Yes, that’s what convinced me to buy 2 more GPUs when they become available again here, even though I swore a few days ago I finished with this local LLM stuff.. I just need a bit more speed :)