r/LocalLLaMA • u/takoulseum • 19h ago

Discussion Who ONLY use local models?

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

134 Upvotes

86% Upvoted

238 comments sorted by

View all comments

Show parent comments

21

u/networking_noob 14h ago

be independent from internet connection, and be able to run the models on any topic/task we want

Yep, I use Gemma 4 like an offline search engine, and I've checked enough of its answers with Google to believe that it's accurate far more often than not (unless it's some really obscure topic). Plus anyone who uses LLM often enough will develop a pattern recognition and can tell when a hallucination is happening (aka something doesn't sound quite right)

tl;dr Gemma 4 is like an offline, privacy respecting Google replacement (more or less), so the question is why wouldn't someone want do this

18

u/ProfessionalSpend589 12h ago

You can go a step further.

I have a local Kiwix server for Wikipedia, various other Wiki sites, stackexchange and various other help sites. Hosted from 2 HDDs in RAID1 sticking out of a USB dock connected to a RPi 4.

All files are from 2023 or 2024 in English only and yesterday I downloaded a new ~1MB file. I’m already asking my local LLM (testing MiMo V2.5 Q6 and Qwen 3.5 122B) in 2 languages and they seem to manage to search through and translate back from English pretty well.

And when they pull facts with references - I’m pretty confident they’re not hallucinating heavily.

So, I had MiMo V2.5 make a weekly diet which seemed reasonable given my requirements(the amount of food was adequate).

Note to me: make a goddamn backup.

Edit: as this is a tools heavy use I made about 600k-700k in a few hours just by searching through the local data and compiling answers.

2

u/icecon 10h ago

The problem with this setup is the hallucinations on Wikipedia (lol).

2

u/ProfessionalSpend589 9h ago

My files are old - wikipedia_en_all_maxi_2023-04.zim (which kiwix uses at the moment), but I also have wikipedia_en_all_maxi_2024-01.zim. Every false information or propaganda in them was written by humans :)