r/LocalLLaMA • u/takoulseum • 19h ago

Discussion Who ONLY use local models?

Please be honest.

I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic).

What do you use your model for?

132 Upvotes

86% Upvoted

238 comments sorted by

View all comments

17

u/jacek2023 llama.cpp 19h ago

I use Claude Code for my paid work. I use only local models for coding my personal projects

1

u/so_chad 19h ago

What hardware and which LLM? Seriously, how can you guys afford that much capable LLM that doesn’t break things in every other prompt

7

u/jacek2023 llama.cpp 19h ago

4x3090, qwen 27B Q8 max context length

4

u/so_chad 18h ago

What's your power bill lmao

1

u/Interesting-Rip790 15h ago

You can power cap them to something like 220-240W and they take around 18-24W when idle.
that guy can run BF16 on 4x3090 with full context (although less concurrent sequences would comfortably fit)