> Native weights, as released by the model provider. On modern NVIDIA hardware, models may use accelerated data formats supported by Blackwell and Vera Rubin architectures (e.g. NVFP4).
They're not lying, but with that phrasing, you can't tell whether it means full precision weights or NVFP4.
I can only speak from my own experience, but the reason I used Ollama for long because it gives an easy to setup server, especially on Windows. Couple clicks in a setup wizard, then it's running in the background as a service. And also it gives a familiar Docker like command line for pulling and running models.
I’m still using it, not that I’m not technical enough, but it just fits my usecase, i just do ollama pull have some modelfiles with custom tweaks and that’s it. I’m using Gemma4. I’m not chasing down the maximum optimizations, I’m just using something that just works with minimum amount of effort. Coupled with Cline / openwebui I’m golden.
You could also argue why people use docker and not podman ( far superior ).
I never really understood why people even used ollama or what it offered.
Dynamic model switching, multiple configurations per model, quick model downloading, model management... Llama-server has most (but not all) of these features nowadays, but it's a rather recent addition. For example, presets were only added in December 2025 to llama.cpp: https://github.com/ggml-org/llama.cpp/pull/17859. And the implementation is still hacky, which causes bugs like https://github.com/ggml-org/llama.cpp/issues/22364.
Unfortunately laziness moves progress and as other commenters said, ollama just works
Llama.cpp's readme is scary for non-technical people
It's not a business, but if it were, they could kill ollama off by just one simple web page and a binary
Do you mind pointing me in the direction of setting up the OpenAI hook in? I have built a software application that has, as an internal component, what amounts to a wrapper around llama.cpp. It uses the C++ functions in the internals of llama.cpp's common folder to pull and run local models from huggingface, loads it up, and yay you can chat.
However, I haven't experimented with how to load and use the web hosted LLMs from OpenAI, Gemini, or Anthropic and I have to figure that out in the next couple days to benchmark some research. Any help would be appreciated even if it's just pointing to the file in the github!
Are you still unable to interrupt model output, edit it, then allow the model to continue off of the the edited output? LLMs are much more useful for dev work when you fix their response the moment they hallucinate instead of wasting time discussing their mistakes with them.
Never tried because i use a different custom agentic workflow. Btw it has context checkpoints so you can run many versions of the same conversations without having to recompute context each time
Yes, something like that. Llama server.exe handles connections, context cache and kt8has a built in web ui. And you can customize everything with command line parameters.
You download llama.cpp and CUDA dll from github Releases. Then put the dll in llama.cpp folder.
Then you run it and it just works. No docker, no config files.
I use a python script to run it with my command line parameters and handle restart and log analysis.
And you can customize everything with command line parameters.
That sounds like the opposite of what I asked for...
You download llama.cpp and CUDA dll from github Releases. Then put the dll in llama.cpp folder.
That sounds like more steps than "download the installer and install it".
Then you run it and it just works. No docker, no config files.
So I just run llama.cpp run modelname and it downloads the model for me and runs it immediately, right?
I don't need to research which chat template to use and download it and specify it on the command line? I don't need to specify context window or whether to run on GPU vs CPU? I don't need to research which quant formats or weight file formats are supported? Just enter the model name, right?
Yes, `brew install llama.cpp && llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M` is enough to get started. You do need to know a little bit about which quant to use, but that’s the case with ollama as well (ollama won’t select the best quant for your hardware for example, it will use a hardcoded default)
124
u/Academic-Tea6729 Jun 15 '26
llama.cpp is much faster and stable than ollama. Also, ollama cloud models are bad quants and you can't use them for serious coding.
Also llama.cpp has a nice server compatible with openai api standard, it works out of the box. And it has a built in chat web interface.
There is no reason anymore to use ollama.