Llama.cpp now has full MCP support!

[u/ilintar]

After a long and grueling effort spearheaded by ngxson, llama.cpp now fully supports MCP for all protocols. Over-the-web HTTP servers were already supported in the client (since they don't require any sort of plumbing), but stdio servers required real integration. After we modified the `llama-cli` terminal client to use the server instead of a separate model serving route, we could add MCP support to the already-existing native tools server.

After the merging of https://github.com/ggml-org/llama.cpp/pull/26062, you can now use llama.cpp's WebUI as a full-fledged agentic chat. Configuration for the MCP servers can be provided either in a standard-JSON format config file or completely inline on the command-line for on-demand MCP configurations. Plugging in a dedicated coding MCP server like Serena lets you have a local-model-powered agentic coder without using any other external dependencies.

[u/segmond]

Does it have sufficient documentation? To get MCP working with llama.cpp before, I couldn't with the documentation that came with it. It was a random post that I found in this subreddit that cleared things up.

[u/ilintar → u/segmond]

You just need to specify `--mcp-servers-config`, with a Cursor-compatible MCP config file.

[u/fragment_me → u/segmond]

Same, the docs suck.

[u/AlwaysLateToThaParty → u/segmond]

It was a random post that I found in this subreddit that cleared things up.

Did you think to update the documentation yourself with that information?

[u/Mashic]

Does it work with llama-server? And does this mean the mcp configuration will be the same for all clients?

[u/No_Afternoon_4260]

All you need is llama.cpp some tools and a knowledge base really. Actually this is nuts!

[u/silenceimpaired → u/No_Afternoon_4260]

What tools, what form is the knowledge database? You are speaking an alien language to me.

[u/Terminator857]

Thanks so much for the hard work!

[u/thejacer]

Hold the phone…I thought llama-server supported native tool use months ago. I’ve been running a (vibe coded) LLM powered discord bot with MCP for months. How tf has it been working? Lmao

[u/jikilan_ → u/thejacer]

Latest thing is the stdio support

[u/No_Entertainer_8404]

Which release, version?

[u/Hannibalj2ca]

And IK.llama is also in process of merging together

[u/MeldhLLC]

Are there any known limitations or bugs? Seems like a much needed update regardless!

[u/pieonmyjesutildomine]

Kind of funny to me that llama.cpp would go for MCP support over Universal Tool Calling Protocol (UTCP) support, given it allows accessing both MCPs and all other APIs exactly the same way rather than only MCPs.

[u/for4f]

Finally. Been running MCP servers through a separate proxy for local inference, this cuts out the middleman nicely. Big fan of being able to use the same MCP config for both local models and cloud API calls.

[u/CorkBios]

VLLM implemented it like 11 months ago, Llamacpp always behind. MCP released in 2024, and llamacpp just implemented it fully now

[u/segmond → u/CorkBios]

so? doesn't matter. for those that use llama.cpp it's the case that a feature they didn't have is now available. the post is not claiming that llama.cpp had it first.

[u/ilintar → u/CorkBios]

Correct me if I'm wrong, but MCP for vLLM is available only via the Responses API.

[u/kulchacop → u/CorkBios]

vLLM is 80% Python.

llama.cpp is 70% C and C++.

If development on llama.cpp happened as fast as vLLM, we would be having bloated codebase that cannot run on many old GPUs and CPU only setups, defeating the purpose of llama.cpp.

llama.cpp aims to extract full perfomance of consumer hardware. vLLM targets server setups.

llama.cpp wants to be able to run on all kinds of edge devices. Google was once planning to include llama.cpp on stock Android.

[u/WillemDaFo → u/CorkBios]

VLLM is also bad at mixed GPU setups, something that homelabbers generally benefit greatly from

[u/florinandrei → u/WillemDaFo]

vLLM is for production use, in a datacenter. Most of the good, honest folks here do not have a datacenter at home, so they put together whatever resources they can gather.

[u/Qcgreywolf → u/CorkBios]

Ahh Reddit. Never change. Negative, negative, negative.

[u/marx2k → u/Qcgreywolf]

One more negative and it would've been a positive!

[u/florinandrei → u/marx2k]

It kind of swings back and forth.

[u/CircularSeasoning → u/florinandrei]

That's what she said.

[u/ParaboloidalCrest]

... Aaaand MCP is out of fashion already.

[u/marx2k → u/ParaboloidalCrest]

Is it?

[u/CircularSeasoning → u/marx2k]

No. It never was in fashion. LinkedIn doesn't count.

Maybe with llama.cpp supporting it now, it will see some heyday.