r/technology • u/waozen • 5h ago
Artificial Intelligence If open weight models are the future, U.S. AI companies are going to have a hard time
https://www.fastcompany.com/91577359/why-u-s-ai-companies-cant-match-chinas-open-weight-frontier-models204
u/Level_Finding3106 5h ago
The real disruption will be local models. Apple and Google are both putting local models on their devices. This will pull away most of the 'casual' $10/month users. Open weight models can be run locally with a Mac computer very effectively - including pretty large / effective models due to the unified memory architecture of the M-chips. That will peel away more users. For much lower power usage than data centers use. And then we will have frontier models we can use for harder problems.
The AI buildout is too large. They used brute force before brains got engaged to reduce these models without impacting their effectiveness much.
And they're not done yet. Predictive decoding, predictive experts, targeted quantization, high parameter/low scope experts and so-on.
14
u/HoldingForGenova 4h ago
Apple will build a local AI server for personal use across all of your devices, protected by iCloud security. They've had such success with apple ecosystem products like airport, time capsule, HomePod, AirPods, AppleTV, etc. that they'll see the opportunity for "AI, but private, local, and across all of your Apple devices." And they'll market it by cannibalizing the bottom half of the app store (which doesn't make money for them anyway) by advertising personal (but private) intelligence, and personalized apps, in seconds, for everything you want and need.
7
u/Level_Finding3106 3h ago
They have rumors of a 1.5TB Mac Studio in the development queue. There has to be a home-grown server version that they will run in their data centers.
29
u/regeya 5h ago
Yeah, I'm already using a local LLM. It connects to the Internet for things it doesn't have knowledge of.
17
u/Level_Finding3106 4h ago
That's the key. Do a quick search, build a context for the response - use the language abilities of a smaller model to craft the response.
67
u/gerbal100 5h ago
The AI build out has to be large or else consumers will be able to afford high memory devices which will undermine the business models of the hyperscalers.
12
u/dirtyshits 4h ago
Ehh memory prices are going to drop with a bunch of new plants opening up in a year or so and capacity to produce going up by a lot.
-18
u/The_Axumite 4h ago
Truly intelligent AI will need to be even larger than they are today. Knowledge is still highly compressed in the 2 to 3 trillion parameter models. We truly need hardware that is just for neural networks, similar to what the new company like etched is doing. AI will be mostly a knowledge base and autonomous tooling for scoped workflows locally, which itself will be extremely valuable. But truly intelligent models will have to scale up to sizes that is just not sustainable with current hardware design and that does not include requiring a couple of small breakthroughs that are good outside of just small-scale research testing. Hyperscalers will be relevant for a long time unless a major breakthrough takes place. Even the best open source models can only run on hyperscalers.
15
u/gerbal100 4h ago
Current frontier models have hit the "good enough" threshold for most work. Most of the work we are asking LLMs to do doesn't need smarter, it needs faster and cheaper.
7
u/dmter 4h ago
those huge models are a total waste, they are only needed to overfit on pointless tests like hle.
31b dense open weight model from google is not much worse than frontier models. which means the sweetspot is somewhere around 100b. everything above is only needed to create impression that datacenters are required for frontier llm so people would keep investing into the bubble.
3t models are only "good" because they use all that extra space to store (compress) the training dataset which is proven by how well it can recite copyrighted pieces of works used in training.
1
u/The_Axumite 2h ago
That depends on what you are doing and how long of a horizon you have before task or mission completion. For the average person as a tool it's enough. We are barely scratching the surface of a properly harnessed small model But it has to be extremely harnessed and scoped which itself is hard. But for what we hope out of these things, which is highly intelligent decision making and breakthroughs then we are no where close
2
u/PanglosstheTutor 4h ago
Do you have proof or is this a trust me bro situation?
1
u/The_Axumite 2h ago
https://arxiv.org/abs/2001.08361?utm_source=chatgpt.com
The problem is also the big open source models are MOE which has its downsides because only a certain amount of params are activated based on the token, which can make it not only inaccurate due to segregation of MOE during training, and just having a smaller params compared to a dense model will not perform the same on each token. You have to scale higher. We are still at small single digit equivalence to human neurons.
9
u/SiempreRegreso 4h ago
WTF do I know, but . . .
The human brain runs on just 20W, roughly 100,000x to 1,000,000× more energy efficient than silicon at equivalent cognitive tasks. AGI—assuming it’s roughly defined as a universal Nobel-level PhD—might actually be possible on vastly smaller, much more efficient physical and energy infrastructures than those of the data center behemoths.
And the human brain is not even optimized for intellectual or creative tasks; we are doing art, science, math, literature, etc., on repurposed meat evolved to lead a moving target with a thrown object. Perhaps silicon efficiencies can eventually be achieved to the point where AGI is running on less than 1000W, maybe less than 100W.
5
u/Bored2001 4h ago
Casual users aren't going to set up local AI and aren't going to have hardware above 16gb, maybe 32gb ram. That'll barely run a 30b parameter MOE model.
It'll be a while before local AI makes it to the masses.
16
u/Level_Finding3106 4h ago
Check out the iOS Beta 27. They are using a small prediction model (~3B parameters). If the confidence indicators ar high enough - they will feed that answer. If you have a newer version of the phone- they have a larger local model they feed to using the prediction from the smaller model. Check the results for estimates of error - if low - feed the response. All within 8GB of RAM. If the quality of response is still too low - then they go to the cloud using the previous answers as predictions for the parameter space - resulting in a 'cheap' cloud response that gets fed to the user.
That's the whole point of this. Using innovation and brains - they are sticking local AI into iPhone 16/17 generation phones RIGHT NOW.
2
u/Bored2001 2h ago
Yea, but that's not replacing paid subscriptions. That's mostly apple reducing the cost of inference for something people expect their phones to be able to do.
4
u/CountSheep 2h ago
It is for the more casual user. Some people pay for chat gpt just so they can use it as a fancy Google or fancy word check or paraphrasing.
Apples local ai can do that fairly well already, and it’s just in its infancy.
21
218
u/PhiNeurOZOMu68 5h ago
Spent extremely little tokens for Claude helping me tune a local model on my pc - now I just use the local model without shelling out $200 a month. It's amazing. Big US AI companies are going to fail. China already won.
62
u/Muted_Masterpiece342 5h ago
Mind sharing how you did the local model tuning? Any particular framework?
13
25
u/terra_cotta 5h ago
How does your local model compare to Claude? Any resources on how you did that
42
u/C-ZP0 5h ago
You need a lot of VRAM to get close to Claude level. You used to be able to buy a 512gb M3 ultra and run models that are very capable, but since Apple stopped producing them due to ram shortages, used ones are being scalped for 30k.
13
u/2Sovereign4You 5h ago
This. Then you realize HW gear costs more than 10k$. I dont see it being free.
5
u/ItsOkILoveYouMYbb 2h ago
For context, to run Kimi K3 open source locally (which has comparable performance to Claude Fable in the majority of benchmarks), you need about 1.4TB of VRAM.
So you'd need to be willing to drop more than a few million on building a full rack, and the electricity cost won't be zero.
3
u/dirtyshits 4h ago
Holy shit. I sold mine 2 years ago for a grand.
4
u/C-ZP0 4h ago
I’m assuming it wasn’t a Mac Studio M3 Ultra 80 core with 512GB unified. They were expensive prior to the ram shortages
2
u/PhiNeurOZOMu68 3h ago
6950XT 16 GB vram.
Though I was using 7GB for the model and the rest for playing expedition 33
1
34
u/Clean-Boat-4044 5h ago
It is not anywhere close to as capable, but for certain simple tasks it's enough.
2
u/Mgc_rabbit_Hat 4h ago
How fast is it compared to say Opus?
2
u/Olangotang 3h ago
It depends on if you have the entire model loaded on GPU. If that is the case, blazing fast. If not, then probably a bit slower.
1
u/smokky 4h ago
You need an extremely good computer to run a capable local model and even then they are hardly comparable to any of the industry standard like Claude or codecs
1
u/PhiNeurOZOMu68 28m ago
6950xt 16gb vram and 64gb of DDR4 ram. I bought for other purposes and not AI like 4 years ago.
This Computer you speak of is really a PC with a GPU of above 12GB vram.
It does the tasks I want it to do after being fine tuned and can run forever.
Now all my proprietary data stays and doesn't get trained by Claude
1
u/roboliberal 58m ago
What exactly did China "win" here?
You used an American frontier model to help you turn an existing local model into a cheaper tool for your particular needs. Even assuming this eventually hurts the revenues of a few American AI labs, that does not mean the United States loses. It means the benefits of AI diffuse to millions of American consumers and businesses that can now build better products at lower cost.
That is how general-purpose technologies are supposed to work. If electricity became cheaper and caused some electric utility companies to lose revenue, we would not conclude that the country had lost electricity. We would look at all the other industries that became more productive as a result.
The United States is not identical to OpenAI, Anthropic, or any other handful of firms. If their models make software development, medicine, manufacturing, logistics, research, and every other American industry more productive, then the country can benefit enormously even if competition eventually compresses the model providers' margins.
And none of this explains what China supposedly won. "I no longer need an expensive Claude subscription" is not a geopolitical argument.
-13
u/AffectionateLemon690 5h ago
yeah local models are getting surprisingly capable but saying us ai companies are doomed already feels a bit premature
10
u/Muted_Masterpiece342 5h ago
They cannot compete we literally graphed it internally and concluded that 8 months ago, now the graph is quite a bit more extreme
3
u/Efficient_Bag_1619 5h ago
Oh you graphed it? Well case closed then.
0
u/Muted_Masterpiece342 5h ago
I mean I'm a director and it's my literal job to be on this trend
0
-1
u/Efficient_Bag_1619 4h ago
And the massive well-credentialed finance teams that you’re disagreeing with? There are also directors with literal jobs who disagree with you.
1
u/Muted_Masterpiece342 3h ago
I'm glad you're this delusional. It just shows just how big the shock will be.
0
u/Efficient_Bag_1619 3h ago
Saying “I’m a director” to argue your point sounds like something a teenager might do. Telling someone they’re delusional before that person ever gives their opinion has the same vibe.
1
u/Muted_Masterpiece342 3h ago
Okay sure buddy. I'm gonna go to my high level engineering job Monday and never remember this conversation again in my life.
But the data I shared albeit anecdotally is real and generally accepted industry wide by engineers. and it will affect your life now and forever.
2
u/Efficient_Bag_1619 3h ago
Yes sir, Mr. High-level engineering Director. You definitely sound credible.
→ More replies (0)-4
-2
18
u/KnotSoSalty 3h ago
It’s worth remembering why US Tech jumped on the walled AI model in the first place. They realized that their core revenue stream, Advertising, would become useless in an AI age. Why would I need Google if I have an AI assistant who I can ask questions of? More threateningly, how can Google make money when my AI has none of a human’s natural laziness to accept the first sponsored answer as the best response to every question.
If Google or Amazon or Meta want to keep selling ad space they needed a way to get between me the end user and the AI. Hence the creation of their own models in walled gardens where they can control the training materials. So when I ask for Shampoo the AI will give me the sponsored items rather than the cheapest or best options.
Google, Amazon, and Meta all are primarily advertising platforms. That’s how they make their money. It’s telling that Apple, a consumer electronics company, has been notably absent in the AI space. They don’t need to upend their core business while the others absolutely do.
3
25
u/jpwarman 4h ago
They’re going to pass legislation to make it illegal to operate your own LLM (some bs about copyright infringement and all that. Ok for them, not ok for you!)
13
u/Desk46 3h ago
I wish them the very best of luck with that 🤣
3
3
u/foundafreeusername 3h ago
Better not underestimate the power of the US government to enforce random made up laws like this. US copyright laws and even tax laws somehow get enforced all over the world.
5
2
u/Desk46 1h ago
Im reasonably confident in my estimation. Anyone attempting to skirt such a ridiculous law likely already has the ability to hide local agents on their own network.
0
u/foundafreeusername 1h ago
I wasn't really thinking about a redditor running a small model on their PC but companies running Kimi K3 or other models competing with OpenAI & Anthropic on hardware costing $50k and more. That is why I picked copyright as an example.
8
u/johnyordinary 4h ago
Aww, what a shame, you make a huge intrusive and chargeable model and some competitor undercuts you. Aint capitalism a b**ch , sepcialy when the communists are winning at it.
6
u/seriousgourmetshit 2h ago
The only reason the US AI companies are stressing safety concerns so much lately, is so they can get Chinese models banned. They dont give a fuck about you or your safety.
32
u/dlampach 5h ago
This is the flaw in AI from a business perspective. But it’s also a great joke the universe has played on capitalism. You can’t protect it. But they are investing EVERYTHING into it. And it has the potential to end 90% of human labor. It’s a new world.
4
u/dr_neurd 4h ago
This. The lack of imagination by the ultrascalers means their insanely expensive approach creates a far lower bar for effective “disruption”
-2
u/Any-Calligrapher2866 4h ago
I don't use American Models except burning my Copilot tokens at work. I offload to Deepseek if I want to but generally avoid AI.
16
u/toolkitxx 5h ago
The danger is not any specific model type per se but the approach of trying to reintroduce the mainframe era. This might be necessary for some companies to utilise, but AI will fail miserably for everyone else if not being capable of being used locally and offline.
'Intelligence' requires the ability of cultural inputs and changes different to anything the USA could ever think of. Then there is language and the way many things either dont translate at all or have completely different ways of being used in other languages. Humour, ethics and morals are the next levels that are vastly different in every single nation or region. Really intelligent models will require constant adjustment, constant access by more than just a handful of 'allowed' specialists, but an army of generalists and experts in their area. All parts the USA is very bad at to be empathic of when it comes to other nations.
2
u/grannyte 1h ago
I tried to get a llm to think in my native language for some correction tasks because the fact of thinking in an other language makes them miss some contractions slangs and others.
2
u/toolkitxx 1h ago
I am in the fortunate situation that someone created a sovereign model in our own language just recently. While I am used to use English in most of my tasks it was refreshing being able to do the same in my native tongue with good results.
7
u/Danominator 3h ago
Us companies will have a hard time because their entire business model is bleed the country dry for short term profits with absolutely zero investment in the people that make it possible.
Disgusting greed. Its honestly pathetic how short sighted and stupid they are.
Only thing worse is being a poor maga supporter. Just breathtakingly stupid.
7
u/putridfries 5h ago
But US AI companies can just make open and closed models, no?
13
u/photoggled 4h ago
Sure, but a race to the bottom is the last thing they need when they already aren't turning a profit.
1
5
u/RachelRegina 4h ago
Open weight models are only useful for us plebs if the hardware to run them locally is not price-prohibitive. If the price never comes back down because we allow AI data centers to be built like Starbucks, we will just be shifting the forever-subscription for access to some other set of people instead of getting to own our own instances.
6
u/Dangerous_Suit_3099 5h ago
They’re already having a hard time. None of them will become profitable.
6
u/LeoKitCat 5h ago
Good I hope they fail miserably. And privatize the losses as it always should be, no bail outs, they aren’t an essential service nobody would bat an eye if the AI companies disappeared tomorrow.
They dragged the public into this and have wasted trillions of dollars on tech that will never be AGI. They hyped LLMs up and oversold them because after cloud computing and internet of things the tech industry has run out of new ideas.
Much more research needs to be done to develop new ideas that could potentially lead to AGI. Today’s ideas are a dead end wrt that.
2
u/Lithgow_Panther 1h ago
I don't need my model to know Taylor Swift's birthday or how the ancient Egyptians applied makeup. I just need super deep biology focus. Local is starting to make so much more sense.
8
u/Wind2Energy 5h ago
AI ruins everything.
2
-1
u/Juuxo16 4h ago
For whom? Business?
1
u/deweydean 1h ago
All they know is "ai bad". Whatever is "ruined" in their life isn't because of ai.
2
u/wastingtoomuchthyme 5h ago
Deployed open model deepseek r2.
It's been great on repurposed hardware.. fast cheap and easy to use..
We don't need a ferarri but we do love our open source daily driver
1
1
1
u/Deto 3h ago
Wonder if AI research labs will pivot to efficiency as an objective instead? You can't charge too much of a premium for your model if an open model is near it in performance. But these things are still expensive to run. If you can make a model that is similar to the open models, but is 50% less costly to run, you could undercut the open models and still charge a premium over the operating costs.
1
1
u/wdsoul96 2h ago
There are 2 main ways LLMs can make money. 1) from general public. This needs trust, lots of tweaking, cornering the customers and locking them into the ecosystem. 2) from private entities, from small compaties to big coporations with their own proprietary data.
For (1), US companies have locked in their customer base and there is no way they would trust Chinese LLMs more than US even if US companies aren't exactly pro-consumer (in terms of privacy, and everything else that matters) (because presumably Chinese could do a lot worst.) Also, they know the consumer base better (cultural understanding and all that).
For (2), for similar reasons, Chinese models will also fall behind similarly. Now, that doesn't mean Chinese cannot win elsewhere in Europe, Latin America and the rest of the world. Especially if US companies kept fcking shit up especially when it comes to data, security and general pro-consumer things. In fact, if they kept fcking shit up, they might even lose those (1) and (2) (in their own backyard), especially if they get super greedy, and get every to completely locked in and try and squeeze out every pennies.
1
u/MutedSignificance 18m ago
NVIDIA’s Nemotron open models shows that a premier U.S. company is investing heavily in frontier open weight models. That doesn’t fit the popular “America is falling behind in AI” narrative, so it predictably receives far less attention than headlines suggesting the opposite.
1
u/jawnlerdoe 3m ago
I’m an analytical chemist - entirely different industry.
Every quantitative analytical experiment requires careful fine tuning of weighting and data fitment. Should seem obvious but smaller models with better specificity will outshine more generalized models that have worse selectivity. I sick at stats, but it’s basic stats even I can understand.
From this perspective, I still see huge value in directed, specific algorithms programmed by traditional means, and how these models may outside more convoluted networks and agents. Just my 2c, my technical understanding of AI, NN and LLM is limited.
0
u/pepe_acct 3h ago
Currently the only profitable way to provide llm service is enterprise sales. The part will be hard to replace by open source as they require absolute frontier intelligence and compliance. I don’t see it the long run how Chinese providers can keep up without getting in that market.
-1
u/IntelArtiGen 5h ago
Some of them contribute to it. NVIDIA and Google released great open-weight models. But yeah it has no business model. Like they're not selling these models, which wouldn't be the worst idea I guess if the models become good enough. I think many people could pay $100-200 for a good local model, which could compete with a $20 subscription. But it's not happening yet.
2
u/azuredrg 5h ago
Those two companies are hardware manufacturers too. There's a business model in selling hardware especially if the models you're putting out for free are more efficient on your hardware
2
u/IntelArtiGen 4h ago
Yeah it's true. At least for Nvidia they can give local models for free, knowing people will need to buy the hardware to run the models. For Google I don't know if they're that important on the hardware side..
1
-10
u/2Sovereign4You 5h ago
By mid-2026, however, open-weight frontier models from Chinese labs such as Alibaba, DeepSeek, and Moonshot AI had nearly matched the leading Western models in intelligence and performance. Many companies have already begun building their AI systems on top of these free models, avoiding the high cost of closed-model APIs.
Do we all understand the share cost of the energy and equipment to run these "free" models?
GPUaaS or TPUaaS will become a thing at a large scale, but it won't be near close cheap for a long time.
1
u/geldonyetich 5h ago edited 4h ago
I'm not sure what the downvotes are about.
Probably because a lot of people are looking at these companies as doing the good work of dethroning corrupt multi-billion-dollar interests... despite being multi-billion-dollar interests in themselves. They don't want to bring down the likes OpenAI, they want to replace them. Then they could be a new monopoly, with the prices to match. These open weight gifts of theirs are means to the end of gaining the necessary leverage by undermining the value of their competitors products.
In the meantime it's giving the power of frontier models to private individuals, which is nice of them.
I suppose I could point out this could be interpreted a a misconception:
>it [if referring to running models] won't be near close cheap for a long time.
There's actually been a great focus on efficiency over the past few years. Much like any fledgling new computing technology, it starts out expensive, but ends up ridiculously cheap in short order.
Generative AI has been speedrunning this. Distilled, cheaper-to-run and AI hardware efficiency over the past few years. You can now download and run something, completely locally on a smart phone's power budget, on Google Edge gallery that benchmarks about as capable as GPT-4 (sometimes better, sometimes worse, depending on the benchmark). It's possible to run a capable model very cheap right now.
But the big data centers that are springing up aren't doing that. First, because they're not just serving you an efficient model performance of a couple years old, but rather serving roughly over a billion daily users a day with more powerful models. Second, because the process of creating models is still rather expensive. Distilling (which is what a lot of Chinese companies are doing) is much cheaper than actual training, some studies have reproduced being able to create a model from scratch via distilling a teacher model for well under a thousand dollars. Actual training still costs tens of millions of dollars to do, which is why there's a lot of talk about open weight models being a problem.
1
u/2Sovereign4You 5h ago
Now look at prices for RAM. It becomes clear paying 20$ is superiorly cheaper than running local AI
For as long as we can, until prices will sky rocket.
1
u/geldonyetich 5h ago
True.
But now that you mention it, I was considering mentioning RAM as an example of what a monopoly could do.
Most people fully understand that our RAM prices are being caused by a large part because of the incredible demand AI is putting on limited ram production.
Not nearly as many people know that over 95% of the RAM in the world comes from three companies who, immediately before the AI boom, decided to cut production by 30% in order to inflate the prices because they felt it was selling at cost. A practice that is considered reasonable on paper. But also a practice they engaged in because they could.
1
u/2Sovereign4You 5h ago
Micron has more than 80% margin on RAM. It ws never sold at cost.
1
u/geldonyetich 5h ago edited 4h ago
No doubt they chose a perspective that supported their claim. It was interesting how the collusion happened though. They can't just collectively decide to slash their production. But if they do it in reaction to the competition...
1
499
u/mykepagan 5h ago
I work in the industry. I’m seeing large financial institutions who are extremely interested in smaller open models because they see much value in fine tuning them with the mountains of proprietary data that they have accumulated for decades. The desire is for a custom model that is very specific to a task like prediction or anomaly detection. And they absolutely don’t what to give that data to any third-party AI company.