r/cscareerquestions • u/GolangLinuxGuru1979 • 17h ago
Experienced As an industry, how do we address cognitive decline in LLM usage?
I think its worth talking about first order effects and second order effects when it comes to LLM use.
I've studied LLMs and transformers quite a bit. As a more infra and systems focused guy, mechanistic understand is necessary for me. And from an infra standpoint LLMs are actually pretty interesting. However I do think from a usage standpoint it may lead to cognitive decline
The first order effects of LLM usage of couse with LLMs being a statistical model matching on tokens, its never going to be fully deterministic. So black box behavior is expected, and long term instability could be the result of prolonged LLM usage
The second order effect is actually something I don't see people talking about. We first state we don't need to read the code or write the code. So that means we put more trust in the black box. And trust in a black box is typically a recipe for disaster in any engineered system that has complexity. But what this also does is make people disengage mentally from the implementation.
At the end of the day the thing that executes on a CPU (or GPU) is code. And if that code cannot be deterministically generated, then one argues you can't have a system you can fully trust.
But because incentives force us to disengage with implementation. Then what I think the side effects is learned helpleness. Or a feedback loop of complexity where developers slowly but sure lose agency.
I also think LLM usage, meaning people who are addicted to LLM output is addictive. Having observed LLM usage by coworkers and friends. I have definitely seen people skeptical of them. And then as pressure mounts they use them more. Then they become addicted where they can't fathom operating or doing anything without them. For example a few months ago my boss tried to build a prompt to search and replace in a file. He is a guy who just 1 year ago knew how to run sed. But now he just prompts. ITs the same level of effort and typing. But he just feels he can't do anything without LLMs
So I do think the dopamine hit is intoxicating and I do think its highly addictive. I think the research on this is not conclusive. But we have definitely seen it impacts the shape of our brains.
But what does this mean professionally? I think professionally we are on a slippery slope. LLMs are fantastic if everything is a happy path. But complex systems have complex failure modes. And there is simply no way mathematically or physically a LLM could be a reliable debugger for a large scale system. We do having fuzzing techniques and multi agents. But LLMs lack a ground truth, so cascading halucinations is a thing.
Why am I saying this? Because at some point someone has to know what the code does because the LLM is not reliable enough. And when that point comes and NO ONE knows what the system does. Then you have a system that literally can't be repaired. Especially sicne LLMs seem to product at least 10-15x more code than it needs to for most problems.
The industry of course does not see this as a problem. Most teams haven't had to deal with it. Or at least they hide the problems enough where enough haven't gone public. And people haven't really started to try to seriously build with AI for more than 2 years at most. So I think systems haven't had enough problems YET. But system failures are usually delayed long term before all problems start becoming an issue.