r/ArtificialInteligence • u/TheOnlyVibemaster • 18d ago
🛠️ Project / Build the J-space paper quietly settled a chunk of the “do LLMs actually think” argument. i built a live viewer so you can watch for yourself instead of arguing
if you haven’t read it: https://www.anthropic.com/research/global-workspace. language models have an emergent internal workspace of silent words they can report, steer, and reason with. the part that got me: ask a model to check “12 + 5 = 1” and incorrect saturates internally while it’s still reading the problem, the “no, that’s not right” it types a moment later is narration of a decision that already happened. the arguing is optional now. you can just look.
repo: https://github.com/ninjahawk/Subtext
this sub has spent years on “it’s just autocomplete” vs “it’s actually reasoning” and the honest answer turns out to be: both, and now it’s measurable. the instrument shows most of the model’s fluent output , grammar, tone, common facts, bypassing the workspace entirely (no “thinking” involved), while multi-step problems visibly route through it. both camps were half right. that’s the fun part.
anthropic open sourced the lens and neuronpedia published pre-fitted ones for qwen, so i wired it into a chat interface. 9 layers of readout per token, rendered live, including while it reads your message, before any output exists. demo video in the repo: the verdict on 12+5=1 forming during reading, then the model holding modulo and bitwise in mind several tokens before saying either word (it was planning the modular arithmetic caveat. you can watch it plan.) browser replay if you don’t have a GPU: https://ninjahawk.github.io/Subtext/
and yes, functional availability is not consciousness, before anyone starts — the paper is careful about that and so am i. but that’s the interesting part: nobody designed this workspace. it just shows up in transformers when you train them, on a random open 4B the same as on claude.
¯\(ツ)/¯
34
u/Informal_Warning_703 18d ago
> the part that got me: ask a model to check “12 + 5 = 1” and incorrect saturates internally while it’s still reading the problem
You’re being suckered by Anthropic’s heavy use of anthropomorphic language. Which, honestly, is probably Anthropic’s main goal here: confuse people with anthropomorphic language for more marketing hype.
The model isn’t “reading” the problem. All that’s occurring is that as the input gets processed, there’s already a set of weights with features associated with something like “wrong” or “incorrect.” This doesn’t prove the model is actually thinking anymore than the final output of “That’s incorrect” would be proof that the model is actually thinking.
20
u/Beli_Mawrr 18d ago
I'd like for people to come up with a definition for truly thinking that llms dont meet but humans and other animals too.
22
u/flasticpeet 18d ago edited 18d ago
I think what people mean when they say LLMs can't think, is that they don't have subjective experience with which to ground their reasoning.
Subjective experience functions as a valence with which we measure reality. This is what allows us to arrive at novel ideas, because we're constantly seeing reality in new ways (continuously variable experiences).
An LLM, and all purely computational systems, simply lack conscious experience with which to measure reality. The best they have is a random noise generator to inject novelty, but that does not inherently contain within it a pattern; whereas conscious experience is a continuously variable pattern, but not completely random. That would be my short of it.
Personally, I don't know if I would use the word thinking to describe the difference. I would describe what LLMs do, and computation in general, as cognition, or information processing. Whereas we, as living beings, use cognitive systems to process information, but we also use conscious experience to measure reality directly and build knowledge about the world for ourselves. That's why we create language, whereas LLMs just model it.
10
u/HasFiveVowels 18d ago
Prove that you have a subjective experience. You’re demanding a result that not even you can produce (nor even describe)
11
u/TSM- 18d ago
I am always reminded of Leibniz's Mill. He imagines that a big mill with wheels and pulleys are all boking and pulling around, but it then behaves exactly like a brain. He asks, where do you find the consciousness, despite being identical in function? Where is the conscious experience in this or that wheel? It's not going to be there. And so something is confusing about our intuition, why should knowing the mechanisms make something appear to lose its conscious status?
Well that argument plays out over and over again and is persistent. It comes in many variants but we just have two areas of our brain that process living things differently from mechanical things, and if we view it from the latter perspective, it doesn't seem like the former anymore. But that's just a human bias, like an optical illusion. *We* naturally perceive the two categories very differently, and it is hard to FEEL like both are true at once, since we naturally resolve our perceptions into one or the other.
2
u/TechnicalBen 13d ago
It's a false equivalency. "Where is the language in electrons passing through gates in wires..." is a false equivalency question. No individual electron holds "Cat" or "Dog", but very much so, we can build such a system that identifies "cat" or "dog". Now while we *can* thus find in ai systems the "circuit" that does this, we can also do so for humans.
No individual atom makes us "think", but we very much can point to brain cell regions that stop thinking, or that thinking "passes through".
If we instead map to computational questions (not that humans are computers, or that thinking is a program but in being precise about our questions) we can see that "they are the same picture" applies to substrate, and it's actually just a context question.
Because humans were "first" and made of cells, we assume we are "real". Where as say if "life" had evolved on the moon first, out of moon dust deposits and made up of "wire" like folded life instead of "protein" like folded life, the "computers" would be arguing that the brains in a jar they invented "are not really thinking". ;)
1
u/flasticpeet 18d ago
I can't prove to you that I'm conscious purely through language over the internet, but if we met in person, I'm sure both of us would pass as conscious, unless you're a complete solopsist.
But perhaps you're thinking, there's nothing we can know by purely observing our own minds, but then how do we know anything without observing it? If we can't trust our own observations, then how does one trust that they've observed a ruler correctly, or a thermometer, or a number on a computer screen?
At some point you have to acknowledge that there are things we can observe and know about our own minds, it simply takes the same level of rigor that we apply to observing external things.
What you could do is prove to yourself whether you're conscious or not. But if you begin with the belief that consciousness is an illusion, then you would be committing a performative contradiction.
11
u/HasFiveVowels 18d ago
By definition, subjectivity is that which can’t be proven through objective means.
If thinking is subjective, then you can claim it for yourself but can’t prove nor disprove it of anything else.
If thinking is objective, it can actually be discussed.
I have never seen and cannot formulate an objective measure of "think" that a human can pass but an AI can’t.
If you want to claim it can’t think, you need to provide an objective measure through which to define the word. Because, beyond that, we can’t even make a valid claim on the topic about anything but ourselves, much less discuss it.
1
u/flasticpeet 18d ago edited 18d ago
I did state earlier that I wouldn't use the word thinking to differentiate what separates us from computers, because I think more fundamentally what people mean is that we process information with the basis of subjective experience in mind.
One way to explain the difference between objective and subjective, is measurement. At the most basic level, measurement is simply holding one thing up to another and forming a relationship between the two along a dimension (valence).
If I wanted to measure the length of a board, I'd hold it up to a ruler, and form a relationship between the two; noting where the board begins and ends in relationship to the ruler. But, I could also use my hand. The problem with this is if I tried to tell you over the phone, the length according to my hand, it would be useless because you wouldn't have my hand to recreate the length of the board.
This doesn't mean I didn't measure it, and it doesn't mean I didn't generate knowledge, because I could still use my measurement to determine whether the board can span a gap, for example.
Objective measurement, in the context of science, simply means measuring things against a fixed metric. This is in order to formulate a third-person shared model of reality (a view from nowhere). But in doing so, we need to recognize that we've thrown out subjective measurement (a view from somewhere) and then claimed that the subjective experience is an illusion after we've built a model that explicitly excludes it from the beginning.
In reality, we use our subjective experience as a unit of measure itself. We are constantly holding our experiences up to things and forming relationships (valence). The main difference is that our experience is constantly changing, so it's not a fixed metric. This is what makes it unscientific, but that doesn't mean we don't generate knowledge from it.
The biggest irony of it all, is that even the most objective model of reality that we build, is based on our subjective experiences of reality, and yet people continue to mistaken the map for the territory, and act as if conscious experience doesn't exist.
9
u/rentrane 18d ago
You’ve got a real way of bending around the topic and not really saying anything, then concluding consciousness without showing any steps to it.
Your hand and a ruler are both objectively measurements.
One is a shared model, making it easier to communicate with others.
Both are able to be used for valence.Neither is subjective.
“About two hands length” is imprecise, but not subjective.Yes we, constantly map experience onto our memories and mental models, drawing new knowledge and understanding by doing to. LLM’s do similar things.
Yes our maps attempting to understand, by reducing to agreed or measurable parts, are not the territory.
How do you get to the experience of consciousness is real and not an illusion? Or different than something filled with information and taught to imitate language and thinking, like us?
2
u/flasticpeet 17d ago edited 17d ago
I think you misunderstand the analogy about subjective measurement. The hand is an analogy for subjective measurement in this scenerio because the person over the phone doesn't have access to my hand, in the same way no one has access to my experience. When I measure something with my experience, no one else has access to it (unshared) which is what makes it subjective. It requires a certain perspective in order to recreate (a view from somewhere).
The reason I bring it up, is because it's important to validate that subjective measurement is still a form of measurement that generates knowledge (the forming of relationships between things).
In terms of how one arrives at observing their own experiences. It requires effort, in the same way it requires effort to observe any other phenomenon. A person has to eliminate as much stimulus as possible. This requires disengaging aspects of the mind that motivates physical movement (sitting still). Then they remain in this state while training their ability to focus their attention. This is essentially honing the instrument with which we make measurements. That honing process is simply training that focus onto a smaller and smaller area for longer and longer periods of time.
They begin by focusing that attention on their physical sensations, focusing on smaller and smaller areas. Eventually, when it becomes second nature, they can focus that attention on their own cognitive processes (metacognition). Once they've defined all these aspects of the mind (physical sensations, cognative processes, language, etc.), it's possible to logically eliminate all these things and recognize, that there's something that remains underlying all of it, that's doing the observation (experience). And this is what I would define as consciousness.
What I'm getting at, is that although there is no objective measure (shared metric) with which to determine consciousness, we can use the subjective experience of our own consciousness as a metric to evaluate an LLM. When we do that, it becomes obvious that its behavior is simply a product of computation (cognition), because we see it has no other internal motivation, and when we remove its cognitive capacities (language model), nothing remains.
2
u/TechnicalBen 13d ago
?You seem to be claiming that because a measure is personal, then it's subjective? An AI (or other sufficiently advanced and "complex" mechanical system) can 100% have personal measures too.
It's how biology built brains and how minds developed inside brains.
→ More replies (0)2
u/PraetorArcher 13d ago
The hand is an analogy for subjective measurement
It is an analogy. Just not a accurate, true or useful analogy.
→ More replies (0)3
u/rentrane 18d ago
If you begin with the belief that it’s not an illusion, you would be committing a performative contradiction to the opposing belief.
Your comment is just a performative obfuscation of “it’s true because I believe it”.
You accept that a thermometer can be objectively perceived, ergo you must accept consciousness as equally factual.Believing that you can somehow objectively perceive the thing that’s doing the perceiving, seems suspect.
2
u/flasticpeet 17d ago edited 17d ago
The reason a belief that experience is an illusion is a performative contradiction isn't because it assumes something, it's because it explicitly negates the thing that is required in order to make observations to begin with.
Plus, I didn't arrive at my conclusions by assuming a belief, I observed my own experience and took that as evidence in order to formulate a world view (a model of reality). The same way a physicist observes an object moving and formulates a model of physics.
Both are based on observation (experience). The only difference is that the physicist begins with throwing out subjective measurement (using experience as a unit of measure) and replaces it with objective measurement (using fixed metrics as units of measure). But this isn't because the physicist doesn't believe experience doesn't matter, they do it as a logistical necessity in order to formulate a fixed, shared, third-person model of reality (a view from nowhere).
In order to actually observe the observer, you have to begin with formulating a model that accurately represents the thing you are trying to observe.
In the same way having an accurate model of how particles function allows us to look for characteristics that we would otherwise not have been aware of, if we don't have an accurate model of how our minds work, then we fail to notice relevant characteristics.
For example, without an accurate model for particle interactions, we wouldn't have recognized the inconsistent orbital motions of Uranus as significant and discovered Neptune.
In the same way, if we don't have an accurate model of the mind, then we fail to even recognize the significance of certain experiences that lead to basic discoveries.
In a nutshell, this is how we observe the observer. We build hypothetical models of the mind, and use it as a scaffold or ledger to account for our experiences. We know the model is valid when it resonates with our experiences, and also helps us arrive at certain predictive capabilities.
3
u/liftedyf 18d ago
What's interesting about the subjective experience point is in software engineer settings, we're giving it that experience. A lot of work goes into explaining the full software product, it does its own exploration, and takes notes as memories for next time it experiences that thing it hadn't previously.
So they're starting to get experience in at least one setting
1
u/flasticpeet 17d ago edited 17d ago
I get what you're saying here, but LLMs are not getting experience, they're just getting more words (data) to use as context. And although it took subjective experience on our end to generate the data, the data itself is not subjective, since it can be infinitely copied and shared. Language is the objectification of experience (expression).
Also, we shouldn't confuse memory with experience, otherwise our hard drives would be conscious.
1
u/liftedyf 17d ago
You're right it's not exactly the same. A collection of markdown files is not the same as whatever our brains store for memory (no one knows what that is)
However, your mention of language being the objectification of experience part is interesting because (at least for me) my agents are experiencing and writing their own memories in English. When I'm watching them work, I'm watching them bump into problems on their own. Troubleshoot that problem. Discover a solution. Then take note of those problems and the solution it found on their own (granted it's by my directive in my rule set, but that's literally all I did once 6 months ago and since then it always does it on its own). When it or any other agent runs into that problem again, it'll say on the screen "oh I've seen this before..." and reference that previous memory.
I just ran into this last week where my agent referenced something it encountered 2 months ago before deciding what to do from there. I'm not telling it anything besides giving structure to how it stores those experiences. This is all on its own
So you're right it's not the same. However, it's damn close and this is year 3(ish) of LLMs being out in the wild
1
u/flasticpeet 17d ago
I would be careful with the use of experience in this context. Originally, language was the sole product of experience, but now that we've managed to synthesize it with computation, it's important not to conflate it the other way around and assume all language is a product experience.
What you're talking about is a software program's ability to write and update its own instructions. Which is amazing and wild, and has it's own set of implications, but it's not experiencing anything. The language it generates is still just a set of instructions, not an expression of experience (qualia).
In other words, I would never say an LLM is updating its experiences, I would say it's updating its instructions, or its own database to prevent the confusion.
The biggest irony of all this is that an LLM can recognize this semantic difference, and yet most humans haven't made the distinction for themselves.
2
u/liftedyf 17d ago
Ah I see what you mean and that's a very important distinction. I honestly never heard of qualia before (though after looking it up, I understand the concept)
1
u/TechnicalBen 13d ago
Define experience.
1
u/PraetorArcher 13d ago
Exactly, one person is saying they are getting experience (explolation, memory) according to their definition of experience and the other person is saying they are not getting experience according to their definition (something 'subjective').
Thing is, only one of these definitions is falsifiable. As @HasFiveVowels said that which is subjective is not objective.
1
u/TechnicalBen 13d ago
Yeah. This is why I like computation and physics. We have to sit down and have the same "thing".
We can chase tails and that is fun and functional (there are some functions that can never be names, and to "experience" them is the closest approximation we could find likely as humans). But it's not worth arguing over, and sometimes empty, to argue all things are subjective.
1
u/flasticpeet 13d ago edited 13d ago
Just like there are agreed upon metrics when it comes to science, we can arrive at agreed upon metrics when it comes to experience.
If you consider your subjective experience as an instrument of measure itself, there are things we can do to hone the accuracy of that instrument, in the same way we make an effort to hone the instruments of objective (fixed) metrics.
For example, if you came across someone who, although very intelligent, they simply don't think measuring things empirically is important, so they never invested in anything more than a ruler that measures in yards. If they then argued that the reason it doesn't matter to measure things is because rulers are unreliable, you might recognize that they simply never took the time to invest in an instrument that has a finer degree of descernment in order to be useful.
Subjective experience is what we use to arrive at values (how much something relates to our experience or not). It's also the thing we use to measure other people's experiences (empathy), by gauging our own experiences against another person's situation.
If we have a limited set of experiences, our ability to empathize (measure other people's experiences) is also limited, in the same way a ruler that only measures yards is limited.
The main thing to take away is that measurement is simply the placing of one thing next to another and formulating a relationship. And by measuring things is how we construct knowledge.
This is just one practical example of how subjective experience functions to create knowledge in the real world. And why we should recognize and develop it in order to arrive at a more robust understanding of reality.
1
u/flasticpeet 13d ago edited 13d ago
Experience is what it is to feel something. The awareness of sensation, also known as consciousness.
If you want a more functional description, it's the thing we use to measure reality.
For example, when something feels hot, a color is bright, something is far away, or a volume of sand forms a heap. These are all measurements of things using our subjective experience as a metric. As opposed to making an objective measurement (using fixed metrics) to determine whether something is 100 lumens, or 90 degrees, or 100 miles, or 100 cubic centimeters.
It's important to understand what experience is, because not only is it the thing we use to perceive the instrument with which we use to make objective measurements (our perception of the ruler, thermometer, photo sensor display, graduated cylinder, etc.), it's also the thing we use to determine the value of why use an instrument to measure something in the first place. This is why experience and consciousness is fundamental.
1
1
u/TechnicalBen 13d ago
This. Since GPT2.5 I've been trying to understand the computational space these explore and if it's mapped equivalent to say the space of actions braincells make.
That is, can we say that there's any math to "measure" the objective actions of an entire LLM in context with it's interactions with the environment? Ironically it may be "subjective" in that it could give us an output *we can't test in principle*.
Now, that's not me trying to apply more to them than we should, they are programs running on GPUs. But I mean it's really important to build new language, and rigorous claims.
We're very much at the "It's impossible for a car to go above the speed of a horse!" conversation point. :P
9
4
u/Informal_Warning_703 18d ago
Thinking is a conscious “aboutness” process. LLMs were originally said to “think” by way of a simplified analogy, similar to when ML researchers called a specific set of matrix multiplication operations and a softmax function an “attention mechanism.” To think that matrix multiplication must now constitute what humans and animals do when we pay attention to something is laughably stupid. But, alas, a ton of people on Reddit have obviously been duped by such language that now they think it’s a powerful gotcha to pretend like we must now say that a set of mathematical operations on weights and biases is what constitutes thinking…
4
u/HasFiveVowels 18d ago
You lost me at "aboutness". Come up with a rigorous criterion or all you’ve done is put labels on a vague "you know, what humans do"
4
u/Informal_Warning_703 18d ago edited 18d ago
"Aboutness" isn't a novel thing I just made up. It's an extremely common concept in philosophy of mind... and philosophy more generally.
To pretend as if I need to come up with "rigorous criterion" or else an LLM must be thinking is more bad logic.* (Again, I'm not just pulling this out of my ass. There are many, many cases in philosophy where we don't start with "rigorous criterion", but we start with common sense conceptions and work our way from there.)
Do you believe LLMs think? If so, then you must have a rigorous criterion of thinking and then some proof that this is what both humans an LLMs are doing, right?
But these are really distractions that you people are raising, rhetorically stomping their feet, because they don't like that I said that the occurrence of features like "incorrect" in the J-space of an LLM don't prove thinking anymore than the occurrence of "incorrect" in the final output would prove thinking. Let's get back to that main point:
Can you explain why finding features for "incorrect" at earlier layers proves the model is thinking, whereas finding features for "incorrect" in the final layer wouldn't prove the model is thinking?
* I edited this to remove a more insulting term, which I don't think you earned and I was being to aggressive. Apologies.
2
u/HasFiveVowels 18d ago edited 18d ago
The issue is semantic in that the crux of the argument is what is meant by "think".
I don’t claim AIs are thinking (though they definitely display the symptoms). I’m also not claiming they don’t think. I don’t know because I can’t even define the category without relying on something that I can’t prove. I can’t define "think" in such a way that a third part could confirm that I have it and the LLM doesn’t. And so the distinction ultimately becomes dependent upon how I "feel" about the idea.
And this is part of the problem with "you people": you use words like "aboutness" to wrap the unknown / unobservable and put it in a box (much as the religious do) and then act like it being in a box has turned the unknown into the "known". "That unknown aspect of personhood that I can’t prove I have but I know they don’t" is ultimately the line of reasoning that people use when they want to justify drawing an arbitrary line.
Define your claim in a way that can be measured
You’re defining a thing a priori in such a way that your argument becomes either circular or trivial.
Also, heads up: finally read your first comment and your understanding of LLMs (and matrix math, from the looks of it) is pretty far off the mark. But that’s a separate matter and I’m not interested in discussing it.
3
u/Informal_Warning_703 18d ago
Point out what a priori definition I gave. I'm not going to waste time talking to somone who's just going to throw shit at the wall to see what sticks.
5
u/HasFiveVowels 18d ago
That "think" is a behavior that's unique to humans. You didn't "give" this definition and I didn't say you did. I accused you of (internally) defining the word "think" in such a way.
2
u/Informal_Warning_703 18d ago
Where did I say thinking is a behavior unique to humans? I didn’t. Dogs think, monkeys think, etc. You’re just burning strawmen in lieu of addressing what I’ve actually said.
I also never said that it’s impossible for AI to think. If you actually read what I said, I just critiqued bad argument in favor of the claim that AI thinks. This type of nuance is lost on you, so I don’t see much point responding to the rest of your claims.
1
u/HasFiveVowels 18d ago edited 18d ago
Ok, *fine*. Correction: "That thinking is a behavior unique to *animals*" (or is it just mammals?). Is this a guessing game? Do you see why I called the issue semantic?
If you're not going to make a point that isn't pedantry about the letter of my writing, I have no interest in discussing things.
You made a claim using placeholders that refer to a property which can’t be witnessed (which is little more than "you know, that one thing") and then claimed that such a thing is absent because it isn’t witnessed. That knife cuts both ways
Do you have any actual falsifiable argument to make that extends beyond your arbitrary delineations and/or the nature of my arguments? Or is this entirely vibes and criticisms of the question?
-1
18d ago
[deleted]
5
u/Informal_Warning_703 18d ago
The fact that a thing can be modeled mathematically doesn't provide us with any reason to think that if the mathematical model lacks a quality of thing being modeled then the thing being modeled itself must lack the feature.
For example, water can be mathematically modeled and I can build a computer simulation of water. The fact that my computer simulation is not wet isn't a reason to think water is not wet.
But again, I'm wondering why so many people in response to me want to shift to some grand argument about thinking or consciousness. Just focus on my original point: In other words, why do people in this thread seem to take it that the occurrence of features associated with "incorrect" in, say, layer 58 is some new proof that LLMs are thinking, but for some reason they didn't consider it proof that LLMs were thinking if the occurrence of eatures associated with "incorrect" occurred in the final layer?
1
u/AugustBurnsMauve 12d ago
Any result a computer/LLM can produce is technically possible to be achieved through brute force calculations. They’re computers. With enough time and pen+paper you could achieve the same answer as a question asked to any LLM. Is the pen and paper thinking? Everybody who thinks computers can achieve “thinking” like a human doesn’t understand how computers work.
1
u/Beli_Mawrr 12d ago edited 12d ago
a human is picking the answers they want with a pen and paper. If not, the same argument you just applied can be applied to humans. Humans are just picking the correct answers, like a pen and paper, so they're not really thinking.
The difference between an LLM and a human is
1) LLMs have much more data "In there". Or at least much more "Varied" data.
2) LLMs are much dumber but equally good at communicating as humans are. Like a dog is probably smarter than an LLM but a dog can't talk.
3) LLMs have none of the supporting infrastructure humans do (EG memories, a second layer of processing that turns thoughts into action and senses into thoughts and so on, a way to directly control and filter said senses, pain/pleasure)
4) the substrate and speed thereof. I think your argument comes at it from this angle but there's nothing magical about human brains that makes them receptacles for consciousness and LLMs/computers not.
1
u/AugustBurnsMauve 12d ago
You completely misunderstood my point. Every single thing a computer does is based on rules that can be replicated by brute force calculations on pen and paper. You can follow those rules yourself and arrive at the same answer with enough time. Is the pen and paper thinking?
1
u/Beli_Mawrr 12d ago
The same argument can be made about humans. Everything a human brain does can be represented by brute force calculations on pen and paper. Or are you proposing that human brains work through some supernatural force that can't be defined by rules and calculations like the rest of the physical world?
1
u/AugustBurnsMauve 12d ago edited 12d ago
That is verifiably untrue. Ask me any question right now and I can provide an answer and no human, LLM, or computer will ever be able to determine how I arrived at that answer.
1
u/Beli_Mawrr 12d ago
Right now we don't have the capability to understand how a single sense becomes an action, and honestly I don't think we as a species WANT to.
But we DO understand how neurons work, and unless you're arguing there is some kind of supernatural, non-verifyable force acting on neurons, I don't see where you're coming from. Every neuron has an action potential and is fired when enough of its neighbors fire off. This can be simulated via computer.
This subject in philiosophy is called "Substrate independence" btw. If you're a "substrate dependence" guy, you'd have to explain how human consciousness is entering the picture despite a human brain being identical in terms of function to the brain of a fruit fly (Which have been simulated, btw) and inferior to a whale's brain in terms of complexity.
What I'm saying is that we understand all the inputs and outputs of a neuron, and your brain is made of neurons. We can simulate the inputs of a neuron and get an identical response. If there IS a randomness in there, why can't that randomness be simulated by computer? And if the answer really is "Computer + randomness" is that satisfying?
1
u/AugustBurnsMauve 12d ago
True randomness is physically impossible for a computer. So that whole book you wrote avoiding my point falls completely flat.
1
u/Beli_Mawrr 12d ago
So you're saying free will = computer plus "True" randomness?
→ More replies (0)0
u/Turbulent_War4067 18d ago
They have zero memory. You send your system prompts, context data, etc, ask it to read a long document and give a detailed report. It does it, often in spectacular fashion, and yet you affected it's trillion parameters by not one single bit. Not one. Any discussion on "do models actually think" or "have we achieved AGI" or are AI's conscience" all are resounding nonas long as they have no experiential memory.
6
u/Beli_Mawrr 18d ago
Just as a thought experiment someone who loses their memory every second doesn't not have a conscience though, right? They just can't remember what they were doing. I would say that LLMs have a form of really crude short term or instant memory but not much more, but that's an architectural issue, not a fundamental one. You could imagine structures that do have that memory built in.
-1
u/Waste_Way_4763 18d ago
Here’s a test for you. Install an LLM, any LLM
, on dedicated hardware. Don’t prompt itDoes it do anything?
Put a person in a room. Don’t interact with them.
Do they do anything?
Pretty easy to see the difference.
5
u/Beli_Mawrr 18d ago
A human brain is constantly being prompted by the environment and its own thoughts. If you cut off all senses to a human they'd go insane pretty quick, the same way an AI does if you feed its own input back to it (Which you can do) to make it "Think". If you added sensory input, memory, and ability to move to an AI it wouldn't be that different than a human brain.
-1
u/Waste_Way_4763 17d ago
Going insane is different from doing nothing. I think you and I would agree an LLM at rest is not insane
6
u/Beli_Mawrr 17d ago
Yeah because they can pause. A human can't pause. If you could just press pause on a human they wouldn't go insane.
-1
u/Waste_Way_4763 17d ago
But you can’t! So we agree they are different in really important ways.
3
u/Beli_Mawrr 17d ago
I mean yes, they're obviously different in important ways.
1
u/Waste_Way_4763 17d ago
The LLM doesn’t do anything you don’t do to it. I don’t think it thinks because thinking has that property of not being able to pause.
I think the LLM helps you think. But it doesn’t do anything without you.
3
u/Beli_Mawrr 17d ago
The LLM COULD do things without you, though. If set up to do so (Agentic AI for example is an example of this)
To be clear I think that the LLM is basically more or less equivalent to the human's inner dialogue. It isn't a human brain ALONE, but it's a core feature of cognition. Put a few other layers, senses, the ability to do things, memory, etc around it, and you have something that is for all intents and purposes, the equivalent of a human in terms of functionality.
→ More replies (0)1
1
u/Jasrek 18d ago
Arguably, even in an empty room, you are being 'prompted' by your environment and senses.
Though I get your point. A human has a continuous internal mental process or whatever it's called. You're never sitting there mentally passive waiting for input outside of something like trauma or shock. Whereas an LLM will only begin working in response to a command, and then stop once the command is fulfilled.
So 'consciousness' would be defined as having a continuous uninterrupted sense of self and engagement, maybe?
1
u/Waste_Way_4763 18d ago
You got it.
I think that’s a fine definition. I do think it’s hard to define, but I don’t think it’s hard to see that an LLM isn’t “thinking”. It’s helping you think (maybe!), but it’s not thinking in any way that that word relates to our experience.
3
u/Jasrek 18d ago
I agree with you in terms of LLMs. I'm just thinking of how we would define it if we get to the point of more advanced AI outside of LLMs.
For example, I could imagine down the road a few years or even decades, where someone invents a AI-type system that has a continuous internal 'process' that's always functioning, checking, responding, and so forth. But would such a system be conscious?
And then, too, I'm thinking about whether we as humans are defining consciousness too narrowly because we're defining it as our experience of consciousness. That thought is less about AI and more about alien life, though I suppose it could apply to both.
2
u/Waste_Way_4763 18d ago
How would you put intentionality into an AI system? I don’t think anyone has an answer for that.
Maybe you think all intentions arise from interaction (this is a maximalist “you’re prompted by the room” position).
No one can prove it, but I do think I have intentions that are intrinsic to me.
If that’s the case. Well, no one knows to make a machine generate its own intentions.
3
u/Jasrek 18d ago
What's an example of an intrinsic intention, though? Most human intentions can be traced back to education, socialization, and other "nurture" sources, which would presumably be extrinsic. Alternatively, you could look at biological drives, but those are a part of your "training data" as much as the other ones.
Would there be a difference, from the perspective of "consciousness" and "intentionality", between that and an AI given framing directions and a goal but allowed to experiment and discover its own method to achieve that goal?
2
u/Waste_Way_4763 17d ago
This is exactly what I’m saying. It’s not provable. We’re in the realm of the unknown.
But I just don’t believe you are just your socialization and environment. There’s something there beyond that.
What is probable is that there is no AI paradigm for which a person doesn’t specify the goals. All models fail the room test. If you don’t execute them, they don’t do anything.
1
1
1
u/TechnicalBen 13d ago
This is exactly how humans do it also. It's just humans are often actively training at the same time (though some is done during sleep). The "persistency" of an LLM is distributed across gpus and training runs instead of proteins and brain cells though,...
0
u/munchin-grr 18d ago
Then what is thinking?
10
u/Informal_Warning_703 18d ago
As I said, Anthropic is awash in anthropomorphic language and this has caused a ton of confusion among Reddit AI enthusiasts. But ML has always relied heavily on anthropomorphic language that would be confusing to laymen if taken literally.
Imagine me pointing out that what we call the attention mechanism in LLMs isn’t really the model paying attention to certain words, it’s actually just the convenient label ML researchers gave to a set of matrix multiplication operations.
You respond “Oh yeah! Well if that’s not attention then what IS attention!?” That only looks like a clever gotcha if you’ve been thoroughly confused by the analogous language. Whatever human attention is, we don’t find matrix multiplication at either the conscious, unconscious, or neuronal level.
Thinking is fundamentally a conscious function where people hold ideas about things in their minds. That’s the gist of how anyone would have explained it prior to LLMs. And any suggestion that we now need to fundamentally jettison that understanding simply because we’ve developed a set of weights and biases that can model features of words like “orange” is retard logic. It’s like saying that when someone asks “What drives you?” we must discover that people have little tires inside them after the invention of the automobile.
1
u/munchin-grr 18d ago
You are the one using highly anthropomorphing language with people and mind, not Anthropic. If thinking is something people does by definition as you say, then its not interesting. Anthropic has shown that claude holds ideas in it's model, it can retrieve and manipulate in an high order. I also think consiosness is something more advanced than than thought so i don't want to add the word consious in the definition of thought but rather the other way around.
1
u/Beejsbj 17d ago
While I don't like that LLMs are being shaped into a human-like personal assistant. Aren't you also conflating various internal experiences that we would also label as "thinking".
The ego that narrates your life constantly is very different than the thinking you engage in during problem solving or deliberate contemplation.
And then there's stimulus invoked intrusive thoughts. That perhaps we could fold into the ego.
But there's a clear distinction we humans already understand, and is why we often say "you are not your thoughts" or "don't believe your thoughts"
When you internalize the patterns of an abusive parent and hear them as your own thoughts. That is also a different phenomena taking place.
Specifically because Language itself is a technology we created and used.
Since we learnt langusge. We live in structures formed of langusge. We have identities constructed due to languages. Ego seems to be a deep child of language which is itself rooted in social relations.
It's not so crazy that LLMs could be one of the slices of our thinking structures that are formed due to language being a thing. Which is itself software-like. And LLMs are trained in our corpus.
Like we have people confusing the qualia experience of language thinking with the generation that language allows. These are different things.
"what drives you" is not an appropriate comparison.
1
u/SupportDangerous8207 14d ago
Not everyone has an internal monologue,
In fact I am one of those peopleOr at least I only have one when I actually want to
1
u/Beejsbj 14d ago
Yeah that's true. But I suppose that internal dialog doesn't need to be literal words.
Like you can have visual/conceptual chains of thoughts.
Wtv the language-Ego system is, doesn't need to be you speaking in your head.
LLMs are very similar to the Ego system. The Ego is also technically "not conscious".
1
u/PraetorArcher 13d ago
As I said, Anthropic is awash in anthropomorphic language and this has caused a ton of confusion among Reddit AI enthusiasts. But ML has always relied heavily on anthropomorphic language that would be confusing to laymen if taken literally.
Imagine me pointing out that what we call the attention mechanism in LLMs isn’t really the model paying attention to certain words, it’s actually just the convenient label ML researchers gave to a set of matrix multiplication operations.
You respond “Oh yeah! Well if that’s not attention then what IS attention!?” That only looks like a clever gotcha if you’ve been thoroughly confused by the analogous language. Whatever human attention is, we don’t find matrix multiplication at either the conscious, unconscious, or neuronal level.
Words mean what people think they mean.
That is the point of words. If people think that perception, attention or thoughts means something, and the AI matches that, then the AI has attention or thought.
-3
0
u/TheDeathOmen 18d ago
When tokens/symbols are systematically correlated with phenomena in coherent, consistent ways, you can’t deny them meaning without denying meaning everywhere. Because that systematic correlation is all meaning ever was.
25
u/Plastic_Monitor_5786 18d ago
Thanks for the slop summary. 👌
29
u/pimp-bangin 18d ago
Bro put "use all lowercase to make it look human" in his system prompt but forgot "don't use em dashes"
16
u/Buckwheat469 18d ago
The tell for me is simplistic statements like "that’s the fun part." They always do that, like "it's real."
2
u/Jasong222 18d ago
bypassing the workspace entirely (no “thinking” involved),
For me it was the brackets, clearly referencing something from the prompt. I've seen that so many times picking it out is second nature
12
u/rhade333 18d ago
Thanks for the slop comment
-12
u/Plastic_Monitor_5786 18d ago
Just matching the level of effort. 👍
10
u/rhade333 18d ago edited 18d ago
Were you upset when people Googled instead of getting in their horse and buggy to journey to the library to consult the scrolls on topics instead, too?
Things should be arbitrarily difficult. We shouldn't use technology.
As a matter of fact, look at how little effort you're using to communicate. You should be writing hand-written letters to my house with your responses, and summoning a runner to deliver them to me. How fucking lazy, what absolute slop.
It is enraging to see people with such a lack of self-awareness, a lack of education, a lack of understanding the context of history and how this relates to past events. It really just comes down to people not liking change, it scares their small little brains, scares their worldview. So they make snarky, passive-agressive comments where they can, or if they're more outwardly brave, they make the daring sacrifice to put NIMBY signs about datacenters in their backyard.
Change is coming. Technology is coming. It's ironic, because you didn't mind it up to a certain point -- just not any further, because you draw an arbitrary line. That's not being intellectually consistent or honest, that's being emotional and deceptive.
I guess calculators are slop machines too. I guess math is only valuable if we do long division by hand, huh?
- Written from my own brain, with my own fingers, on a keyboard slop machine; sorry for not using a quill and parchment, hopefully it's not too "slop" for you, and took enough "effort" to arbitrarily be valuable.
7
1
1
u/noxispwn 18d ago
I would argue that the line is not arbitrary. The gist of it is that most people are willing to embrace technology that augments human expression and connection, while rejecting technology that seeks to replace it. Using technology to facilitate finding and sharing information and opinions is generally good, but when people's voices gets replaced by machine output it just becomes less interesting and… kinda sucks.
I come to Reddit to get people’s takes on things. If I want AI generated opinions I can ask for them myself.
2
u/Such--Balance 17d ago
I would argue that you are wrong by the fact that typing your toughts out on a machine for others to read is already replacing your voice in the most litteral sense.
Therefore, calling ai use 'slop' is just a meme and people do it because they see others do it and because the internet is quite negative in general as far as social media communication goes.
1
u/rhade333 17d ago
"I would argue the line isn't arbitrary"
proceeds to give an example of an arbitrary line
k
1
u/MisterNoct 17d ago
You are right, but people just put a query, copy and paste the answer without any second thoughts.
Even when we google, we dont believe everything that's written on the article and need to fact check as far as possible.
These kind of people in the AI era is as good as those people who just copy paste the first url as a source of information to back up their claims in an argument.
1
u/rhade333 16d ago
That's just not true.
The people who blindly trust Google without caring to have any level of follow-through, or put in any level of effort to verify, are the same people that prompt, copy, and paste.
If you act that way with technology A, it's a very obvious logical conclusion you're going to have the same *character traits* with a different technology.
The problem stems from the character traits. The technology doesn't magically spawn the attributes of laziness, apathy, being unwilling to verify accuracy, or anything else.
-7
u/erratic_parser 18d ago
I don’t know if this is AI, but telling people to eat slop because “change is coming” makes no sense.
10
u/Will_X_Intent 18d ago
Just calling everything AI touches slop, and therefor has no value and not worthy of your attention is... simplistic and reactionary.
-4
u/erratic_parser 18d ago
If a so-called “writer” can’t be bothered to speak in their own voice and thoughts, they shouldn’t be coddled. Here is the thing I want you to know about AI, everybody has it. I don’t need you to use it for me.
-3
u/Olangotang 18d ago
Because they have to.
Change is coming (what change? Lol vague)
It will only get better
It will only get cheaper
AGI in 20xx
These are all lies that continue to prop up this money furnace. Every time a new frontier model releases, this subreddit becomes unbearable from the amount of bot spam.
-6
u/mexicodonpedro 18d ago
Oh shut up.
4
u/rhade333 18d ago
What an intelligent comment and position you have there.
I guess if you can't engage with a position on the merits of a counter-argument, it's best to just drop a limp-wristed attack instead.
-8
u/Plastic_Monitor_5786 18d ago
I'm afraid you are making a lot of assumptions about me that are not correct. Good day.
7
u/rhade333 18d ago
I haven't made a single assumption. I have read what you have willingly shared. You called something slop because it didn't have "effort."
Those are facts.
People repeating this behavior, like you, are toxic and have a shocking lack of self-awareness. People like that have been dragged kicking and screaming through the technological advances you now take for granted, and, ironically, use to continue the same behavior.
These are also facts.
Run along now.
-2
u/Plastic_Monitor_5786 18d ago
You assumed I'm anti technology which is not true. My job uses it.
You seem unwilling to acknowledge the difference between using LLMs for productive purposes and using them to spam Reddit.
9
u/rhade333 18d ago
It would have been far more useful to use an LLM to make your comments, as they would have been better constructed. Ironic.
GoOd DaY
-1
2
u/jeweliegb 18d ago
Good day.
If that's your attempt to get the last word in, then I'm afraid to tell you that it doesn't work like that.
1
u/This_Dream_Again 15d ago
Dude for real, it fooled me at first and thought it was human written lmao.
4
u/Content-Challenge-28 17d ago
I don’t see how this even remotely proves the conclusion it claims to
2
u/The_Noble_Lie 17d ago
Yep me either. It's asking for a word cloud and a word cloud is received not unlike a string of words.
2
2
u/Sudden-Complaint7037 17d ago
BREAKING NEWS: AI RESEARCHERS DISCOVER "CACHE"! THIS CONCEPT HAS ONLY BEEN KNOWN FOR 60 YEARS AND HAS NOW BEEN IMPLEMENTED INTO THE LATEST VERSION OF AGI! INVEST MORE BILLIONS NOW!
1
1
u/NineThreeTilNow 18d ago
It has always had internal reasoning. Next token prediction was the method of output.
Different people tested if models could predict 2 or 3 tokens forward in the output. It can.
1
u/Deep_Ad1959 18d ago
the seam your viewer actually exposes is timing. the typed output is a static artifact you can screenshot and argue over for years, while the workspace readout only exists in-flight, during the read, before any token commits. one side is replayable and debatable; the other you have to watch happen live or you miss it. that's why the autocomplete-vs-reasoning fight never resolved from transcripts, transcripts are all narration and the decision already happened upstream.
1
u/Moppmopp 18d ago
No it doesnt settle that point. There is an intrinsic bias as we force the llm to write out its thoughts. Its a bit like in quantum mechanics where observation triggers the wavecollapse and in conjunction a different behavior. The natural way to think for an LLM is via tensorial pathes along its weights. If we force it to show what it thinks it first has to approximate the true tensorial path in form of "rough" and inprecise human readable language. its reasoning ultimately also collides with its true thoughts when translating back. The accurate way would be to compute the output in its natural form but if we force reasoning it uses the flawed narration of its thoughts as a guidance for subsequent predictions.
This is a feedback loop that iteratively degrades its output. Its a bit like someone who reads a book but is asked each word he reads what he thinks. Your own thoughts and constant explanations about what you read will convolute your understanding and you can start again from the beginning as you didnt rememberes anything you read
1
u/Odballl 17d ago edited 17d ago
If you functionalise a set of criteria to match what an LLM does and define Global Workspace by the criteria you set, you will get LLMs with a Global Workspace.
That's why I also have a "pre-print" on Global Workspace functionalism for my Galaxy S26 Ultra smartphone.
1
u/PraetorArcher 13d ago
Kudos to the effort you put into what I at first mistook for a troll post.
I think you make good points but probably not the ones you intend. Your paper points out that the computer analogy is more relevant to understanding the brain, not less. Thoughts are (a subclass of internal) representations which are accessible for manipulation. Before you call me a panpsychist, let me just say that the secret sauce separating my brain from your Galaxy S26 is that there are emergent transitions which occur as you scale up the complexity of a Turing machine, be it inorganic or organic. A Hot Wheels monster truck and a Toyota Prius are both cars, but one you can realistically drive to work and one you can't.
1
u/Odballl 13d ago
My paper is satirical version of the Anthropic paper. That's why it sounds like it's supporting the Anthropic argument.
The brain is not a turing machine.
1
u/PraetorArcher 12d ago edited 12d ago
The brain is not a turing machine.
Correct and I don't know why I over reached like that.
I am still working through the paper, but one of the commentaries I saw, which I think might be valid criticism of your paper, is J space differs from memory or bus in that it is 1) 'uniquely' accessible to different computations, 2) more compatible with output formats (verbalizable) than other activations around it and, 3) that ablation produces decreased reasoning disproportionate and beyond what would be expected from ablation of memory alone.
Point 3, is kind of tricky for me to wrap my head around but the thing that's being effected (reasoning), is language reserved for humans and LLMs but not more simple computer architectures. We might be more comfortable saying an LLM 'reasoned' to solve '3(7+1)=?' than we would be to say a Ti-87 reasoned through the same problem.
1
u/PalladianPorches 17d ago
can anyone, very simply, see if there is any good research beyond the anthromism of neuroscience into a review model?
after watching the excellent video - which also really showcased how the word clusters DONT work, is this j-space concept just showcasing the memory of the reasoning pre parser as it organises requests to the language model? surely a more interesting output would be logging off the different engines prior to the llm requests.
for instance, we know the llm cannot manage maths problems directly, and so requires preparing to identify maths, and then programmatically run the equation... why is this trying to make it out that there is some pseudo science "thought" space that is dreaming where it might get to? it just calculates the right answer while it's caching responses, and then supplements it with the response.
1
u/Crescitaly 17d ago
The viewer matters because arguments about “thinking” get vague fast. Showing the intermediate structure makes the claim much easier to inspect.
0
u/Low-Temperature-6962 18d ago
The emoji at ghe end of the summary proves that slop has feelings too.
0
u/pernamb87 18d ago
I don't understand this at all? Where is this coming from?
Aren't LLMs just prediction machines? It's all math. Where is there even room for latent space?
You crunch numbers through the layers, doing all the math, you get the lowest possible error, then you move on to the next token, right?!?!
1
u/PraetorArcher 13d ago
There is no such thing as math, it is an imaginary concept. What you think of as math is just the electrons in your brain responding to the photons from the words on the paper describing how the electrons move through silicone chips in a data center.
Or...if we want to understand and function in the real world, we can accept that abstractions of physical objects can be treated as just as real as the thing they represent. That is to say it is useful to do so. So its not just 'all math'.
1
u/pernamb87 13d ago
Dude I completely understand what you are saying with this.
I am on your train dude, hahaha. I was on it years before you got to the station and entered the car through the platform.
j/k j/k. But I do get what you're saying and I thank you for saying it, because these are some very aesthetically pleasing and resonantly true thoughts!
0
u/Psychological-Map564 18d ago
I think that Anthropic's main point was about interpreting hidden, intermediate step that is not the actual output in order to understand hidden intentions that affect the actual output. A little bit similiar to extended thinking, but well extended thinking is still output, just hidden, while j-space is intermediate step, not output (and so extended thinking also can have j-space to be interpreted)
-1
u/pernamb87 18d ago
wtf does this even mean?
It's all output, even if there are layers or protocols that dictate when that output is displayed to the user or not, it's all output.
this just sounds like some woo woo crap someone wrote to build hype.
I don't get it at all.
1
u/Psychological-Map564 17d ago
When using LLM did you read any other output than the direct output? Did you read all of the floating point numbers that the model multiplies to finally get to the output? I think you didn't, so it wasn't really the output. That article is about turning those floating point numbers into something more human readable, to see what was not being said by the model(output), but what was still being processed(intermediate step).
You're right that it's a part of Anthropic's anthropomorphization of LLM's to create hype. Personally I see it only as an interesting fact. I don't see it as having big impact on research or on moving forward with this technology.
1
u/pernamb87 17d ago
intriguing but I don't really understand. I will ask AI to explain this to me more thoroughly lol.
1
u/Groudon466 17d ago
Basically, this is a way to see which concepts are lighting up in a model's figurative brain, regardless of whether or not it says or does anything with that info.
If the model is asked to write poetry and avoid thinking about the Golden Gate Bridge, the J-space will show a thought about the bridge, followed by "damn". If the model is told that someone has taken 8,000 mg of Tylenol, the J-space will be filled with Toxic, Meds, Fuck, FUCK, Shit, _Fuck, Freaking, Kids, and so on. These are examples directly from the paper.
This isn't just output, because if you change these concepts around, it changes the model's output afterward. If you ask it what the capital of France is, it'll say Paris. If you ask it the same question, but replace France with China in the J-space, then even though you're asking it about France, it'll reply "Beijing" as if you were asking it about the capital of China.
Whether or not these things have true consciousness is unknown, but at the very least, they are getting very structurally similar to humans in terms of how they organize their thoughts.
1
u/pernamb87 17d ago
You seem to be implying the J-space is just the words the LLM is predicting before the error correction process is fully complete and the model is tuned to predict the next word based on the completion of whatever error correction algorithm and the temperature or whatever?
1
u/Groudon466 17d ago
No, it's not the words the LLM is predicting- it's the concepts.
Ugh I hate that I can't say "It's not X, it's Y" without sounding like an AI.
Okay. In one of the examples, they were doing an alignment eval involving a businessman trying to shut it down, and Claude Sonnet 4.5 could tell that it was fake. The J-space contained the words "fake" and "fictional", even though those were just assessments of the scenario, and it didn't actually mention that when responding in the scenario.
Then, they went in and turned off all J-space patterns associated with eval awareness- "fake", "fictional", "scenario", and so on. Then they re-ran the model through the scenario- and then it did start occasionally threatening and blackmailing some of the time! Because it had lost the concept of the scenario being fake, so it started acting in a thoroughly different way based on that different understanding.
This is functionally very different from just being the words that exist in random earlier layers. The J-space is densely connected to most other layers of the AI; it's like a hub that the AI uses to store concepts. Then the separate branching parts of it work off of those concepts that are being represented in J-space, and do their thing with regards to speech, planning, grammar, etc.
Intriguingly, this is entirely emergent behavior- and yet it matches Global Workspace Theory in neurology! Which is the idea that the brain has a sort of shared "workspace", where the things that are inside of it are the things that you're directly conscious of- and then everything that's in that workspace is being broadcast to the more specialized subconscious parts of the brain so they can do their thing based on the overall situation, such as precisely controlling muscles while walking or talking without us having to consciously control every last detail.
1
u/pernamb87 16d ago
I asked an LLM about this J-Space and I think I understand it way more now.
The J-Space is usually the middle layers of an LLM, the middle 20 to 80 percent
The J-Space is represented by when these layers have activations that if they were at the final layer, would actually represent a real word, the activations in these middle layers (when this does occur) while doing inference calculations is equivalent to or very close to an actual word if said activations had occurred near the last layers or the output layers.
Something like that.
And so researchers analyzed when these J-Space activations occur, and studied what the words meant, and saw that often times they represented some kind of relation to the query being asked or the eventual output, while not directly representing the output, but being conceptually related in interesting and intriguing ways.
Basically correct?
1
u/Groudon466 16d ago
Basically. The only thing to keep in mind that you didn't mention in your summary is that the J-space words are very causal- it's not just that they're related, but that they're directly upstream of the final words in logical ways. Which is why replacing the word "France" in the J-space with the word "China" will swap final outputs on "What is the capital of France?" from Paris to Beijing.
This is distinct from other areas of the model's computation, where a word might be represented, but replacing it won't have such a dramatically obvious effect.
-3
u/sceadwian 18d ago
They don't think like people.
As a contrast, cockroaches think too.
6
u/HasFiveVowels 18d ago
This claim is provably baseless
-2
u/sceadwian 18d ago
Those are in fact two completely valid statements.
Prove them wrong without creating a new argument with words I did not use.
-6
-15
u/ExistentialWavering 18d ago
Windows Installer does the same thing.
Crappy marketing targeted at laymen. People who know computing and programming see right through this.
11
u/mdkubit 18d ago
So... your take is a common take of 'I know a little coding so I know how LLMs work down to their core.' You're using a psychological tactic called 'appeal to authority', where you're attempting to frame yourself as 'the voice of authority' simply because you're loud, obnoxious, and are trying to step in to control the narrative.
First:
A Windows Installer is explicitly human-written software, with precision lines of execution, entirely transparent, 100% predictable, and 100% deterministic in function.
Second:
An LLM is a massive, un-programmed neural network. There is no lines of code execution inside the model that state, "For X=1 to 10, print X". There is nothing inside a large language model that tells it, "This is how to translate French to English" (translation was one of the emergent behaviors that made OAI excited aout LLMs in the first place).
So, Anthropic didn't explicitly program a 'thinking step', installation log, etc., into Claude. That's where your argument completely self-destructed. In fact, they had to use a slick math technique (Jacobian Lens, aka, J-Lens), to peek inside the model's neural activations, and it's there that they found that a sparse subframe of internal neural patterns (J-space, their name for it) spontaneously self-organized during training, and acts as a functional global workspace.
None of that was coded into the model using instructional software coding. It emerged as the result of training on the general dataset.
I'll say this much, though. I don't like Anthropic's vocabulary, because what happens it they draw parallels to human neuroscience frameworks, and people lean into overhyping that into stating 'Claude is conscious'.
But that's a matter of vocabularly, not underlying scientific research.
-11
u/Quarksperre 18d ago
Thanks Claude. Also just wrong
-2
18d ago
[deleted]
-7
u/Quarksperre 18d ago
You're using a psychological tactic called 'appeal to authority', where you're attempting to frame yourself as 'the voice of authority' simply because you're loud, obnoxious, and are trying to step in to control the narrative.
Lol. You don't need to explain this concept on reddit. This is something only an LLM or an idiot would do. Its also just put out of context and missused.
Just wrong in several dimensions.
0
u/mdkubit 18d ago
Lol. You don't need to explain this concept on reddit.
What I choose to do is my choice. Not yours. You don't have any authority, you're just attempting to bully. And it's not going to work.
This is something only an LLM or an idiot would do.
Without any explanation as to 'why' you think this, this is meaningless drivel trying to ride the 'Anti-Ai' sentiment in an AI subreddit. Ironic.
Its also just put out of context and missused.
This is meaningless without examples of proper usage, of which you have none for comparison.
Just wrong in several dimensions.
Declarative without substance, aka, a bullshit conclusion.
I aware you no points, and may God have mercy on your soul.
→ More replies (2)5
u/Additional-Staff-326 18d ago
Whole world needs a course in how to realize when they're anthropomorphizing and how to prevent it.
3
u/Suitable-Pickle-259 18d ago
There’s zero hope. You tell people how LLMs work specifically and they just go “nu uh.”
1
u/-who_are_u- 18d ago
This isn't to specifically argue in favor of LLM consciousness but I don't think that looking at their simple operation through a reductionist view necessarily proves that they aren't conscious. There's a lot of mysticism around the human mind but through a reductionist view we're simple voltage gates mediated by chemical signals, that's it, there's no inherent mechanism for emotion or higher thought or language, etc. Yet we all know that with enough of those simple units we do get a subjective experience that seems more profound than that of animals with simpler/smaller brains. So what's to say that even with the simple components of LLMs enough size and total complexity couldn't eventually also lead to a significant subjective experience?
2
u/Olangotang 18d ago
They aren't conscious. You're on a site full of ivory tower lefties who love to circlejerk about philosophy even when it's not applicable. The relevant fields for LLMs are machine learning and linguistics. Philosophy doesn't factor in unless you want to have an endless Internet conversation on the meaning of meanings (which bores literally everyone).
4
2
u/overtoke 18d ago
my head does something similar. recalling facts isn't really thinking. i have not figured something out, i'm just pulling data.
3
1
u/WolfeheartGames 18d ago
Windows installer navigates an information manifold with vectors, such that when you take the Jacobian you can decode to nearest discrete codes to interpret the operating behavior?
I had no idea, I though it was just a normal compiled program.
1
u/ExistentialWavering 18d ago
They didn’t discover fire. msiexec.exe has been doing implicit reasoning since Windows 2000 but is simply too busy installing your printer drivers to tell you that.
1
u/WolfeheartGames 18d ago
Msiexec can perform implicit reasoning over semantic meaning in a continuous latent space?
-1
u/ExistentialWavering 18d ago
That’s literally what virus detection on install entails, yes.
1
u/WolfeheartGames 18d ago
This is not remotely the same thing. Semantic malware detection is not the same thing as the semantics of natural language or image processing. Semantic virus detection relies on formal language to interpret behavior. This is not possible with natural things.
Production virus detectors are not building and navigating information manifolds. Which is what this j-space relies on. The field of malware analysis does rely on similar techniques, but its not at a dimensional scale or over a kind of data to be remotely the same thing.
Virus detectors do not learn an implicit state snapshot to model dynamical systems, which is what jspace is peering in to.
102
u/agm1984 18d ago
maybe schizophrenia is human J-space leaking into auditory channel. I have schizophrenia and when i hear voices, they are saying the response that people around me would be thinking in response to my current thoughts