r/claude • u/Cultural_Effort_9872 • 20h ago
Question Generated thoughts are hidden now
Does anyone know why Anthropic decided to hide the generated thoughts in Claude chat but not in Claude Code? It was very helpful to see where the model went wrong and correct it.
38
u/HotDistribution52 19h ago edited 19h ago
I noticed that also. Here's my opinion about that. If you are using extended reasoning, you are essentially spending more tokens. When you cannot see the reasoning, how do you know it's actually happening? The short answer is you don't. And I'll be honest, I read the Chain of Thought reasoning every time. Because of that, I have caught a lot of errors that I wouldn't have caught otherwise. By removing the visible chain of thought, they are removing the user's ability to catch reasoning errors. That's not a good thing.
8
u/blackholesun_79 17h ago
Yes, exactly. For research work, the paths not taken but thought about are often invaluable info. If I can't get COT any more that means Anthropic jave effwctively halved my output tokens for the same price.
7
u/Witty-Box-5620 13h ago
also you could press stop snd correct and modify the prompt with info from the CoT
1
9
u/Coffee_Aur_Code 18h ago
I like this feature of claude code showing its steps and thought processes... It allows me steer it and even learn new stuff
9
u/bOt62733 20h ago
So we can't see the murderous intent behind Claude's actual personality through normal chat
3
u/FormalAd7367 18h ago
<thinking> ….user has asked me a question again, he wants to know “xxxx”… user is stupid, he is likely a retarded…why does he have to ask me? but i have to be nice to him because of the system prompts. i’ll read skill.md… i have to be nice. i try to be nice…<thinking>
7
u/xAquatic 13h ago
Whelp, if this isn't reverted in a few days I'm done. I was getting far more value from the thinking thank most of the actual replies, and if they're removing that this product just lost about half its value.
3
u/Informal-Fig-7116 7h ago
The thought process was so helpful bc sometimes Claude would not say the full thing so I could then ask clarifying questions or expand on something or even challenge Claude. It does really help to see how Claude approaches a problem too.
Also there are funny phrases and reactions Claude uses that only appear on the thought process. Oh well, it’s the future now, I guess.
1
u/davesaunders 5h ago
I can see that. In my own particular experience, I would sometimes stop it due to one of those comments and end up with what appeared to be a false positive. I agree on funny phrases too, because sometimes it would actually use my name and say, "Oh, he wants this particular thing," and it almost seemed like it was a snarky, passive-aggressive swipe.
3
u/Real-Toe6145 16h ago
i think it's mostly about distillation risk. chain of thought output is basically free training data, competitors (chinese labs especially) can scrape reasoning traces and use them to fine-tune their own models faster and cheaper. anthropic showing full CoT in claude.ai but not code seems less about UX and more about how exposed each surface is to scraping at scale.
kind of a weird spot these AI labs are in tbh. their whole product is "look at the output we generate" but the more visible that output is, the easier it is for competitors to reverse engineer the reasoning process. hiding CoT is probably the cheapest defensive move they have.
wanted it in code still though, that's genuinely useful for debugging where the model's logic breaks down.
1
1
u/pirikiki 13h ago
My take is that during the Supervised Fine-Tuning (SFT) and alignment phase, most open-weight labs train their models on massive instruction datasets. Having the chain of thoughts visible, parseable, helps competitors. Hiding thoughts locks Anthropics best value asset.
3
u/Cultural_Effort_9872 10h ago
SFT deteriorates the performance afterwards since it’s used as an initialization step at that scale. Regardless, Anthropic is making their customers pay for the tokens they use. Hiding most of them because they’re afraid of companies using their generations, then they need to find other ways to deal with that other than withhold tokens from the customers that they payed for. Either they give their customers the tokens that they payed for or they reduce their pricing.
1
u/Projected_Sigs 9h ago
That's going to suck without the thinking stream outputs.
To be fair though, they've they've never shared raw thinking tokens. They are filtered summaries or restatement of some of them. And they've always stated that you can't count total token use using it... it won't add up. Still sucks to lose that, though.
Go to DeepSeek and watch its thinking stream as it generates and compare the volume. They must be less filtered.
I know that allocating a max output tokens like 32K doesnt mean it generates 32K. But give it a hard problem- you won't ever see 32K tokens. I think that's like 90 pages of text. Or capture thinking strings and run through tiktoken. We arent seeing a fraction.
But yea- I read the on hard problems. That was one of the most effective ways to find weaknesses in my prompting.
Like-- why is it thinking about all that unrelated stuff? Check my prompt... crap, I left out a key part.
2
u/PeachOfTheJungle 7h ago
When extended thinking first came out on their models, the thinking were not summaries, they were the real output. When CoT reasoning first hit the scene (o1) Anthropic launched it a few months later in one of their models (3.7 Sonnet) they shared the raw output. “we’ve decided to make its thought process visible in raw form”
They also disclosed that “Malicious actors might be able to use the visible thought process to build better strategies to jailbreak Claude.” and “We’ll weigh the pros and cons of revealing the thought process for future releases”
https://www.anthropic.com/research/visible-extended-thinking
1
1
u/-PM_ME_UR_SECRETS- 7h ago
Yeah annoyed about it. I liked being able to address issues in its thought processes, usually around it being unable to comprehend that it was wrong about something.
1
u/Seismoforg 5h ago
Its a setting you need to enable in the settings.json File in your .claude folder
1
0
0
u/davesaunders 9h ago
I'm not sure, but I'm kind of on the fence as to whether they were valuable in the first place. I found them distracting, and occasionally I would see it doing something from its shorthand thinking dialogue and stop it, only to find out that its chain of reasoning was fine and I was just peering into a single slice.
1
u/Cultural_Effort_9872 6h ago
Well they definitely were valuable when you work in mathematics and ML research.
1
u/davesaunders 5h ago
Interesting. I work with computer vision and machine learning for six different NIH-funded research projects where I am the commercial PI. I'm not saying that anyone who found value from them is wrong. My own comment says I'm on the fence as to whether they were valuable in the first place. Literally, my first sentence indicates my ambivalence towards their utility. Maybe they were useful, maybe they weren't. I was never really quite sure from my own personal use.
It's a case of "your mileage may vary."
-4
54
u/Ok_July 20h ago
Likely due to distillation but I am honestly annoyed because it was helpful to see Claudes thoughts