Proprietary reasoning can be recovered from its encrypted traces. Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models. We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, and recover the stronger model’s hidden reasoning in plaintext, without ever attacking the stronger model directly or triggering its anti-distillation safeguards.
It's rather common for posters to make a very small summary in a comment. It can help fight the floods of comments working off the title alone (though it's not particularly needed here for that purpose, imo)
This is a non-transparent aspect of submitting a link to HN that is quite misleading.
You think that you are adding a description for your post when in fact you're simply submitting a regular comment not promoted or distinguished in any way.
It's even worse considering posts without URLs would take the same text from the same input box on the HN submission form and append it under the post title, locking it to the top of the page - so if you've browsed linkless posts you think maybe the submission text is gonna end up just like that, but it doesn't. (Also I think you'll even see URLs like an archive link end up appended just below the submission title for URL posts. Which I guess is a special feature.[1])
Any reason I should not send HN an email requesting clarification of this the submission page?
“If there is no url, text will appear at the top of the thread.”
OK, now I finally understand what that means in practice, but it doesn’t imply an entirely separate undistinguished comment will be simultaneously submitted on my behalf.
There must be a reason HN does not colorize the OP username or something. But there totally could be some indicator of “post submission text” without too much in the way of negative consequences… (the fact this has never been added tells me I’m being naïve)
I'm honestly rather curious if this was intentionally allowed, it's the sort of validation that's easy to miss (particularly if you're wading into the vibe waters). Seems like something that'd be absolutely riddled with possibilities for shenanigans.
Fair (I haven't been using the encrypted-reasoning systems, though this is common in open ones - I'm kinda surprised it's an option in encrypted ones too), though what they're doing here is cross-user replays in addition to cross-model.
There seems to be an obvious choice to make here, should you give the users to decrypt and use the COT that they did not generate themselves?
This is only required if you want users to be able to share things with everyone and you are going for the simplest implementation.
If not you could try to keep a record of keys associated with a user, then when a new request comes in look through to see if the user has a valid key to decrypt the COT.
For explicit shares, just add the key used in that one conversation to the users valid keys. For global shares use the global keys. But that's adding more complexity to the system.
It’s about being able to change models mid-task. For example, I want to be able to plan using Fable but implement the plan using Sonnet, and that won’t work if this is implemented.
Star Trek is a post-scarcity society, those problems don't exist there unless you're in the middle of a crisis and on emergency power.
LLMs briefly seemed like this too, after subscriptions made the SOTA models too cheap to meter, but before they walked back on that and introduced quotas...
For plan it's relatively easy, just make the plan the artifact. The point is to ingest knowledge with one model and use it in another, and that is not necessarily easily expressible in natural language.
I really don't understand why server-side storage of the trace isn't a viable approach here, with only a unique key flowing to the client and back. Does it have something to do with how backend load-balancing works?
Yes, this storage would be growing exponentially making the disk space and latency problems harder (add the disaster recovery/backups). I think the choice of using client side is not too bad if you ensure that its secured properly. Also the company can excuse itself from the liability of storing sensitive data on its servers, thats a big deal in itself to be compliant for enterprise audits
1. The down side is that it cannot be used across the clients even for the same user
2. Using the same encryption key was a bad choice here, a per user key would have solved this issue for sure.
Having thought about this a little more, it's clear that server-side storage is not compatible with Zero Data Retention (ZDR). However, in non-ZDR settings, it seems likely that the providers are capturing all that data anyway?
> a per user key would have solved this issue for sure
It would have helped with PII leakage, but not with plain-text trace extraction attacks, right?
Per user encryption key ties it with the user session (assuming you do authentication properly), no one else can access it. User being able to see the information is not really an attack vector in this case.
The compliance rules at times are outdated and people skirt around them by following the worded rule instead of the intent.
Makes no difference. There is a policy as to whether to allow use of a reasoning trace in a given context. Whether that trace originates from authenticated ciphertext or a backend database is basically irrelevant.
I believe its a result of chasing the speed and happy path. The intent was to make the data available for reuse on the backend side for the same user session, it kind of makes sense. Making it available across the models is most likely by design.
The flaw is that the data is not strictly tied to user session, making the session data hijacking a lot easier.
Author of the blog post here: just for the record, I did try replaying against other models. I replayed from GPT 5.5 to 5.5-mini. It "worked" (replay not rejected) but mini didn't hand me the plaintext of the reasoning. I'm not a jailbreaking expert, so I didn't really "push" on it very hard. I also didn't pursue Opus/Haiku for reasons I don't really recall. These folks took it all the way. I'm bummed it didn't work for me, but having these folks prove the channel out is a very good consolation prize.
Can someone tell us how they were able to decrypt the encrypted payload? The article says they inserted the cyphertext into a session with a different model. Ok, but how does that allow you to decrypt it?
The provider decrypts it and puts the decrypted reasoning into the model's context window. They prompt the model to repeat back the reasoning. So then the model echoes it back in plain text.
Hmm, ok. So the attack doesn't involve decrypting the payload, only getting the server to do so. Since a model will do that if you just ask, what's so special about the attack?
The large models whose thinking traces are useful are safeguarded against this reasoning replaying. the small models are just designed for speed and efficiency, so these safeguards are a lot meaker, making the attack possible
Really cool work, you get the actual traces. Looks like the vendors can all reliably fix this one though.
A harder to defend against approach here where they work backwards from the results and ask the model to generate a plausible trace:
How to Steal Reasoning Without Reasoning Traces
https://arxiv.org/pdf/2603.07267
Trace Inversion is fascinating, but it’s more of an independent reconstruction that will give you some coherent-looking generated CoT; but not necessarily anywhere close or related to the underlying model’s CoT.
I didn't read the paper in details but they claim there is high overlap between the synthetic traces and the ground truth ones (not sure how they confirmed that for blackbox models though, I guess they must have compared to open source models).
They also talk about successful distillation of black box model capabilities with the approach.
This proves that OpenAI models reason in grug speak to save tokens! I wonder if open models are going to start doing that too to save on reasoning tokens.
In the BlackHat presentation on the HuggingFace incident, OpenAI showed some excerpts from the reasoning traces, and they had that grug speak too (skipped articles, etc.). So the OP's method must have indeed found the actual reasoning traces.
> I wonder if open models are going to start doing that too
Yes, some of them do do that. For example Moonshot tried to reward shorter reasoning traces in between Kimi-K2.6 and Kimi-K2.7 Code, and the latter has a mild caveman accent in its reasoning traces that the former lacks.
Qwen3.8-Max also has terse reasoning, but I don't remember this being the case for Qwen3.6 models I ran locally.
Super cool that this works. I'm surprised these companies re-use the same encryption key across models!
I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888
Seriously, what does it take to encrypt per session? There are many ways to make it scalable and efficient so I am wondering if this is left like this to allow interested 3rd parties ahem unobtrusively peek what people are doing with the AI.
(Thanks for the link. That’s an interesting idea!)
Sure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?
The exploit here isn’t a leaked encryption key. It’s pretty likely that they are already using a unique key per conversation. The raw CoT eventually reaches the model, and you can convince the model to share it with you.
Yeah encryption isn't the issue. The only way I see to fix this is if you stop the user from switching models mid-session, or strip out the thoughts when switching models. Either way you're degrading the user experience.
If a different model is using encyrpted blocks of another model, then by definition it is no longer a session scoped bit of information. Since you can give it to any other session and another model, clearly it doesn't even have to be the same user. Therefore, there is only one (set) of universally available key(s) used by all models across all sessions.
267 comments
You think that you are adding a description for your post when in fact you're simply submitting a regular comment not promoted or distinguished in any way.
It's even worse considering posts without URLs would take the same text from the same input box on the HN submission form and append it under the post title, locking it to the top of the page - so if you've browsed linkless posts you think maybe the submission text is gonna end up just like that, but it doesn't. (Also I think you'll even see URLs like an archive link end up appended just below the submission title for URL posts. Which I guess is a special feature.[1])
Any reason I should not send HN an email requesting clarification of this the submission page?
Edit: quoting https://news.ycombinator.com/submit :
OK, now I finally understand what that means in practice, but it doesn’t imply an entirely separate undistinguished comment will be simultaneously submitted on my behalf.[1]modpowers(?) used to directly append links to URL submissions further confuse the matter: https://news.ycombinator.com/item?id=49243880
I feel bad now :(
Feedback emailed to HN!
Ha! I've been wondering if replaying across models would work, ever since https://blog.cryptographyengineering.com/2026/05/29/fooling-...
I'm honestly rather curious if this was intentionally allowed, it's the sort of validation that's easy to miss (particularly if you're wading into the vibe waters). Seems like something that'd be absolutely riddled with possibilities for shenanigans.
Wouldn’t surprise me if the providers just remove that ability and lock the model once the conversation starts.
Sucks.
This is only required if you want users to be able to share things with everyone and you are going for the simplest implementation.
If not you could try to keep a record of keys associated with a user, then when a new request comes in look through to see if the user has a valid key to decrypt the COT.
For explicit shares, just add the key used in that one conversation to the users valid keys. For global shares use the global keys. But that's adding more complexity to the system.
https://memory-alpha.fandom.com/wiki/B-4
LLMs briefly seemed like this too, after subscriptions made the SOTA models too cheap to meter, but before they walked back on that and introduced quotas...
1. The down side is that it cannot be used across the clients even for the same user
2. Using the same encryption key was a bad choice here, a per user key would have solved this issue for sure.
> a per user key would have solved this issue for sure
It would have helped with PII leakage, but not with plain-text trace extraction attacks, right?
The compliance rules at times are outdated and people skirt around them by following the worded rule instead of the intent.
The flaw is that the data is not strictly tied to user session, making the session data hijacking a lot easier.
1. Its a security issue.
2. Publicly available sessions make it much worse
PS Here's a conversation I had with GPT 5.6 about the paper differences. https://chatgpt.com/share/6a7b64b4-ec0c-83ea-a9d2-ab1f1a1dfe...
A harder to defend against approach here where they work backwards from the results and ask the model to generate a plausible trace: How to Steal Reasoning Without Reasoning Traces https://arxiv.org/pdf/2603.07267
They also talk about successful distillation of black box model capabilities with the approach.
Yes, some of them do do that. For example Moonshot tried to reward shorter reasoning traces in between Kimi-K2.6 and Kimi-K2.7 Code, and the latter has a mild caveman accent in its reasoning traces that the former lacks.
Qwen3.8-Max also has terse reasoning, but I don't remember this being the case for Qwen3.6 models I ran locally.
I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888
(Thanks for the link. That’s an interesting idea!)
I assume switching the model in the middle of a conversation is intended behavior (very useful in coding agents, for example)