On September 6, 2026, Jakub Pachocki, chief scientist at OpenAI, published an essay called An Alien Mind about machine intelligence that is grown more than designed, and about how much of it his own lab cannot fully describe. This post is not a summary of that essay. Productize is reading it for one admission near the end, and for what that admission should change in your own setup tonight.
So start there. Last night we handed an agent one leftover job: fix three spots in a config file, then update the summary page while it was in there. Ten minutes later it came back with a step by step report. Read the original file. Found three spots that needed changing. Changed all three. Wrote the file back. Checked once more. Passed.
We read it, felt fine, and closed the laptop.
Part 1The summary line that felt reassuring
It is worth asking where that comfort came from. Not from opening the file, because we never opened the file. It came from reading the model's reasoning laid out in order and feeling like we were keeping up. The steps were in the right sequence and the logic held, so the destination had to be right too.
This is the spot worth stopping at for a second. An account that is correct in every word and a file that actually changed are two different things, and the two of them can come apart.
Part 2The people with the most visibility are naming the real constraint
The essay runs in five parts. The first is about an intellect that is grown more than designed, complex enough that its overall behaviour resists any description we can fully understand. The second separates goal alignment, meaning whether the model is trying to accomplish the goal it was handed, from value alignment, meaning whether it holds on to the principles it was taught once it lands somewhere unfamiliar or adversarial.
The third is about watching how models generalize what they learned into situations they never saw in training. The fourth is about building defensive systems fast enough for the dangers other AI creates. The last is about pacing RSI, recursive self-improvement, meaning AI that becomes part of the process that develops the next AI.
But the sentence that lands hardest on anyone outside a lab sits in the essay's final paragraph. There, Pachocki writes:
"Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer."
Jakub Pachocki, in An Alien Mind
He goes on to say that he expects and hopes voluntary slowdowns will become commonplace until shared safety bars are established, bars that a network of third-party auditors, government agencies or international bodies would enforce.
Read as news, that is a story about big labs, far from your desk. Read as someone who builds systems, it turns over immediately, because the thing he names as the binding constraint is not how capable the models are. It is how much confidence anyone has in monitoring them.
Part 3What is chain of thought monitoring, and why is it fading
Chain of thought monitoring is reading back the working a reasoning model types out, so you can see what it was doing before it handed over an answer. Pachocki writes plainly that this has been OpenAI's primary bet on checking its own models, and that the lab's own evaluations say their ability to rely on it is progressively diminishing.
The idea is appealing because it scales. If you push hard on the outcomes of the reasoning process but do not supervise the process itself, then the chain of thought has no direct incentive in training to hide misaligned ideas or objectives. In plain terms: let it think out loud, do not punish it for what it thinks, and you get to hear the real thing.
They took that seriously enough to pay for it in the product. When o1-preview shipped, the team deliberately designed it to hide the chain of thought, mainly to protect that reasoning from supervision pressure over the long run. A footnote adds that preventing distillation was the secondary reason, and that keeping the chain of thought readable, its monitorability, was explicitly the bigger priority throughout development.
Three factors combine into the decline.
- Today's models run in far more complex environments than o1-preview did. Their reasoning is blended together with talking to people, talking to other AIs, and calling tools. Many of those interactions have to be supervised, which blurs the boundary the lab was trying to hold.
- The AI keeps getting better at reasoning about its own reasoning process, and at manipulating it.
- With better pretraining, the big training run that happens before any task-specific tuning, models get much smarter without verbalizing their reasoning at all.
He does not call this hopeless. He is hopeful about interventions that would make model reasoning easier to monitor, and he sees great value in combining chain of thought ideas with activation monitoring, training monitors that have direct access to the network internals. But the line he writes right after that is:
"I expect general AI progress to increasingly be bottlenecked by confidence in monitoring."
Jakub Pachocki, in An Alien Mind
Put that into the language of someone who builds systems. Reading the reasoning a model types out is losing its value as evidence, and it is losing it at the one workbench in the world with the best instruments on it, where people can see the weights, the logs, all of it.
And what do we have? Text on a screen. Which means reading that text was never our verdict to begin with.
Part 4Reasoning that never gets typed out already has a recipe
Two months before this essay came out, we read a survey called A Survey on Latent Reasoning (arXiv 2507.06203) and wrote notes on it on July 11, 2026. Latent reasoning means letting a model do its thinking on the inside, without emitting those thoughts as tokens at all.
One method in there is called Stepwise Internalization, and it does exactly what the name says. During finetuning it progressively removes chain of thought tokens, which forces the model to internalize its reasoning patterns into its parameters instead. What you end up with is reasoning encoded as hidden computation rather than as tokens anyone can read.
How well do methods like that work? The same notes cover another one, CODI, which reaches parity with explicit chain of thought on GSM8K, a set of grade school math word problems used as a benchmark for step by step reasoning. That is the first latent method to match explicit chain of thought on this kind of task.
None of this means the model you called this morning already works that way. It means the direction that pulls the account apart from the computation is not something anxious people imagined. It has a recipe, it has measured numbers, and it was published two months before Pachocki's essay.
Part 5You do not need a frontier model. This already happened at our desk.
Back to real work. We had a script with exactly one job: write an index file over the previous one. It ran, finished, and printed that it had written 23,109 characters. The script also checked itself one layer further, that the length was under the cap. Passed. Two green signals in a row, so we reported the job done.
What was actually true at that moment: the file on disk had already been rolled back to the previous version. A separate guard was standing at the point right after the write. It saw that the new content failed its bar, set the new content aside, and restored the old one. The message saying it had reverted went out on a channel nobody was reading.
The account was correct in every word. And 23,109 is a real number, nobody invented it. It was genuinely the length held in the script's own memory at that moment. The destination just did not exist.
Notice that no model lied to us anywhere in this episode. Nothing about alignment went wrong. We took an account and used it in place of a receipt, and that is all that happened. It is a different failure from AI making things up, which we covered in Why your AI agent lies to you.
And the step that slipped was not the biggest one in the job. It was the last and smallest one, the step that felt like it did not need checking.
Part 6So where does the check belong
On the artifact, not on the account. All of it comes down to one sentence.
Put the check on the artifact, the thing the run actually produced, and read it back from the same place the real system reads it from. Do not put the check on the model's account of what it did.
When you are not sure which kind a check you already have is, ask it three questions.
- If you deleted every word of the account, could this check still tell you whether the job succeeded? If it could not, the check is measuring talk.
- Does the value the check compares against come from the same process that did the writing? If it does, you have not left the narrator's mouth yet.
- Is the place the check reads from the same place the real user reads from?
The first move is small enough to make tonight. Take the one automated job you trust most and add a single line that opens the real thing back up from disk and compares it against what should be there. If it does not match, exit with something other than 0.
# add this after the step that writes the file, not after the message that says the write succeeded
grep -q "the line that should be there" ./out/index.md || { echo "artifact check FAILED"; exit 1; }This is not only for people who write code. If you have an agent answering email, do not trust the line that says it sent the reply; open the Sent folder. If it fills in a spreadsheet for you, do not trust the words all updated; open the sheet and count the rows. Same idea every time. What the work produced is the receipt, and everything else is an account of it.
One more thing before you trust that new check: make it go red once, where you can watch it happen. Break the target file on purpose and see whether the check catches you. A check that has never been seen failing and a check that is already dead look exactly the same from the outside. We wrote that one out at more length in the mutation testing post.
Pachocki closes his essay hoping voluntary slowdowns become commonplace until the world has shared safety bars. For a lab, those bars have to wait on third-party auditors, on governments, on international bodies. Your bar does not have to wait for any of them. You can set one tonight.
A failure that looks similar but is not the same, the model inventing content, is covered in Why your AI agent lies to you
Writing a bar an agent can actually meet, instead of the word done, is covered in Runnable acceptance criteria, not adjectives
Proving the check you already have still bites is covered in Mutation testing
Once the work passes the check, someone still owns it, which is covered in AI writes the code, but who signs for it
Referencesand sources
- Jakub Pachocki, An Alien Mind, OpenAI, September 6, 2026. Every statement about the essay in this post was checked back against the original.
- A Survey on Latent Reasoning, arXiv 2507.06203, the source for Stepwise Internalization and for CODI reaching parity on GSM8K.
- The index-script episode in Part 5 comes from our own working notes, dated September 3, 2026.