TL;DR
Anthropic removed human approval from Claude Code because it was granted 93% of the time, not because the check itself was failing: it had become a reflex.
AI saves 11 hours a week, but checking it back eats 6.4 hours, leaving a real net of 4.6 hours, with 58% of the gain gone.
69% of AI users ship work they haven’t fully verified, and 41% admit they sometimes couldn’t explain it if asked.
For two years, building Claude Code meant asking permission. Every time the agent wanted to delete a file outside the expected folder, run a command with side effects beyond code, touch a system setting, a prompt appeared. A binary question, yes or no, take it or leave it. As of August 11, 2026, that step is gone by default. Starting August 14, according to reports at launch, Auto mode becomes the standard behavior on Pro, Max, and Team plans.
The detail that convinced Anthropic to retire that step isn’t how many dangerous actions people missed. It’s how often they said yes. 93% of permission prompts got approved. Nine requests out of ten, an almost automatic click. That doesn’t mean people using Claude Code were careless or distracted. It means that, faced with a chunk of generated code, human judgment had nothing solid to lean on. There was no fast way to tell if that specific command was harmless, so the thumb went up out of habit. A ritual survives long after it stops working. That’s why it took two years before anyone turned it off.
Whoever read this as a story about trust in AI read the wrong story. The interesting fact isn’t that people trusted the system. It’s that Anthropic could publish, in the same post, another number: 17% false negatives, the genuinely dangerous actions the new automated system still lets through, called in their own words “the honest number.” Next to it, 0.4% false positives on real traffic, how often the system blocks something harmless.
Both numbers exist only because Claude Code has what I’ll call an oracle from here on: a log, a test that passes or fails, a filesystem where the file is either there or isn’t, something that can say with certainty whether an action went wrong. Anthropic can measure its own error rate because it has a judge that isn’t an opinion. Ask for the same percentage on an email an agent wrote, or a meeting summary, and nobody publishes it, because nobody can calculate it. It isn’t a failure of will to measure. It’s a missing measuring instrument.
Underneath, it isn’t a rule written into the program: it’s a second model acting as the judge. One layer screens what Claude reads — a server-side probe sweeps files, web pages, and command output before the agent ever sees them, against prompt injection. The other screens what Claude does: before an action fires, the transcript goes to a classifier running on Sonnet 4.6, a separate model from the one doing the work, tuned to block anything irreversible, destructive, or aimed outside your environment. When it gets it wrong it prefers recovery, denying that single action and carrying on, over slamming the session shut. The detail that matters is that this judge is deliberately blind: it sees your messages and the agent’s tool calls, and neither Claude’s own reasoning nor the contents of the files. You can’t explain to it why the action makes sense — it can’t be talked around by the agent, or poisoned by a document. Below that sits something that isn’t AI at all: an operating system sandbox that allows writes only inside the working folder and denies the network by default. That’s what removes 84% of the permission prompts that would otherwise remain, and it isn’t a judgment call: it’s a wall. It’s there because the judge lets that 17% through. Boris Cherny, who leads Claude Code, put it plainly:
Checking has never been a problem of attention. It’s a problem of having, somewhere, something that can tell you when you’re wrong.
Glean’s Work AI Index 2026, published June 10 from a sample of 6,000 full time digital workers across the US, UK, and Australia, puts the same mechanism at scale. 87% of digital workers already use AI at work. It saves an average of 11 hours a week, more than a full day out of a five day week. But 6.4 of those hours come back as what Glean calls botsitting: giving the AI context, checking output, correcting mistakes, cleaning up. The net is 4.6 hours. 58% of the gain disappears, and botsitting eats up 37% of all time spent with AI, more than the time spent actually producing with it.
The number that closes the loop is 69%: the share of people who admit to shipping work they haven’t fully verified, don’t fully understand, or couldn’t confidently endorse. Inside that group, 41% say they sometimes hand over work they couldn’t explain if someone asked them to walk through it. It isn’t a verdict on anyone. It’s proof that where there’s no oracle, checking doesn’t cost time. Most of the time, it simply doesn’t happen.
There’s a line I read under one of my notes, in the comments, that says this better than I’d managed to say it myself. It isn’t mine, and I’m paraphrasing rather than quoting: checking is cheap only where a machine can do the checking. Code has a compiler and a test suite, so what’s left to check by hand is only whatever the tests don’t cover, a small slice. A paragraph of prose has nothing like that. No compiler tells you a sentence is right. So if you want to check it for real, you read it line by line, with the same attention you’d have given writing it yourself.
That’s where I borrowed the word I’ve been using for a few paragraphs now. I didn’t coin it. I took it from a reader who’d already understood it better than I had.
This piece had its own moment of truth on August 11, while I was writing it. The 93% that opens it, the share of permission prompts users approved instead of reading, was supposed to be a different number. Four points higher: 97%. That’s the one circulating on TechCrunch and on Cybersecurity News, convenient, rounder, ready to be pasted into the lede and sent to roughly 1,400 subscribers with my name on it.
I went to the source, found the 93% in Anthropic’s engineering post, and wrote in my notes that the press had inflated the number. I put together a list of banned figures, 97%, 89%, 13.6%, 1,053 testers, and dropped it into my instructions as a final check. I was fairly pleased with myself.
They were all real. They’re all in Anthropic’s launch post, on their own blog, published the same day as the engineering one: “users approve 97% of permission prompts,” and the controlled experiment with 1,053 paid testers where human review catches 13.6% of dangerous commands and auto mode catches 89%. TechCrunch hadn’t inflated anything. It had quoted. My check had found a real discrepancy, two posts from the same company four points apart, with neither one explaining what the other is counting, and it handed that back to me as a lie by the press. I verified, I passed my own verification, and I was wrong anyway.
That’s the point of this article, landing on the person writing it. “Go to the primary source” looks like an oracle, and it isn’t: it assumes you already know which source is primary, and that part of the judgment has nothing that can tell you you’re wrong. On one item the check did work: a per user cost for Claude Code, searched for on purpose and cut because it exists in no verifiable source at all. There, the absence was checkable. On the rest it produced a false negative, with exactly the same confidence. Nobody would have noticed. Both numbers are equally plausible, the argument doesn’t shift by a comma, and no reader fact checks a background statistic in a piece about something else. Not even the company that built a classifier to measure its own errors has an oracle over its own prose. Neither do I.
Here’s the tool I use to decide, task by task, what I can safely hand off and what I can’t. The first question doesn’t ask for a judgment about quality. It asks you to name the specific thing that would say “wrong.” That’s the one phrasing that closes off the easy escape of “I’ll check it after,” because it forces you to point at something concrete instead of good intentions. The second question exists because a signal that only shows up after the damage is irreversible isn’t an oracle. It’s an autopsy.
To use it, swap the last line for a task you actually do, with real detail. The more specific, the more useful the answer. And expect a surprise or two: run a throwaway script through it and the answer comes back “no oracle,” because the compiler tells you the code runs, not that it does the right thing. The oracle isn’t the language you work in. It’s the test suite that already exists around that work.
You are helping me decide what I can safely delegate to an AI and what I cannot.
I will describe one task I do regularly. Work through these five questions in order. Do not skip any, and do not soften your answers.
1. THE ORACLE
If the AI did this task badly, what would tell me? Name the specific thing: a test that fails, a number that does not reconcile, a system that rejects the input, a customer who complains. If the only thing that would tell me is my own judgment reading the output, say exactly this: “No oracle.”
2. THE DELAY
How long after the work is finished does that signal arrive? Immediately, hours, weeks, or only once something has already gone wrong in public?
3. THE COST OF BEING WRONG
If a bad output got through undetected, what is the worst realistic consequence? Be concrete and unflattering. No reassurance.
4. THE CLASSIFICATION
Based on 1 to 3, put the task in exactly one class:
FULL ORACLE, an automatic signal arrives fast.
PARTIAL ORACLE, part of the output is checkable and part is judgment. Say which part is which.
NO ORACLE, nothing but me can tell that it is wrong.
5. THE INSTRUCTION
One line telling me what to do:
FULL ORACLE, delegate it fully and check the signal, not the output.
PARTIAL ORACLE, delegate it, then check only the part that nothing else can check.
NO ORACLE, either do not delegate it, or build the oracle first. If you say build it, tell me what the cheapest possible oracle would look like for this specific task.
Rules for you: do not reassure me. “The AI would probably do this well” is not an oracle. If a task has no oracle, say so even when the AI would likely do a good job, because that is exactly the case where I will stop looking. If you write “No oracle,” write it plainly, with no conditions in brackets after it. End with a single line, in exactly this format, with no text after it: CLASS: [FULL ORACLE | PARTIAL ORACLE | NO ORACLE] - ACTION: [12 words maximum]
My task: [DESCRIBE ONE TASK YOU DO REGULARLY]
If you want to adapt this to your own case, change only the last line. Everything above it is already calibrated to leave you nowhere to hide.
That one’s yours, whether you ever pay me a cent or not.
The mistake I see most often isn’t buying AI. It’s buying it per seat, one license per person, and calling adoption success. 87% of digital workers already use AI at work. Only 13% say their organization is meaningfully better off for it. The right measure isn’t per person. It’s per task. And the question worth asking, for every task, is one: is there an oracle here or not?
Nobody has ever put a budget on checking time, and the difference between delegating and verifying was never taught to anyone, not in school, not on the job. That’s the gap I spent the last year closing with From User to Orchestrator, a program that starts exactly here, with how you classify a task before you hand it to an AI.
JPMorgan cut 30-40% of jobs in some units thanks to AI, and costs still went up, as I laid out when I walked back my read on AI driven layoffs a few weeks ago. The work didn’t disappear. It moved to wherever nobody’s measuring it. That’s the tax on checking, at the scale of an entire company.
If you’ve just run three of your own tasks through the Oracle Test, you’ve probably already spotted the real problem. It isn’t the tasks with a full oracle, those get delegated and forgotten. It’s the ones classified NO ORACLE, the ones the prompt above helps you spot but doesn’t help you fix. How do you build an oracle for a task that doesn’t have one today, and how do you know whether building it costs less than checking by hand every week forever? Below you’ll find my filled in map of twelve recurring tasks, the prompt that designs the missing oracle, and the arithmetic that tells you how many weeks it takes to pay for itself. Take your 3 no oracle tasks and run them through it.