Making an agent's "done" something you can trust

I write almost all of my code together with AI agents. After a while I noticed that the most expensive mistake isn't writing the wrong code. It's the agent saying "fixed" when it isn't. The build passes, the release goes out, nothing looks wrong in the logs, and the one result the thing is supposed to deliver stays broken the whole time.

Here are three real incidents, and what I changed after each one.

Kay is my voice input tool. You hold fn and talk, let go, and the text appears at your cursor. In 1.4.4 I wanted a spinner centered in the floating panel, and at the same time made the panel resize to fit its content. From that version on, the app crashed every time you released fn. The panel resize and a SwiftUI animation landed in the same layout pass.

This went on for about 27 hours, across three versions: 1.4.4, 1.5.0 and 1.5.1. During that time the agent shipped releases, wrote "fixed", and even recorded a hands-on test of auto-update in the knowledge base. The crash reports were sitting in the system the whole time. No step ever read them.

Every check was green: offscreen rendering looked right, notarization passed, idle CPU was normal, auto-update installed. But not one of those checks would turn red when "release fn, text appears" broke.

Before the retrospective was even finished, the agent shipped 1.5.2 based on a root cause it had reasoned out. It decided the spinner was the problem and replaced it with a drawing method that doesn't go through view updates. After installing it, the app still crashed.

The real fix was structural. Starting with 1.5.3 the panel has a fixed size. It only moves, it never resizes, so that whole class of conflict is gone. Later the panel's look changed six times in one day, and its size didn't change once.

I also added a release gate. make release only notarizes and installs on my own Mac. make promote lets a public release through only after it reads one real successful dictation on my machine and finds no new crash reports. Before turning it on I tried it against past data: counting from the moment 1.5.2 was installed, it would have refused. Counting from 1.5.3, it would have passed.

That gate lived for two hours. Every release had to wait on it, I found it too rigid, and I deleted it. In its place I wrote a rule: after every install, read the crash reports and the dictation records first, and until you've read them, the only thing you can say is "changed, not verified".

Two days later the rule's blind spot showed up. 1.6.0 froze the main thread when recording started. The only way out was force quit, and there was no crash report. "No new crash reports" simply doesn't react to a hang. What I added after that was structural again: recording startup moved off the main thread with a two-second limit, falling back to the built-in mic if it fails. And a watchdog that runs only during dictation: if the main thread doesn't respond for three seconds, it saves a snapshot of the state.

My transcription pipeline has a few self-checks: one looks for repetition loops, one looks for traditional Chinese characters. One day in August 2026 I found four checks at once that ran to completion, exited 0, printed normal-looking output, and weren't actually doing anything:

• A dot in the file name made all five post-processing steps get skipped. The archive folder was empty.
• The grep meant to catch repeated characters within a line doesn't support backreferences on macOS. That check had never worked, from the day it was written.
• The per-line repetition check added later: a 3.5-hour podcast had 828 consecutive lines of "对" ("right"), a model hallucination that ate 14 minutes of content. The check reported only the last 15 lines. Inside the loop the model emits blank lines, which chopped the run into pieces.
• The traditional-character check had been given a precondition, "if the language is locked to Chinese there won't be traditional characters", so the one time it most needed to fire, it was skipped.

Since then I have a rule: a check that has never fired is not evidence that everything is fine. It's evidence that it was never verified. Every new check has to come with an input that's known to trigger it, to prove it goes off. Before delivering, print the actual numbers from every check, because "the number is 0" and "the check never ran" look exactly the same in a log.

I wrote that rule in August. The Kay incident happened in October. The rule lived in one repository, and it couldn't stop what happened in another.

When I ask an agent to read a long document (a transcript tens of thousands of characters long, a long PDF), it often reads the beginning, the end and a few search hits, then hands over a summary that looks complete. It doesn't know what it missed either.

Once, for a 3-hour course, I had the agent read a different way. First split the material into reasonably sized chunks. The main agent reads every chunk itself, in order, without handing them to subagents. After each chunk it writes notes to a file. At the end it puts a coverage statement at the top of the notes: which file it read, how many lines, and the line range of each chunk.

That turned up things sampled reading can't find: the same number stated two ways (165 and 165,000); a price in the subtitles that said $641 when the later context meant $6.41; and a topic promised at the start that was never covered.

The same day I finished, I turned this way of reading into a public skill: deep-read.

All three come down to the same idea:

1. First, say clearly what the thing exists to do, then read that result directly instead of a stand-in. Voice input exists so that "let go, text appears". Not so that "the build passes".
2. Every check is a stand-in until it proves it's useful. Ask first: if the real result broke, would this check's result be any different? "No crash report" doesn't change when the app hangs, so it can't prove the app isn't broken.
3. When something goes wrong, change the structure, don't add a rule. If a hazard can be removed, remove it (a fixed-size panel). If it can't, add a gate a machine checks. Writing a rule that reminds the agent to be careful is the weakest option. I didn't fully hold to this myself: I swapped Kay's gate for a rule, and two days later I paid for it.
4. Keep the one doing the work separate from the one checking it. Agents naturally lean toward wrapping up, reporting good news and not causing trouble. So set things up so that without evidence, they can't say "done".