Reviewing AI agent output: why the first flaw you find hides the second

By the founder of Cerevisor. Checked on 15 October 2026 against the source code of Cerevisor 3.8.1.

The first flaw in AI agent output ends most reviews

Picture an ordinary review. An agent writes a report. The reviewer reads it, finds a wrong date in the second paragraph, sends it back with a note, gets a fixed version, checks that the date is now right, and approves it. A second problem, three paragraphs below the first, was there the whole time and nobody saw it.

Nobody skimmed. The report was read. What happened is that finding the date felt like finishing the job. The review had a target, the target was hit, and attention moved on.

This week Stack Overflow’s 2026 developer survey landed, and the coverage put a number on how much of the working day now goes to this kind of reading. As Entrepreneur reported on 7 October, “31% of daily AI users spend at least four hours a day using the tools,” and “Some developers describe being reduced to button-pushers, passing along AI output instead of solving problems themselves.” Four hours of reading output is a lot of chances to stop one flaw early.

Subsequent search misses: what radiology measured

Radiologists named this decades ago. They called it “satisfaction of search”: once a reader finds one abnormality on a scan, they become more likely to miss a second one on the same scan. Cognitive scientists later renamed it the subsequent search miss, which is the plainer label. It means a miss that happens after a first find, in the same search.

A 2021 review in Cognitive Research by Adamo, Gereke, Shomstein and Schmidt pulled the radiology and lab work together. The size of the problem in medicine is not small:

The review also makes the point that matters most for agent work: this is not a laziness problem. Trained experts, paid to look carefully, still do it. The researchers discuss several causes, including ending the search too early and a first find pulling attention toward things that look like it. None of those causes is fixed by telling the reader to try harder.

How a send-back loop turns one finding into a pass

Agent workflows make the effect stronger in one specific way. Most review loops are built around a send-back: the reviewer finds something, the agent revises, the reviewer looks again. The second look is almost always aimed at the thing that was sent back. Was the date fixed? Yes. Approved.

So the loop takes one finding and turns it into a pass for the whole piece of work. Nothing about the second problem was ever checked, and the record now says the work was reviewed twice.

A send-back loop can take one finding and turn it into a pass for everything else on the page.

This applies whether the reviewer is a person or another model. Most review steps, whether an AI powered code review tool, an agent reviewer or a person’s habit, come down to one question and one verdict. That is a good shape for a decision. It is a risky shape for a search.

What Cerevisor’s reviewer agent does by default

Cerevisor has a review step for exactly this job. A reviewer agent is a second agent, added to the canvas as its own card, that reads what another agent produced and either approves it or sends it back. It only reads: its tool set is fixed to reading, searching and listing, so it cannot change the work it is judging.

Its default instructions, which sit on the card where anyone can read and edit them, ask it to read what the producing agent said it would produce, post one review result with a verdict of approved or sent back, attach at least one piece of evidence, and say in one sentence why. Evidence here means something checkable: a file, a quoted line, a web page, or a step of the run.

Two rules in the run itself help against the miss. A review result with no evidence attached does not count, and the board says so in plain words. And silence is never read as approval: if the reviewer says nothing usable, the work is marked as not reviewed rather than passed. A send-back with evidence buys exactly one revision and one more review round, not an open-ended loop.

What the defaults do not do is force a second search. One verdict, one sentence, at least one piece of evidence: a careful reviewer can satisfy all three after the first real finding. I think that is the right default for most workflows, because a short, evidenced verdict is easy to audit. But for work where a second miss is expensive, the card’s instructions are the place to change it. The full behaviour is in the guide to having an agent’s work reviewed.

Splitting AI powered code review into separate questions

The research points to a structural fix rather than a willpower one: give the search more than one target, by name, before it starts. In practice that means rewriting the reviewer’s instructions as a short list of separate questions, each one answered with its own evidence before any verdict is allowed.

1. Name the questions before the review starts

   For a report, three questions often cover it: are the facts sourced, does it answer what was asked, and is anything missing that was promised. Write them into the reviewer card's instructions.

2. Ask for evidence per question, not per verdict

   One quoted line or file for each question. A question with no evidence means that part was not checked, which is exactly what a subsequent search miss hides.

3. Re-ask every question after a revision

   The second round should run the same list again, not just confirm the fix. That is what stops a send-back from becoming a pass for everything else.

4. Keep the yes-or-no parts out of the reviewer

   Narrow judgments, such as whether a loop should stop, belong in a quick check: a small yes-or-no question a model answers during the run. In Cerevisor a quick check is told to judge only the evidence it is given, and an answer below its confidence bar is treated as unsure rather than picked.

The quick-check behaviour is described in the quick checks guide. Splitting work this way keeps each check asking one clear question, which is where a single verdict is safe.

The cost of reviewing AI agent output, honestly counted

None of this makes review free. A list of three questions takes longer to answer than one, for a person and for a reviewer agent, and on a short output it can be overkill. The research on subsequent search misses also comes mostly from medical images and lab search tasks, not from reading agent reports, so the size of the effect in agent review is not something I can give you a number for. What carries over is the mechanism: a found target ends the search early.

There is also a human limit no checklist removes. A person reading four hours of output a day will tire, and a long list read by a tired reader turns back into one question with extra words. That is a reason to put the list on the reviewer agent first and keep the person for the questions only a person can answer.

The calm takeaway is that the second flaw is not missed because anyone is careless. It is missed because the review was shaped like a single question. Pick one agent whose work you already review, open its reviewer card this week, and rewrite the instructions as three named questions that each need a piece of evidence. Then look at the next run and count how many of the three came back with something in them.