Verification & Feedback Loops

Module 9 · Master Course

45 min · Adding verification improves output quality 2–3× (Cherny)

The Cherny claim

Adding verification improves output quality 2–3× (Boris Cherny, Claude Code founder). The cost of an extra LLM call or test run is real; the quality multiplier is larger.

Four verification methods

Method	Reliability	Cost
Computed (linter, tests, build)	deterministic	low latency
Visual (Playwright, pixel-diff)	high for UI	medium
Model-judged (independent subagent)	flexible; semantic	high (extra LLM call)
Human	highest	highest latency

Tight vs staged vs loose

Tight: verify after each tool call. Highest quality; highest cost.

Staged: verify at checkpoints (post-plan, post-impl, post-test). Balanced.

Loose: verify at task end; restart if failed. Lowest cost; most rework.

Retry budget + flaky tests

Without a retry budget, a flaky test traps the agent in verify-fail-retry (a Module 7 stuck-loop variant).

Flaky handling: run 3×, majority vote (2/3 pass = pass). Or quarantine flaky tests as warnings, not failures.

Verification as security control

Beyond quality: verification answers does the output do what was intended? — also did the agent do what it was supposed to, not what an injection told it to?

Computed verification catches injected code changes (they break tests). Model-judged catches semantic deviation. Verification is Module 6's layered defense, not just QA.

3-tier verification as n8n

Agent output
   │
   ▼
Tier 1: Computed (lint + compile) ─── fail → return to model
   │ pass
   ▼
Tier 2: Model-judged (independent subagent) ─── fail → return to model
   │ pass
   ▼
Tier 3: Human (optional low-risk; required destructive/external)
   │
   ▼
Verified output

Tiered: cheapest+deterministic first; expensive+flexible second; human last resort. Retry budget caps retries.

Takeaways

Verification 2–3× quality (Cherny). The cost is worth it.
4 methods: computed (default), visual (UI), model-judged (semantic), human (last resort).
Retry budget prevents flaky-test stuck-loops.
Verification = security control. Catches injected deviations; defense in depth.

Next: Module 10 — Observability & Debugging.