Verification loops for AI agents
A verification loop does not just generate more output. It checks evidence, repairs, and stops only when the evidence passes or the approval boundary is reached.
What is a verification loop?
Looking for a fix for a Cloudflare or CAPTCHA 'verify you are human' loop? This page is about AI agents checking their own work.
A verification loop does not just generate more output. It checks evidence, repairs, and stops only when the evidence passes or the approval boundary is reached. The check sits outside the draft: a test, a source, a screenshot, or a person. That is what an AI loop is when the goal is proof rather than another paragraph.
Verification loop vs confirmation loop vs closed-loop verification
These three names get mixed together. They are different jobs, and a single agent run can use more than one.
| Kind | What it does |
|---|---|
| Verification loop | An independent check of evidence before stopping. |
| Confirmation or approval loop | A human confirms before an action. |
| Closed-loop verification | The result of the check feeds back into the next attempt until it passes or hits the budget. |
A confirmation loop is the human gate. Closed-loop verification is what happens after a failed check: the failure goes back into the next attempt instead of ending as a confident draft. The verification loop is the check itself, kept separate from the part that made the draft.
How to add a verification loop to an agent
Add the check before you ask the agent to keep going. The point is an outside signal the maker does not grade alone.
- Choose an external verifier.
- Keep the verifier separate from the maker.
- Record evidence each pass.
- Cap retries and budget.
- Escalate to approval at public, financial, legal or account boundaries.
An external verifier is a test, a source check, a diff, a screenshot, or a browser result that the draft did not write for itself. Keeping it separate means the agent that edits the work is not the only judge of that work. Recording evidence means each pass leaves the command output, the source note, or the screenshot, not only a sentence that says it passed. The cap is a stop condition: retries and budget end the run when the check keeps failing. The glossary entry for verifier is the short form of that evidence check.
Common verifier failures
A named verifier can still be fake. These are the failures this library treats as design errors.
- The agent edits its own test.
- The rubric is too vague.
- The verifier checks style instead of evidence.
- No stop on repeated failure.
If the agent can rewrite the test, the check no longer proves the work. A vague rubric lets a confident paragraph count as a pass. A style check can approve wording while the claim, the test, or the page is still wrong. And without a stop, the same failure becomes another turn. Pair the check with a budget and an approval boundary so a repeated miss ends in a report instead of another unattended edit.
Verification types
| Verifier | What it proves |
|---|---|
| Tests | Behavior still works after the change. |
| Source checks | Claims match source material. |
| Diff review | The changed files are scoped and intentional. |
| Browser checks | The user-visible flow works in a real page. |
| Screenshots | Visual output matches the intended state. |
| Human approval | The loop has hit a public, reputational, legal, financial, or account boundary. |
Deterministic vs judgment loops
| Loop type | Good for | Risk |
|---|---|---|
| Deterministic loop | Tests, page speed, lint, broken links, source checks | Safest because the verifier is external and measurable. |
| LLM-as-judge loop | Architecture cleanup, docs quality, product review | Useful but squishier; needs a narrow rubric and budget. |
| Human-gated loop | Publishing, sending, deleting, production changes | Slower, but safest for reputation, accounts, money, and legal risk. |
Relevant loops
| Loop | Category | Difficulty | Cadence | Verification |
|---|---|---|---|---|
| Pre-Publish Source Check | Content | Intermediate | Before publishing factual work | Every checkable claim is supported by a current source or visibly flagged for an editor. |
| Social Source to Insight | Content | Intermediate | After saving a high-signal source | Source captured, takeaways extracted, draft angles written, and no public post is published without approval. |
| Accessibility Repair | Design | Advanced | Before launch or when audits fail | No confirmed accessibility blocker remains in the agreed pages, components, or tasks. |
| Error Message Rewrite | Design | Intermediate | When users hit confusing errors | Every in-scope user-visible error is accounted for, rewritten or blocked, and verified in a reachable state. |
| Architecture Rubric Refactor | Engineering | Advanced | When architecture work has a defined scope | Scoped module meets the written rubric, tests pass, and unresolved objections are explicit. |
| CI Optimization | Engineering | Advanced | Monthly or when CI is painful | CI p50/p95 improves against the same workflow without weakening tests or hiding failures. |
| Claude Code Repo Readiness | Engineering | Beginner | Before major agent work | Repo has agent instructions, documented commands, architecture notes, risk areas, and a docs/loops scaffold. |
| Cold Load Trim | Engineering | Advanced | When first visit feels heavy | Initial screen downloads fewer bytes while screenshots and behavior remain unchanged. |
| Fresh Clone Onboarding | Engineering | Intermediate | Before onboarding | A clean machine reaches the documented ready state using only the README. |
| Project Docs Freshness | Engineering | Beginner | Nightly or after meaningful code changes | Changed behavior, APIs, CLI commands, config, and workflows are reflected in docs. Docs checks pass. |
| Test Flake Stabilizer | Engineering | Intermediate | When tests are inconsistent | The repaired test and full suite pass for the required consecutive-run streak. |
| Test and Logging Coverage | Engineering | Intermediate | Weekly or before release | Critical flows have useful tests and structured logs for representative success and failure paths. |
| Adversarial PR Review | Evaluation | Advanced | For meaningful PRs | An independent critic approves the unchanged version or only accepted findings remain. |
| Browser Quality Streak | Evaluation | Intermediate | Before release | N realistic scenarios pass consecutively, and earlier failures have regression coverage. |
| Open Loop and Stale Memory Cleanup | Knowledge | Beginner | Weekly | No current open loop is contradicted by recent daily or project notes. |
| Research to Artifact | Knowledge | Intermediate | Whenever research must support a decision | The artifact meets acceptance criteria, traces important claims to sources, and states uncertainty plainly. |
| Source Library Ingestion QA | Knowledge | Intermediate | After each source capture | Metadata complete, transcript/article state honest, useful takeaways present, and qmd retrieval verified or refreshed. |
| Living Story | Operations | Intermediate | Weekly or per project window | Every prior thread is carried forward, closed with evidence, or flagged stale/needs-review. |
| Production Error Sweep | Operations | Advanced | Daily or after incident | Actionable errors are fixed with reproduction or tests, or explicitly classified as noise. |
| Dependency CVE Burndown | Security | Advanced | After security scan | No exploitable high or critical CVE remains without an explicit risk decision. |
FAQ
What is a verification loop in AI?
A verification loop makes an AI agent prove its work before it stops. Each pass checks evidence such as tests, source notes, a diff, a screenshot, or a browser result. The agent may repair and recheck. It stops when that evidence passes or when the budget or approval boundary says to stop.
What is closed-loop verification?
Closed-loop verification means the result of the check feeds the next attempt. A failed test, a missing source, or a broken page becomes the input for the repair, and the same check runs again. The loop ends when the evidence passes or the budget is spent.
What is the difference between a verification loop and human-in-the-loop?
A verification loop checks evidence, and that evidence can be a test or a person. Human-in-the-loop means a person reviews or approves at defined points, usually at the approval gate. Use a check for facts you can measure, and a person before public, financial, legal, or account actions.
Which loops in the library use verification?
The Evaluation category groups loops built around checks. Browse Evaluation loops to see which ones use verification, then open a loop for its verifier, stop condition, and approval boundary.