Verification

Verification loops for AI agents

A verification loop does not just generate more output. It checks evidence, repairs, and stops only when the evidence passes or the approval boundary is reached.

01MakeProduce an artifact
02ChallengeUse an independent check
03AcceptKeep the evidence
01

What is a verification loop?

Looking for a fix for a Cloudflare or CAPTCHA 'verify you are human' loop? This page is about AI agents checking their own work.

A verification loop does not just generate more output. It checks evidence, repairs, and stops only when the evidence passes or the approval boundary is reached. The check sits outside the draft: a test, a source, a screenshot, or a person. That is what an AI loop is when the goal is proof rather than another paragraph.

02

Verification loop vs confirmation loop vs closed-loop verification

These three names get mixed together. They are different jobs, and a single agent run can use more than one.

KindWhat it does
Verification loopAn independent check of evidence before stopping.
Confirmation or approval loopA human confirms before an action.
Closed-loop verificationThe result of the check feeds back into the next attempt until it passes or hits the budget.

A confirmation loop is the human gate. Closed-loop verification is what happens after a failed check: the failure goes back into the next attempt instead of ending as a confident draft. The verification loop is the check itself, kept separate from the part that made the draft.

03

How to add a verification loop to an agent

Add the check before you ask the agent to keep going. The point is an outside signal the maker does not grade alone.

  1. Choose an external verifier.
  2. Keep the verifier separate from the maker.
  3. Record evidence each pass.
  4. Cap retries and budget.
  5. Escalate to approval at public, financial, legal or account boundaries.

An external verifier is a test, a source check, a diff, a screenshot, or a browser result that the draft did not write for itself. Keeping it separate means the agent that edits the work is not the only judge of that work. Recording evidence means each pass leaves the command output, the source note, or the screenshot, not only a sentence that says it passed. The cap is a stop condition: retries and budget end the run when the check keeps failing. The glossary entry for verifier is the short form of that evidence check.

04

Common verifier failures

A named verifier can still be fake. These are the failures this library treats as design errors.

  • The agent edits its own test.
  • The rubric is too vague.
  • The verifier checks style instead of evidence.
  • No stop on repeated failure.

If the agent can rewrite the test, the check no longer proves the work. A vague rubric lets a confident paragraph count as a pass. A style check can approve wording while the claim, the test, or the page is still wrong. And without a stop, the same failure becomes another turn. Pair the check with a budget and an approval boundary so a repeated miss ends in a report instead of another unattended edit.

05

Verification types

VerifierWhat it proves
TestsBehavior still works after the change.
Source checksClaims match source material.
Diff reviewThe changed files are scoped and intentional.
Browser checksThe user-visible flow works in a real page.
ScreenshotsVisual output matches the intended state.
Human approvalThe loop has hit a public, reputational, legal, financial, or account boundary.
06

Deterministic vs judgment loops

Loop typeGood forRisk
Deterministic loopTests, page speed, lint, broken links, source checksSafest because the verifier is external and measurable.
LLM-as-judge loopArchitecture cleanup, docs quality, product reviewUseful but squishier; needs a narrow rubric and budget.
Human-gated loopPublishing, sending, deleting, production changesSlower, but safest for reputation, accounts, money, and legal risk.
07

Relevant loops

LoopCategoryDifficultyCadenceVerification
Pre-Publish Source CheckContentIntermediateBefore publishing factual workEvery checkable claim is supported by a current source or visibly flagged for an editor.
Social Source to InsightContentIntermediateAfter saving a high-signal sourceSource captured, takeaways extracted, draft angles written, and no public post is published without approval.
Accessibility RepairDesignAdvancedBefore launch or when audits failNo confirmed accessibility blocker remains in the agreed pages, components, or tasks.
Error Message RewriteDesignIntermediateWhen users hit confusing errorsEvery in-scope user-visible error is accounted for, rewritten or blocked, and verified in a reachable state.
Architecture Rubric RefactorEngineeringAdvancedWhen architecture work has a defined scopeScoped module meets the written rubric, tests pass, and unresolved objections are explicit.
CI OptimizationEngineeringAdvancedMonthly or when CI is painfulCI p50/p95 improves against the same workflow without weakening tests or hiding failures.
Claude Code Repo ReadinessEngineeringBeginnerBefore major agent workRepo has agent instructions, documented commands, architecture notes, risk areas, and a docs/loops scaffold.
Cold Load TrimEngineeringAdvancedWhen first visit feels heavyInitial screen downloads fewer bytes while screenshots and behavior remain unchanged.
Fresh Clone OnboardingEngineeringIntermediateBefore onboardingA clean machine reaches the documented ready state using only the README.
Project Docs FreshnessEngineeringBeginnerNightly or after meaningful code changesChanged behavior, APIs, CLI commands, config, and workflows are reflected in docs. Docs checks pass.
Test Flake StabilizerEngineeringIntermediateWhen tests are inconsistentThe repaired test and full suite pass for the required consecutive-run streak.
Test and Logging CoverageEngineeringIntermediateWeekly or before releaseCritical flows have useful tests and structured logs for representative success and failure paths.
Adversarial PR ReviewEvaluationAdvancedFor meaningful PRsAn independent critic approves the unchanged version or only accepted findings remain.
Browser Quality StreakEvaluationIntermediateBefore releaseN realistic scenarios pass consecutively, and earlier failures have regression coverage.
Open Loop and Stale Memory CleanupKnowledgeBeginnerWeeklyNo current open loop is contradicted by recent daily or project notes.
Research to ArtifactKnowledgeIntermediateWhenever research must support a decisionThe artifact meets acceptance criteria, traces important claims to sources, and states uncertainty plainly.
Source Library Ingestion QAKnowledgeIntermediateAfter each source captureMetadata complete, transcript/article state honest, useful takeaways present, and qmd retrieval verified or refreshed.
Living StoryOperationsIntermediateWeekly or per project windowEvery prior thread is carried forward, closed with evidence, or flagged stale/needs-review.
Production Error SweepOperationsAdvancedDaily or after incidentActionable errors are fixed with reproduction or tests, or explicitly classified as noise.
Dependency CVE BurndownSecurityAdvancedAfter security scanNo exploitable high or critical CVE remains without an explicit risk decision.
08

FAQ

What is a verification loop in AI?

A verification loop makes an AI agent prove its work before it stops. Each pass checks evidence such as tests, source notes, a diff, a screenshot, or a browser result. The agent may repair and recheck. It stops when that evidence passes or when the budget or approval boundary says to stop.

What is closed-loop verification?

Closed-loop verification means the result of the check feeds the next attempt. A failed test, a missing source, or a broken page becomes the input for the repair, and the same check runs again. The loop ends when the evidence passes or the budget is spent.

What is the difference between a verification loop and human-in-the-loop?

A verification loop checks evidence, and that evidence can be a test or a person. Human-in-the-loop means a person reviews or approves at defined points, usually at the approval gate. Use a check for facts you can measure, and a person before public, financial, legal, or account actions.

Which loops in the library use verification?

The Evaluation category groups loops built around checks. Browse Evaluation loops to see which ones use verification, then open a loop for its verifier, stop condition, and approval boundary.

Define the exitBrowse evaluation loops