Engineering
Webhook Replay Harness
Turn unreliable webhook mysteries into replayable fixtures before touching live traffic.
Use when
An integration bug depends on third-party webhook payloads, retry timing, signatures, or idempotency behavior that is hard to reproduce from normal tests.
Intermediatedifficulty
Engineeringcategory
Mia daily expansionsource
Cadence
When webhook integrations fail, drift, or need safer changes
Verification
A captured or synthetic payload replays through a non-production handler, expected state changes match assertions, idempotency is proven, and live provider/customer side effects remain approval-gated.
Structured loop spec
| Field | Value |
|---|---|
| Name | Webhook Replay Harness |
| Category | Engineering |
| Trigger | When webhook integrations fail, drift, or need safer changes |
| Objective | Turn unreliable webhook mysteries into replayable fixtures before touching live traffic. |
| Allowed inputs | Relevant files, source notes, logs, tests, screenshots, metrics, or task state for this loop |
| Allowed actions | Collect representative payloads, headers, timestamps, signatures, retry metadata, and observed outcomes from logs or provider dashboards, redacting secrets and customer data.; Build a local, staging, or test harness that can replay the payload against the handler without sending emails, charging cards, posting messages, or mutating production data.; Add assertions for parsed event type, authentication, idempotency key handling, expected database changes, ignored duplicates, and safe error responses.; Patch the smallest proven handler, schema, queue, or retry issue, then rerun the replay harness plus nearby integration tests.; Stop if reproduction requires live production traffic, provider credentials with broad scope, destructive retries, or customer-visible side effects; queue those actions for approval. |
| Verification | A captured or synthetic payload replays through a non-production handler, expected state changes match assertions, idempotency is proven, and live provider/customer side effects remain approval-gated. |
| Stop condition | Stop when the verifier passes, the budget is exhausted, no progress is made, a blocker appears, or approval is required. |
| Budget | Set a time, turn, token, retry, file, or dollar cap before running the loop. |
| Approval boundary | Human approval required before publishing, sending, deleting, spending, changing accounts, touching production, or making reputational/legal/financial commitments. |
| Safe output | Pull request, patch, report, or evidence log |
| Works with | Claude Code, OpenAI Codex, Cursor, Gemini CLI, any tool-using coding agent |
Steps
- Collect representative payloads, headers, timestamps, signatures, retry metadata, and observed outcomes from logs or provider dashboards, redacting secrets and customer data.
- Build a local, staging, or test harness that can replay the payload against the handler without sending emails, charging cards, posting messages, or mutating production data.
- Add assertions for parsed event type, authentication, idempotency key handling, expected database changes, ignored duplicates, and safe error responses.
- Patch the smallest proven handler, schema, queue, or retry issue, then rerun the replay harness plus nearby integration tests.
- Stop if reproduction requires live production traffic, provider credentials with broad scope, destructive retries, or customer-visible side effects; queue those actions for approval.
Prompt
Run the Webhook Replay Harness loop. Given a webhook failure or integration change, collect representative payloads, headers, timestamps, signatures, retry metadata, and observed outcomes, redacting secrets and customer data. Build a non-production replay harness before editing live handling code. Assert authentication, parsing, idempotency, database effects, duplicate handling, and safe errors. Patch the smallest proven issue and rerun replay plus nearby integration tests. Do not replay against production, trigger customer-visible actions, send messages, charge cards, or use broad provider credentials without explicit approval.Run in Claude Code
Paste this into Claude Code (or any tool-using agent) to run the loop bounded: one change per round, the same verification every round, durable state files, and explicit stop conditions.
Run the "Webhook Replay Harness" loop from AI Loop Library (https://ailooplibrary.com/loops/webhook-replay-harness/) as a bounded loop.
Goal: Turn unreliable webhook mysteries into replayable fixtures before touching live traffic.
Rules: one change per round; run the same verification every round (A captured or synthetic payload replays through a non-production handler, expected state changes match assertions, idempotency is proven, and live provider/customer side effects remain approval-gated.); append each round to docs/loops/webhook-replay-harness/progress.md and update docs/loops/webhook-replay-harness/state.json; stop on verifier pass, 8 rounds, 3 consecutive failed verifications, no progress, a blocker, or anything needing human approval (money, production, outbound, deletion). Finish with a proof report: rounds used, changes made, verification output, remaining risk, and the next human decision.