Human in the loop

Read these first

• The agent loop implemented from scratch
• Architectural mitigations

What you will know

• Insert an approval gate before a consequential action

  Retries, parallel branches, and tool-side side effects must not bypass the checkpoint.

• Choose which actions need one, and defend excluding the rest

  Irreversible external writes deserve more friction than read-only or reversible actions; otherwise operators rubber-stamp every pause.

• Let a user edit a plan rather than only accept or reject

  The editable representation needs validation and a clear diff so changes cannot smuggle in a new unsafe operation.

• Handle an approval that never comes

  Define who owns the case, what happens on expiry, and whether work is cancelled, retried, or escalated.

• Measure approval, override, timeout, and post-action outcome rates to tune confidence thresholds

  Operator behavior is shaped by workload and alert quality, so raw model confidence is not enough to calibrate routing.

Chapters

1. 1

   An approval attaches to a payload, not to a tool

   The gate that matters sits in front of the action, and it has to hold when the route to that action is unusual: a retry after a timeout, a parallel branch, a tool that produces its side effect before returning. Work the case where a send is approved for one recipient, times out, and the retry carries a different address. The approval no longer applies, and the gate must fire again.

   Read chapter
2. 2

   Gating everything that writes is how the gates stop working

   Choosing where friction goes is the design decision, and "gate anything that writes" fails by volume: a queue full of reversible and low-stakes actions trains operators to approve reflexively, so the refund gets the same two seconds as a ticket search. Take an agent with eleven tools, place the gates, and then do the part that is actually hard — defend every exclusion on reversibility and blast radius.

   Read chapter
3. 3

   An approval nobody can decide in context becomes a reflex

   Volume is one reason a decision takes four seconds; what the request put in front of the operator is the other. A decidable request shows the exact payload about to execute rather than a paraphrase of it, the reversibility and blast radius of the action, the diff against what was expected or previously approved, and what happens if nobody answers. Anything less asks an operator to approve a tool name. Design the request alongside the gate, because narrowing the gate set is only one of the two levers you have when approvals turn into rubber-stamping.

   Read chapter
4. 4

   Editing a plan demands validation that accepting it does not

   Letting an operator change the agent's plan rather than only accept or reject it is better feedback and a wider hole. An edited plan is untrusted input: it needs the same validation as a generated one, plus a visible diff so that nobody approves an operation quietly introduced during the edit. Which representation makes that diff readable in the first place is part of the design.

   Read chapter
5. 5

   An unanswered approval needs a default, an owner and an expiry

   Silence is the case teams forget to design. Define the window, decide what expiry does — cancel, retry, or escalate to a named person — and make sure the pending case belongs to someone rather than to a queue. Without that the agent's work sits in an indeterminate state and the real policy becomes whoever notices first, which is not something you can describe to an auditor.

   Read chapter
6. 6

   A three-day wait cannot live inside an in-memory await

   Some gates are answered in seconds and some in days, and the second kind breaks any implementation whose lifetime is a process. A deploy, a crash or a scale-in loses the callback and the run with it. Replace it with durable state: persist the pending approval and the run's position, let the process exit, and resume from storage when the decision arrives — including when it never does.

   Read chapter
7. 7

   A 97% approval rate is a measurement of your operators

   Approval rate, override rate, timeout rate and post-action outcomes describe how people behave under load, not how often the model is right. A median time to decision of two seconds and falling reads as rubber-stamping whatever the percentage says. What to check next is whether rejections correlate with anything at all, and those rates rather than raw model confidence are what should tune the routing thresholds.

   Read chapter
8. 8

   A confidence threshold routes an action three ways

   The rates from the previous chapter are inputs to something, and this is what they feed. Each action arrives with some confidence signal — a verifier score, a judge score, the retrieval support behind it, or the model's own self-report, which is the weakest of the four — and a threshold decides whether it executes unattended, goes to a gate, or is refused outright. Set the first thresholds from the cost of each error rather than from the signal's scale, then move them on measured approval, override, timeout and post-action outcome rates, so a gate that is always approved rises out of the queue and a class of action that keeps being overridden drops into it.

   Read chapter