Thirty hooks with no statement of what they enforce

Two numbers arrived in the same week. The first was from a dashboard nobody had looked at in a while: median time from the model emitting a tool call to the tool actually running had gone from 40 ms at the start of 2023 to 260 ms, with a p95 of 2.1 seconds. The second was a support ticket from a developer whose test command had been rejected with Blocked: 'timeout' is not an approved binary (SEC-11). He had run pytest tests/unit -x.

The trace explains itself once you have it:

PreToolUse:Bash  hooks matched: 6
  [1] add_timeout.sh      rewrite: "pytest tests/unit -x"
                              ->  "timeout 300 pytest tests/unit -x"   exit=0  9ms
  [2] binary_allowlist.py head token = "timeout"  not in allowlist
                          stderr: Blocked: 'timeout' is not an approved binary (SEC-11)
                                                                       exit=2  6ms

Hook [1] was added in March 2023 after an agent left a pytest process running for six hours. Hook [2] was added in September 2024 after a review found the agent could invoke arbitrary binaries. Both were reviewed, both were correct with respect to the incident that produced them, and neither author knew the other hook existed. The failure needed no bug in either file.

Thirty hooks is a symptom, and the disease is not quantity#

The natural reading is that the platform accumulated too much, and the natural response is to consolidate. That reading gets the shape of the problem wrong in a way that leads to a partial fix being shipped as a complete one.

Every one of the thirty hooks was individually justified. Each traced to an incident, each had a reviewer, and thirty over two years is roughly one every three weeks, which is a reasonable rate for a platform running agents in production. What never happened even once in those two years was a statement of the form this invariant holds, it is decided at this boundary, and here is the evidence. The unit of work was always a hook, so the artefact was always a file, and a file records a mechanism rather than a claim. Two years of mechanisms produced a system whose behaviour is the composition of thirty predicates over a shared input, which is not a thing anyone designed and not a thing anyone can read.

The absence shows up in the specific pathologies chapter 8's probe run found. Nothing was ever deleted, because deleting a hook requires knowing what it was for and whether anything still needs it, and neither fact was written down. A hook whose matcher stopped matching after a tool rename stayed registered for eighteen months, because nothing asserted the claim it was supposed to support. An invariant in the security review document had no implementation at all, because the document and the hooks directory were never connected by anything but intent. Hooks [1] and [2] contradict each other because no artefact anywhere lists what is decided at PreToolUse:Bash and in what order.

What consolidation fixes, and what it leaves exactly as it was#

Consolidating into one policy engine — a single program holding every predicate as an ordered rule list and evaluating them in one pass, in place of thirty separate programs the harness launches — is a real improvement, and it is worth being precise about which of the two problems it addresses.

The latency is almost entirely fixed, because almost all of it is per-hook overhead rather than predicate evaluation. Instrumenting the fleet showed the 220 ms delta decomposing as roughly 7.2 ms of process spawn and interpreter startup per hook, 1.1 ms of JSON serialisation of the event to each hook's stdin, and about 0.4 ms of actual predicate work per hook for the twenty-six that do no I/O. Thirty processes at 7.2 ms is 216 ms, which is the entire increase. The team merged the twenty-six pure-predicate hooks into a single Python process and p50 went from 260 ms to 71 ms, with the remaining four I/O-bound hooks left as separate programs so that a slow dependency could not block the fast path.

The coherence is not fixed at all, and this is where the argument usually stops too early. Inside the engine, add_timeout and binary_allowlist are two rules over the same input, and their interaction is now governed by the order they appear in a list. That is more visible than before, which is worth something, and it is the same contradiction. A reviewer reading the rule list sees thirty entries and still cannot tell you whether the agent can drop a production database, because that question is about which effects are reachable through which paths and the list is organised by predicate. The engine also creates a new failure mode the thirty processes did not have, in that one rule raising an unhandled exception can take down evaluation for all of them, so the failure-direction question from chapter 7 now has one answer for thirty rules instead of thirty answers.

The artefact that was missing#

What the platform needed alongside its hooks is a registry whose entries are claims rather than programs. Each entry names an invariant, states it in a sentence someone outside the team can evaluate, and answers the questions the previous chapters raised: where it is decided, what it does when it cannot decide, which paths are covered, and when that was last verified.

- id: INV-TF-1
  statement: >
    No terraform apply against staging or production completes without a
    plan output attached to the change request, generated from the same
    configuration revision.
  owner: platform-infra
  decided_at: PreToolUse:Bash            # see also: CI gate on merge
  mechanism: local receipt (.terraform/plan-receipt.json), signature checked
  on_unknown: deny                        # fail-closed; receipt is local
  covered_paths: [ "terraform apply", "terragrunt apply" ]
  known_uncovered: [ "make deploy-infra wrapper", "console apply by a human" ]
  probe: probes/inv-tf-1.yaml
  last_verified: 2025-04-02
  incidents: [ INC-2211 ]

Five fields carry the weight, and each corresponds to a mistake this topic has already worked through. decided_at is the difference between a control and an alarm. on_unknown is the fail-open decision that was previously four characters of exception handling with no name. known_uncovered is the sentence that stops a README claiming more than the mechanism delivers, and it is the field that makes an entry honest rather than aspirational. probe connects the claim to the empirical method from chapter 8, and last_verified is what prevents the registry from becoming documentation, because an entry whose probe has not run in thirty days is displayed as unverified rather than as true.

Three rules follow from having the registry, and they are what actually changes day-to-day practice. No hook is merged without an invariant id, which forces the author to write the claim before the code and frequently reveals that the claim is a judgement rather than a predicate. Each invariant names the boundary where it is decided, and if more than one boundary is involved, all of them are listed with what each contributes, so the layered residency design from chapter 5 is expressible without pretending it is one hook. Any hook that no registry entry references is deleted. That last rule removed nine of the thirty in the first pass: four duplicated another hook's predicate, three enforced things nobody was willing to write down as a claim once asked, and two were the dead matchers.

What the discipline costs#

Adding a guard used to take twenty minutes and now takes half a day, because writing the statement, choosing the failure direction, enumerating paths and writing a probe is most of the work and the predicate was never the expensive part. That cost lands hardest at the worst moment, which is during an incident when somebody wants a block in place tonight. Refusing on process grounds is how a registry gets bypassed in its first month, so the workable arrangement is an explicit stopgap lane: a hook may ship without an entry if it carries an expiry date no more than fourteen days out, after which CI fails until it has an entry or is removed. That is the same shape as chapter 6's patch-with-an-expiry, applied to process rather than to code.

The requirement that each invariant name its decision point is artificial for a minority of rules that are genuinely cross-cutting, and forcing them into one row produces either a fiction or an entry with five boundaries listed and no clear owner. The registry handles this badly and the honest answer is that some claims are properties of an architecture rather than of an interception point, which means they belong in a design document with the registry entry pointing at it.

There is also a failure mode the registry invites, which is a team that maintains the table and stops thinking. An entry that says last_verified: 2025-04-02 against a probe that has quietly been in NOT_ATTEMPTED for a month is worse than no entry, because it carries the authority of a verified claim. The registry is only as good as the probes behind it, and the probes are only as good as their ability to actually induce the violating action.

Working from the registry inverts the order in which these things get built. The starting point is a list of the three or four outcomes that would be an incident if they happened once, written as sentences with an owner. For each one, decide whether a machine can decide it at all, and if it cannot, send it to review rather than to a hook. Then choose the boundary that can still prevent the effect, decide what happens when the answer is unknown, write the probe that attempts the violation, and only then write the predicate. The hook is the last artefact rather than the first, and by the time you write it the interesting decisions are already made, which is why it usually turns out to be about twenty lines.