A rule with a check behind it holds; prose holds about half the time

Broken out by convention, the 400-run comparison stops looking like a single 31-point improvement:

The two rules that end above 95% are the two with a check that runs inside the agent's loop. The three with nothing behind them land between 44% and 61%, and style-009 barely moves at all. err-001 is the interesting middle: it had a verifier written for the comparison but not wired into the repository's own test command, so the agent only encountered it if it happened to run the right file — and it still reached 88%, because the convention is concrete and the surrounding code mostly demonstrates it.

The mechanism behind the split is not that lint rules are more forceful. It is that a check changes the category of the thing. A sentence in a config file is an instruction, present once, near the start of a transcript that will grow to sixty thousand tokens. A failing check is an observation, arriving in the middle of the work, about the specific line the agent just wrote:

[agent runs: ruff check services/billing]
services/billing/api/refunds.py:44:9: T201 `print` found
[agent edits refunds.py]
services/billing/api/refunds.py:44:9: fixed

That exchange takes one turn and does not depend on the model remembering anything. Chapter 1 made the point that hook output and tool results land adjacent to the work; this is the payoff. The 97% is not compliance in the sense of obedience — it is a self-correcting loop, and the residual 3% is patches where the agent never ran the linter before finishing, which is a harness problem rather than a convention problem.

This also explains the difference between log-003 at 97% and err-001 at 88%, which is not a difference in how well the rules were written. ruff runs as part of the command the agent already invokes to check its work; the endpoint verifier existed as a file that nothing invoked. A check the loop never runs is a check that only affects human review, and its effect on the agent is limited to whatever the prose alongside it achieves. Wiring matters more than authorship, and the wiring is usually one line:

check: ## the single command the agent is told to run before finishing
	ruff check .
	python -m verifiers.endpoints services/
	import-linter --config .importlinter
	pytest -q

Naming that target in the conventions file, once, does more for compliance than any number of restatements of the individual rules, because it converts every check behind it into feedback the agent receives about its own work. A PostToolUse hook that runs the fast subset of make check after every edit is the same mechanism moved even closer to the work, at the cost of running the checks far more often.

What the paragraph actually buys#

The temptation, having seen that table, is to conclude that prose is decoration and only checks matter. Take the repository-layer rule as the test case, because it is the one where the argument is genuinely close. The convention is that every database query goes through the repository layer rather than touching the ORM session directly. It can be a paragraph in the conventions file, a lint rule that fails the build, or both.

The paragraph gives three things a lint rule structurally cannot. It generalises, because a rule stated with its reason — repositories own the tenant filter; a raw session query will silently return other customers' rows — extends to cases the checker was never written for, such as a new ORM helper added next month or a raw text() call routed through a connection pool the checker does not inspect. It names the right destination, so the agent writes InvoiceRepository.get_for_tenant(...) instead of inventing a private helper that satisfies the checker while defeating its purpose. And it is read by humans during onboarding, which is where a convention's real enforcement has always lived.

The lint rule gives three different things. It produces a number, so the compliance rate is a fact rather than a recollection. It produces a signal when compliance slips, which prose cannot do at all — a rule at 61% that drops to 40% after a model update looks identical from the outside until someone re-runs the comparison. And it costs zero tokens per request, forever, whereas the paragraph is on the bill of every session in the repository.

If only one ships this quarter, ship the check. The thing given up is real: the agent will route around the checker in ways nobody anticipated, and it will occasionally satisfy the letter of the rule with a helper that re-implements the query one directory away. That loss is bounded by the fact that a checker's blind spots become visible the first time someone reads a patch, whereas an unenforced paragraph's failures are invisible by construction — nobody knows the rule is at 61% because nobody measured it.

The version of the argument that goes the other way is worth stating precisely, because it is the one that sounds wiser in a design review: ship the prose, because it explains the reasoning and the model generalises from it, whereas a lint rule is brittle and will need exceptions. Both halves are true and the conclusion does not follow. The generalisation is worth having and it arrives with compliance somewhere in the middle and no instrument attached, and the exceptions a lint rule needs are precisely the conversations a team should be having explicitly rather than delegating to a model's judgement in the middle of an unrelated task.

The checker, and the exception it forces#

The rule is expressible as an AST check with a path condition, where AST is the abstract syntax tree — the parsed structure of a source file, which is what lets the checker ask whether an attribute is being read on a variable named session rather than grepping for a string that also appears in comments and docstrings:

"""Flake8 plugin: DB100 — ORM session use outside the repository layer."""
import ast

ALLOWED_PREFIXES = ("packages/persistence/repositories/",)
SESSION_ATTRS = {"query", "execute", "scalars", "add", "delete"}

class SessionUseChecker(ast.NodeVisitor):
    def __init__(self, path: str) -> None:
        self.path, self.problems = path, []

    def visit_Attribute(self, node: ast.Attribute) -> None:
        if node.attr in SESSION_ATTRS and isinstance(node.value, ast.Name) \
                and node.value.id in {"session", "db", "conn"}:
            self.problems.append(
                (node.lineno, node.col_offset,
                 f"DB100 direct session.{node.attr} outside the repository layer")
            )
        self.generic_visit(node)

def run(tree: ast.AST, filename: str):
    if filename.startswith(ALLOWED_PREFIXES):
        return
    checker = SessionUseChecker(filename)
    checker.visit(tree)
    yield from ((ln, col, msg, SessionUseChecker) for ln, col, msg in checker.problems)

For the import-boundary half of the same architecture, a purpose-built tool beats a hand-written visitor. import-linter expresses layer contracts declaratively, which is why dep-004 in the table starts at 94% even in the control arm — the rule was already enforced before anyone wrote it down for the agent, so the config file had nothing left to add.

Then there is migrations. Alembic revisions — the numbered scripts that carry the database schema forward one version at a time — legitimately execute raw SQL against the session, because they run before the models they would import exist and because a data backfill on four million rows through a repository would take hours. The exception is real and permanent.

The prose version handles it with a clause: …except in packages/persistence/migrations, where raw session access is expected. That clause has to be evaluated by the model, at the moment it is writing code, on the question of whether the file it is editing counts as migration-adjacent. Given that this family of rule holds about 61% of the time when it is unconditional, a conditional version has two failure modes rather than one: the agent applies the rule inside migrations and produces a revision that imports models it should not, or it decides that a backfill script in scripts/ is migration-like enough and writes raw SQL in application code. A carve-out evaluated by judgement is strictly harder than the rule it modifies.

The lint version handles it with an entry:

[flake8]
# DB100: migrations run before models exist and backfill in bulk.
# Owner: platform-data. Reviewed: 2026-03-11. Re-review if the repository
# layer gains a bulk path (see PERSIST-482).
per-file-ignores =
    packages/persistence/migrations/*:DB100
    scripts/backfill_legacy_invoices.py:DB100

That entry is dated, attributed, and greppable. The second line is the one that shows the mechanism earning its keep: a single legacy backfill script needed the exception, somebody had to write it down, and the act of writing it down put a name and a ticket next to it. Under the prose version that script's raw SQL would simply exist, indistinguishable from a violation, and nobody would ever be able to say which of the repository's raw queries were sanctioned.

Not everything can be checked, and some of it should still be said#

style-009 — prefer small, focused functions — moved from 49% to 54%, which on 200 runs per arm is inside the noise. It is not checkable in any useful sense; a cyclomatic-complexity threshold, which counts the branching paths through a function and fails anything above a limit, is a proxy that punishes a flat 40-line dispatch table and passes a deeply nested 12-line function. The honest read of that row is that the rule costs about 40 tokens on every request in the repository and buys an effect the comparison cannot distinguish from zero.

At roughly 4,000 agent sessions a month, 40 tokens is 160,000 tokens a month, which is loose change. The reason to remove it anyway is not the tokens; it is that a config file containing rules with no measurable effect trains everyone reading it that the file is a wishlist, and the next person adds their own untested preference on the same standing. Keep the unenforceable rules that carry a reason the model can act on — the tenant-filter explanation behind db-002 is worth its 120 tokens even at 61% — and drop the ones that express taste with no mechanism.

There is a category that genuinely cannot be checked and genuinely matters, which is anything about intent: when to introduce an abstraction, which errors are worth surfacing to a caller, what a module is for. Prose is the only carrier available for those, their compliance will be partial by nature, and the correct response is to expect a rate in the fifties rather than to keep rewriting the sentence.

A second uncheckable category is worth separating from the first, because it is the one where prose earns the most. Rules that exist to prevent a specific, remembered failure — the tenant filter, the egress allowlist, the analytics schema — carry a reason that transfers to situations no checker enumerates, and their value is not really their compliance rate at all. A rule at 61% that stops one cross-tenant leak in a year has paid for itself several thousand times over, and the compliance number understates it because the checker's own coverage gaps are precisely where the prose was needed.

That gives a usable test for what stays in the file. A rule survives if a check carries it and the check runs in the loop, or if it states a reason the agent can act on in cases the checker misses. It goes if it expresses a preference with no mechanism and no reason, which is where style-009 lands. Applying that test to a hundred-line file usually removes a third of it and improves nothing measurable in the short term, which is why it needs to be a rule rather than a judgement call made rule by rule in a review.

Every one of these rules, checked or not, is still text sitting in the request. The comparison in the previous chapter recorded three columns nobody has looked at yet — test outcomes, tokens, and latency — and the case for the whole apparatus depends on what they say.