A unified diff asks for arithmetic the model cannot check

Two weeks of telemetry from a coding agent that emits unified diffs, 412 attempted edits, and a failure pattern that nobody can explain by looking at the failures one at a time. The patches that fail apply look fine. The code in them is correct — a reviewer reading the rejected patch by hand agrees with the change and often merges it manually. What separates the successes from the failures is not the quality of the edit but the number of hunks in it, and within a failing patch, the position of the hunk that broke.

Underneath that pattern sits a choice every coding agent has to make, usually early and usually without much deliberation. A language model cannot reach into a file and change a line; it can only produce text. Something has to fix a convention for what that text looks like and for how a program on the other side turns it into a modified file on disk, and that convention is the edit format. It exists because the model's only output channel is a stream of characters, while the thing the task needs changed is a file with a precise existing content that mostly has to survive. Three formats dominate practice: return the whole file with the change in it, return a patch describing only the difference, or return the old text and the new text and let a program swap one for the other. In a real system the format shows up as a line of configuration and a paragraph of prompt — --edit-format diff when running aider, a str_replace tool schema in an agent SDK, or a hand-written instruction block and a parser in a home-grown loop. This topic is about how that one choice determines which failures the agent has, and how often.

The agent above used unified diff, the patch format that git diff prints and that git apply and the Unix patch command consume. A diff is a description of the difference between two versions of a file; unified names the layout, which interleaves the removed and added lines in one listing with a few unchanged lines of surrounding context rather than printing the two sides separately. A patch is divided into hunks, each hunk being one contiguous region of change together with its context, so a patch that alters three separate places in a file carries three hunks. Applying the patch means handing it to a program that reads each hunk, locates the region it describes in the file on disk, and rewrites that region. That program is what produced the numbers below: for each attempt it either rewrote the file, refused, or — the case that matters most in this chapter — rewrote the wrong part of it.

Across every multi-hunk patch that failed, the first hunk applied. That is 412 edits and not one case where hunk one was the problem in a patch with more than one hunk. Whatever is going wrong is not a property of the model's ability to write a hunk, because the first hunk demonstrates that ability every single time. It is a property of what the second hunk has to know about the first.

What the four numbers in a hunk header promise#

A unified diff hunk begins with a header that carries four integers:

@@ -142,7 +146,9 @@ def charge_invoice(invoice, gateway):
     if invoice.status != "open":
         raise InvalidState(invoice.id)
-    amount = invoice.total
+    amount = invoice.total_with_tax()
+    if amount <= 0:
+        return Receipt.zero(invoice.id)
     result = gateway.charge(invoice.customer, amount)
     invoice.mark_paid(result.id)
     return result

The -142,7 says this hunk covers seven lines of the original file starting at line 142. The +146,9 says the corresponding region of the result file will be nine lines starting at line 146. Those numbers are not decoration. git apply reads the old-side start as the location to look at, and it checks the counts: the number of lines in the hunk body marked with a space or a minus must equal 7, and the number marked with a space or a plus must equal 9. Get either count wrong by one and the patch is rejected before a single byte of the file is touched, with corrupt patch at line N.

The old-side start line has a property that matters enormously here. It is stated against the original, pre-patch file, and so is the old-side start of every subsequent hunk. This is exactly what diff -u produces, because diff computes both files in full before writing anything and can therefore afford to number the old side against the old file and the new side against the new file. The +146 in the header above is 142 plus the cumulative net line delta of every hunk earlier in this patch — in this case, hunk one added four lines somewhere above line 142.

A model generating this patch token by token has to reproduce both of those conventions from nothing. It must keep the old-side numbers pinned to a file it is in the middle of describing changes to, and it must keep a running sum of its own net line deltas for the new side, and it must count the context lines in each hunk body correctly for the length fields. Three separate arithmetic obligations, all of them stated in the header, which by convention comes before the body it is counting.

The drift, on an actual file#

Here is a 210-line billing module and a three-hunk patch against it. The correct patch, produced by diff -u after a human made all three edits:

--- a/billing/charge.py
+++ b/billing/charge.py
@@ -38,6 +38,8 @@ class Receipt:
     amount: int
     currency: str
+    tax_amount: int = 0
+    gateway_ref: str | None = None
 
@@ -142,7 +144,9 @@ def charge_invoice(invoice, gateway):
     if invoice.status != "open":
-    amount = invoice.total
+    amount = invoice.total_with_tax()
+    if amount <= 0:
+        return Receipt.zero(invoice.id)
@@ -197,4 +201,5 @@ def refund(invoice, gateway):
-    return gateway.refund(invoice.charge_id)
+    ref = gateway.refund(invoice.charge_id)
+    return Receipt.from_refund(invoice, ref)

And here is the header line the model emitted for hunk three in the run that failed:

@@ -199,4 +203,5 @@ def refund(invoice, gateway):

The body was byte-identical to the correct version. The old-side start was 199 rather than 197 — a drift of exactly two, which is the net number of lines hunk one added. The model numbered the third hunk against the file as it would exist after hunk one had been applied, which is the intuitive thing to do when you are walking down a file describing changes in order, and which is wrong under the unified diff convention. Hunk two happened to survive because 144 landed within the applier's tolerance of the real location; hunk three did not.

The tolerance is where this stops being a clean error and starts being a silent one. GNU patch does not treat the line number as authoritative. It uses it as a starting guess and searches outward for a region whose context lines match, and it will report success:

$ patch -p1 < agent.patch
patching file billing/charge.py
Hunk #1 succeeded at 38.
Hunk #2 succeeded at 142 (offset -2 lines).
Hunk #3 succeeded at 214 (offset 15 lines).
$ echo $?
0

Exit code zero. Hunk three found a place fifteen lines away from where it said it would go, because refund and void_charge in this module share a three-line tail — return gateway.refund(...) versus return gateway.void(...) differing on one line the fuzz factor was willing to discard. Fuzz is the applier's tolerance setting, the number of context lines it is permitted to ignore when deciding that a region matches a hunk closely enough, and patch allows up to two by default. The patch applied. The wrong function got the change. git apply is stricter by default and would have rejected this, which is why the choice of applier is part of the format decision and not an implementation detail: patch -F2 and git apply disagree about the same bytes, one producing a loud retry and the other producing a defect that reaches review looking like a clean apply.

Why the model has no way to catch this itself#

The arithmetic is not hard. Summing three integers is not a reasoning task that defeats a frontier model — one of the largest and most capable models available at a given moment — in isolation. What defeats it is that the arithmetic is over a quantity the model cannot observe reliably and cannot re-derive cheaply.

Line numbers are not something a language model perceives. It sees a token stream in which a newline is one token among many, and counting to 142 requires either a positional aid in the input or a genuine count. If the file was presented in the prompt without line numbers, the model is estimating. If it was presented with line numbers, the model can read the start line for hunk one accurately and then must still compute the rest by addition against its own not-yet-emitted output. There is no register holding "net lines added so far", only whatever the attention pattern reconstructs from the text it has already produced.

The obligation also runs backwards through the generation order. The header states the length of a body that has not been written yet, so the counts are a prediction about the model's own future output. When the body then comes out one context line longer than planned — which is a completely ordinary thing for a generative process to do — the header is retroactively false and nothing in the loop notices, because the model does not re-read and correct its own header.

Nothing inside generation supplies a signal either. The model gets no feedback until the patch reaches an applier, which is after the response is finished, after the tokens are paid for, and after the turn has ended. Compare that to the code inside the hunk, where a wrong variable name will be caught by the type checker in eight seconds and fed back. The hunk header is metadata about the edit rather than the edit itself, and none of the verifiers that make code a good agent domain have jurisdiction over it. No compiler checks a patch header. The patch is either accepted, rejected, or misapplied, and only the middle case is loud.

The compounding follows from the structure. Each hunk's correctness depends on the cumulative delta of all hunks above it, so an error in hunk one poisons every header below it whether or not the model makes any further mistakes, and independent errors add rather than cancel. A patch with k hunks has roughly k opportunities for the drift to exceed the applier's search window, which is what the table at the top of this chapter is measuring.

Two of the three arithmetic obligations can be handed back to the tooling, which is worth knowing before anyone spends a week on prompt engineering. git apply --recount instructs git to ignore the length fields in the header and recompute them from the hunk body, so a miscounted ,7 stops being fatal. patch has always been willing to search for the right location, and git apply -C1 reduces the number of context lines that must match, while --3way falls back to a merge using blob contents when the direct application fails. Each of these buys back some of the fragility, and none of them touches the start-line drift, because none of them can distinguish a hunk that landed fifteen lines late from a hunk that was meant to be there.

The context lines are a second exact-match requirement

The line numbers get the attention, and the unchanged lines carry an obligation just as strict. Every line in a hunk body prefixed with a space is a claim that the file contains exactly that text at that position, and the applier compares it byte for byte. A model that reproduces result = gateway.charge(invoice.customer, amount) as result = gateway.charge(invoice.customer, amount) # charge — adding a helpful comment to a context line, which happens more often than it should — has stated something false about the file, and the hunk is rejected with error: patch failed. The same goes for a context line the model reflows to a different width, or one whose indentation it normalises.

That obligation survives every fix to the numbering. A format that removes hunk headers entirely still needs the context lines to match, so the model is still copying text it saw thousands of tokens ago. Removing the arithmetic makes the format substantially more reliable and leaves a residue of failures that have nothing to do with counting, and that residue turns out to be the dominant failure mode of the format two chapters from now.

What this failure is not#

It is not the model being bad at writing code. Every rejected patch in that two-week sample contained a change a reviewer would have accepted. The defect lives entirely in the coordinate system, and separating those two things matters because the obvious remedies for bad code — a stronger model, a lower temperature (the sampling setting that controls how much randomness enters generation), more context, better instructions — do very little for an addressing problem. Teams often discover this by upgrading the model and watching the multi-hunk apply rate move by two points.

It is also not an argument that unified diff is a bad format in general. It is an excellent format, and git produces millions of them a day without error, because git derives the numbers mechanically from two files it holds in memory. The format was designed in 1990 for a producer that can count. The mismatch is between that assumption and a producer that generates left to right and cannot verify what it wrote.

Nor is it a token-cost problem. A three-hunk patch against a 210-line file is perhaps 300 tokens of output where a rewrite would be 2,400, and the diff is cheaper by any measure. Cost is the argument for the format, and it is not the axis on which it is failing.

The remedy the field converged on is to keep the diff's shape and delete the arithmetic. Aider's unified-diff format omits line numbers from the hunk headers entirely and matches hunks by their context alone; the write-up "Unified diffs make GPT-4 Turbo 3X less lazy" reports the effect that had on both laziness and apply rate. That points at the real question, which is what an edit format uses to say where, and how much of that the model is expected to compute rather than copy.

There is a blunter response available, and plenty of teams reach for it first: if the coordinates are the problem, emit no coordinates. Have the model return the entire file, and let the write be a straight overwrite with nothing to apply and nothing to locate. That format has exactly zero apply failures, which is a real property and a genuinely attractive one. The next chapter looks at what it costs, starting with a 1,400-line module that came back ending in the middle of a function with no error anywhere in the loop.