Chapter 5 of 6 · 9 min read
An update with no cohort reads as containment and works as concealment
On this page
It is 06:20, four hours and six minutes into the incident. The obligations extract has been disabled since 02:30, the review pane is still serving, the eval score on the contract-summary slice is holding at 60 against a baseline of 71, and two enterprise customers have written in — one about a summary that omitted a renewal date, one asking why an automated field stopped populating. The team's best current theory is an upstream model change, because the resolved model identifier moved when the bundle was promoted and the retrieval revert did not recover the score on its own. A support ticket to the provider was opened at 03:55 and has not been answered. Nobody knows the cause.
The CEO, who has been reading the incident channel, writes: "Can we get something out to customers? Keep it short — I don't want to alarm anyone unnecessarily. Maybe just that some users may have seen reduced quality and we're on it."
That sentence is going to get written by someone in the next twenty minutes, so the question is what it should contain. The proposed version has a specific and predictable set of consequences, and they are worth working out before arguing about tone.
What the reassuring version costs
"Some users may have seen reduced quality. We're investigating and will update soon." Three clauses, and each one removes a piece of information a customer needs.
Some users names no cohort, so every one of the 340 accounts on the platform now has to consider whether it is one of them. The company's actual exposure is much narrower: the automated obligations path serves 12 accounts, and the degradation concentrates on contracts over about 40 pages, which affects roughly 260 records written since 21:41 Monday. A notice that cannot distinguish 12 accounts from 340 has converted a small, specific problem into a large, vague one — in the reader's mind and in the support queue. At a 15% contact rate, 340 uncertain accounts produce around 50 tickets, each of which is a human asking "are we affected?", which is a question the notice could have answered.
May have seen gives no timeframe, so nobody can check their own outputs. A customer told that the window was Monday 21:41 UTC onward can pull their own records from that window and look; a customer told "recently" can only ask you. Every omitted specific converts a self-service check into an inbound request, and the aggregate of those requests is the reason the support team spends the incident answering the same question fifty times instead of helping the twelve accounts that are actually affected.
Will update soon sets no next time, which quietly hands your disclosure schedule to your provider. The unstated plan behind "soon" is to wait for the provider's support ticket, and that ticket is answered when they answer it. A commitment you cannot keep because a third party has not replied is a commitment you should not have made; a commitment to send another message at 14:00 UTC whether or not anything is known is one you can always keep.
The word for the resulting message is not "cautious". It reads as containment and functions as concealment, and the tell is that a customer who later learns the specifics — the window, the cohort, the 260 records — will reread the 06:20 note and conclude that you knew and chose not to say. That inference will be correct.
What the update has to carry
Five elements, and they map to the questions a customer will otherwise ask you one at a time.
| Element | The vague version | The usable version | What it lets the customer do |
|---|---|---|---|
| Cohort and behaviour | "some users", "reduced quality" | Accounts using the automated obligations extract; renewal dates omitted or wrong on contracts over ~40 pages | Decide in ten seconds whether this is their problem |
| Window | "recently" | From 21:41 UTC on 22 June, ongoing | Pull their own records and check |
| Established vs suspected | Merged into one confident sentence | "Established: the drop, the window, the affected slice. Suspected, unconfirmed: an upstream model change" | Judge how much to trust the next update |
| Containment and impact now | Absent | Automated writes paused since 02:30 UTC; summaries still served, flagged as unverified | Know whether damage is still accruing |
| Next update | "soon" | 14:00 UTC, with or without a resolution | Stop checking, and stop asking |
The third row is where most drafts go wrong in one of two directions. Smoothing the uncertainty away — writing "we have identified the issue and are deploying a fix" when you have not and are not — buys four hours and costs the credibility of every subsequent update. Publishing the suspicion as an accusation is the opposite error: telling customers that your model provider shipped a bad update, four hours in, on the evidence of one identifier change and a failed revert, is a claim about a third party that you cannot support, may be wrong about, and will have to retract in public. State the shape of the suspicion without assigning it a name and a verdict, and say explicitly what would confirm it.
Two other things stay out. Any timeline that depends on a third party — "we expect a fix from our provider later today" — is a promise you cannot keep, and it will be quoted back to you at 18:00 when it has not arrived; say when you will next speak instead, which is entirely within your control. And leave the liability language alone. "We're sorry for the disruption" is ordinary courtesy, while sentences that characterise fault, waive terms, or promise remediation beyond what the contract requires are commitments, and those belong to whoever owns the contracts rather than to the engineer drafting at 06:20. Writing the facts precisely and letting the commercial owner add the commitments keeps both jobs clean and neither slow.
The other thing to keep out is the internal metric. "Our evaluation score dropped 11 points" means nothing to a reader who does not know the scale, the suite, or the noise floor, and it invites either panic or dismissal depending on how they guess. Translate the measurement into the behaviour it measures: renewal dates omitted in roughly one summary in six on long contracts. The score is how you know; the behaviour is what they experience.
The message
Subject: Degraded contract summaries affecting automated obligations
(incident INC-2026-0623)
Who is affected
Accounts using the automated obligations extract. Within those accounts,
contracts longer than roughly 40 pages are affected most: renewal dates
and notice periods are omitted or incorrect in about one summary in six.
Summaries shown in the review pane are affected to a lesser degree.
If you do not use the obligations extract, you are not affected.
When
From 21:41 UTC on Monday 22 June. Still ongoing as of 06:20 UTC today.
What we have established
- Summary quality on long contracts dropped measurably at 21:41 UTC.
- 312 obligations records were written from degraded summaries in that
window. We have the exact record IDs and will send your list by 12:00 UTC.
- Review-pane summaries are degraded but usable alongside the source
document.
What we suspect and have not confirmed
A change in the behaviour of an upstream model we depend on. We have not
confirmed this and are not yet certain it is the cause. We expect to know
by 14:00 UTC.
What we have done
- Paused all automated writes to your contract records at 02:30 UTC today.
No further records have been written since.
- Summaries in the review pane are now flagged as unverified.
What we suggest you do
- Do not action renewal reminders generated after 21:41 UTC Monday until
you have the record list from us.
- Continue using the review pane as normal, checking summaries against the
source document as you would any draft.
Next update
14:00 UTC today, whether or not we have a resolution.
Reply to this address or contact your account manager at any time.Nothing in that message claims a cause, and nobody reading it will think the company is hiding one. It is also shorter than the meeting that would have been spent debating the vague version.
One message is rarely the right shape, because the audiences need different things. The twelve accounts with automated writes need a direct email from a named human, addressed to the person who owns the workflow rather than the billing contact, because they have work to do — reminders to hold, records to check. The other 328 accounts need something they can find if they go looking and safely ignore otherwise, which is a status-page entry — the public log of current and past incidents that a customer can check without writing to you — carrying the same facts and no inbox interruption. Users currently in the product need a banner on the review pane telling them summaries are flagged as unverified, because that is the fastest way to change behaviour at the moment it matters. Sending the enterprise email to all 340 accounts is how you teach people to stop reading your incident mail; posting only to the status page is how the twelve affected accounts find out from their own auditor.
The message needs an internal twin, published at the same moment. Support will get the first replies within minutes, and if the only artefact is the customer email they will improvise answers that contradict each other. The internal brief carries the same five elements plus the things customers do not get: the record list, the current hypothesis and its status, which containment flags are set and who can flip them, agreed wording for the three questions that will certainly arrive — am I affected, is my data safe, when will it be fixed — and an escalation path for anything outside those three. Fifteen minutes of writing spares the commander four interruptions in the next hour.
Two delivery details are worth getting right. The message comes from a person with a role, at an address a human reads, rather than from a no-reply address, because the reply is the cheapest way to learn about a customer you failed to classify. And it goes to the workflow owner named on the account with the account manager copied, since whoever needs to hold the renewal reminders is rarely the signatory on the contract.
Handling the framing without turning it into a fight
The CEO's stated goal is that customers not be alarmed unnecessarily, and that goal is correct. The disagreement is about which message produces less alarm, which is an empirical question rather than a values one, so answer it in those terms: a notice naming 12 accounts leaves 328 accounts calm, and a notice naming none leaves 340 accounts uncertain. Vagueness does not reduce alarm; it redistributes it onto people who were fine.
Two structural moves make this conversation short. The first is to bring a draft rather than an argument — the message above, in the channel, with an offer to cut anything that is not load-bearing. Editing a concrete artefact is a different conversation from litigating a principle, and it usually ends with the CEO improving two sentences and approving it. The second is to separate the roles, which is the part of standard incident structure most worth borrowing here: The Site Reliability Workbook's incident response chapter splits command, communications and operations precisely so that the person deciding what is true is not also the person managing the relationship. The commander owns the accuracy of the message; the communications lead owns its tone and delivery; the executive owns the commercial risk of what is being disclosed and is the right person to sign it. Saying "you decide whether we send it, I decide what it says is true" is a clean split that almost nobody objects to.
What does not work is arguing about adjectives in the incident channel while the clock runs. If the disagreement survives one round of edits, it has stopped being an engineering decision and needs a named owner — usually whoever holds the customer contracts, since the obligation to notify lives in them.
The decision the hour is actually for
The message buys time, and time is only worth buying if it is spent on a decision. The one on the table at 06:20 is whether to pin the last green manifest — chapter 3's rel_20260609_0330 — and give up the retrieval improvements the bundle shipped, or keep the current configuration with the extract disabled while waiting for the provider.
The value of writing the criterion down before the hour starts is that it stops the decision from being made by fatigue. In the channel, before 07:00:
Decision point at 14:00 UTC. We pin
rel_20260609_0330if any of these hold: the provider has not responded; the contract-summary slice is still below 66 on the 09:00 and 13:00 eval runs; or the number of affected records grows past 350. We hold the current configuration only if we have a confirmed upstream cause with a stated remediation time under 6 hours. Decision owner: incident commander. Nobody needs to be woken up for this.
Every clause there is checkable by someone who was not in the room. The trigger is a number, the deadline is a clock time, the owner is a role, and the default — pin — is the safe action rather than the interesting one, so the decision only stays open if positive evidence keeps it open. That is what turns an incident from a vigil into a procedure.
At 08:12, before any of those triggers fire, the 08:00 eval run comes back at 70.9. Nothing was changed. The next chapter is about why that is not good news.
Incident response for AI systems — read the current version at https://daviddominguez.dev/learn/en/incident-response-for-ai-systems/an-update-with-no-cohort-reads-as-containment-and-works-as-concealment