A description that says what a skill contains never matches a request

The db-migrations skill went in on a Tuesday and by the following Friday week it had loaded zero times. In the same period the agent had handled twenty-six requests that touched db/migrate/, including three that added columns without a rollback file and one that put a backfill in the same statement as an ALTER TABLE, which is the exact thing the skill exists to prevent. Nobody noticed for a while, because the agent answered every one of those requests confidently and the answers looked fine.

The frontmatter that shipped was not the version in the previous chapter. It was this:

---
name: db-migrations
description: Contains the team's database migration standards, including naming conventions, backfill policy and rollback expectations.
---

Everything in that sentence is true, and it is a description of the document rather than of the occasion for reading it. The requests it was meant to serve looked like "add a nullable deleted_at to users", "make org_id optional on invoices", "we need an index on events.created_at, can you write the migration", and "why did that ALTER lock the table for so long". Read the description against those, word by word, and there is nothing in it that resembles adding a column, altering a table, creating an index, or writing a migration at all. The words that would have matched — the verbs people actually type — are missing, and the words that are there describe categories of content that appear in nobody's request.

The description is the routing key, and it is matched against phrasing#

At the point where the runtime decides whether to load a skill — the runtime being the harness your agent runs inside, the code that assembles each request to the model and chooses what text goes into it — it has the user's message and a list of skill names and descriptions. The body is not in the decision; the reference files certainly are not. Whatever your document says about CREATE INDEX CONCURRENTLY, the choice to bring that document into context was made from one sentence, and the sentence is competing against how a hurried engineer phrased a request at 16:40 on a Thursday.

That makes the description a different genre of writing from the rest of the skill. The body is written for a reader who has already arrived; the description is written to be recognised by someone who has not. What recognises well is a list of situations, in the vocabulary those situations get described in:

---
name: db-migrations
description: Use when writing, reviewing, or applying a database migration — adding or altering a column, making a column nullable or NOT NULL, creating or dropping an index, backfilling a table, or writing a rollback. Also use when a migration has locked a table or run long in production.
---

The second version is longer, which is fine — descriptions of a couple of hundred characters are normal and the cost of the extra words is trivial next to the cost of a skill that never loads. What it added is not detail about the content. It added the verbs and the nouns of the request: add a column, nullable, index, backfill, rollback, locked a table. A skill description earns its length by naming occasions.

Guessing at those occasions is the part people skip#

The temptation once the defect is understood is to sit down and write the list of triggering situations from memory. That produces a description that fires on the requests you imagined, which are the requests you would write if you were the user, and you are not the user. The requests that were actually missed are sitting in the transcript store, and they cost an afternoon to extract.

The mechanical version of this is unglamorous. Take four weeks of sessions, filter to the ones that ended up touching migration files or running the migration tool, sample forty of them, and record the first user message from each verbatim. Then sample twenty sessions that a keyword filter caught but that turned out to be something else — schema questions, ORM model changes, a query that was slow for reasons unrelated to a migration — because those are the near-misses, and a description that cannot tell them apart is the other failure mode arriving.

{"id": 1, "text": "add a nullable deleted_at to users", "should_fire": true}
{"id": 2, "text": "make org_id optional on invoices", "should_fire": true}
{"id": 3, "text": "we need an index on events.created_at, write the migration", "should_fire": true}
{"id": 4, "text": "why did that ALTER lock the table for 11 minutes", "should_fire": true}
{"id": 5, "text": "backfill plan_tier from the subscriptions table", "should_fire": true}
{"id": 6, "text": "what type should we use for a currency amount", "should_fire": false}
{"id": 7, "text": "add a plan_tier field to the Invoice model", "should_fire": false}
{"id": 8, "text": "this query on events is slow, can you look at the plan", "should_fire": false}
{"id": 9, "text": "restore last night's staging dump", "should_fire": false}

Case 7 is the interesting one, and it is the reason a labelled set beats a discussion. Adding a field to an ORM model is usually a migration in disguise, so a reasonable person could label it either way; what matters is that the team writes down which it is and then measures against that, rather than discovering the disagreement six weeks later when someone complains the skill is noisy.

The opposite failure draws the boundary around a subject#

The incident-response skill on the same team had the mirror-image defect. Its description read "Use for anything relating to production incidents", and it worked beautifully in testing, because the author tested it by saying "walk me through our incident response process". In real use it fired on requests to summarise the #inc-4471 Slack channel and imposed a blameless-postmortem structure — impact statement, timeline, contributing factors, action items with owners — on what somebody wanted as four bullet points before a meeting.

The boundary there is drawn around a subject rather than an action. "Anything relating to production incidents" covers running an incident, writing the postmortem, summarising a channel, looking up a severity definition, and asking who was on call in March, and the skill has useful things to say about exactly one of those. A description that names a subject inherits every request that mentions the subject, which is a large and growing set.

The repair has two halves. The first is naming the single task: "Use when running or coordinating an active production incident — declaring severity, paging the right rota, opening the incident channel, posting status updates." The second is naming the neighbours explicitly, because in a crowded skill directory the description is doing negative work as well as positive: "Not for writing the postmortem after the fact (see incident-postmortem), and not for summarising or searching incident channels." Exclusions feel redundant when you write them and they are the reason the summarise request stops matching.

Settling the trade with numbers rather than argument#

Widening a description to catch the requests it was missing pulls in neighbours, and tightening it to shed the neighbours loses requests again. There is no formulation that makes that tension disappear, which is why the labelled set is the deliverable and the description is just the thing you iterate against it. Forty-one requests that should fire, twelve near-misses that should not, three versions of the description:

Between v2 and v3 the team gave up one true trigger to remove six false ones, and the reason that was the right trade is specific rather than general: a missed load means the engineer gets an unaided answer — the answer the agent would have produced with no skill in context at all, from its own general knowledge — which is the status quo they were living with anyway, while a false load means the agent silently restructures work someone did not ask to have restructured. Where the false-positive cost is lower — a skill whose only effect is to remind the agent of a naming convention — the trade goes the other way, and you can only tell which case you are in by knowing what the skill does when it loads.

The three-line table is also the artefact to keep. Six months later someone will want to widen the description again for a request it missed, and the twelve near-misses are what tells them what widening will cost.

The fix that looks free and is not#

There is a shortcut that will be proposed the first time a skill under-fires, and it is worth naming because it works in the demo. Rather than fixing the description, add a line to the always-loaded configuration: whenever the user mentions production, check the incident-response skill. The next session, the skill fires. The problem is solved in the sense that this particular skill now loads.

What it costs shows up over quarters. That line is resident on every turn of every session, paying for the guidance whether or not anyone is near an incident, which is the resident-versus-on-demand trade 12.1 laid out — except that here you are paying it not for the skill's content but for a pointer to the skill. The next under-triggering skill earns its own line, and the one after that, until the configuration is a directory of skills the agent is being told to remember, which is the job the descriptions were already doing and doing badly. The original defect is still there, untouched, in a description that names no situations; you have added a second mechanism to compensate for the first one being wrong.

What a good description still cannot do#

Getting the routing right does not make the skill correct. A description is a claim about when the document applies and says nothing about whether the document is any good, and the two failures look completely different in a transcript: a routing failure shows up as an unaided answer, while a bad body shows up as a confident answer with the wrong procedure in it.

Nor does a good description guarantee a load. Matching is a judgement made by a model under a context budget with other skills competing, and a request that plainly should have fired sometimes does not. Descriptions move the rate; they do not produce a switch. The right expectation is that a well-scoped skill fires on the large majority of the requests you labelled, and that the ones it misses are visible to you because you have the labelled set to check against.

With the skill now loading on the right requests, the question becomes what the agent does differently when it does. The db-migrations body from the previous chapter reads well and is not obviously verifiable: two engineers who both read it could disagree about whether a given run followed it, and neither could point at the sentence that settles the argument. That is the subject of the next chapter.