Choosing tool granularity

Two engineers on the same team submit two designs for the same analytics agent, which answers questions in a Slack channel that the finance and sales leads live in. The first design has one tool, pointed at a read replica, which is a copy of the production database kept continuously up to date and used for queries that only read.

{
  "name": "run_query",
  "description": "Run a read-only SQL query against the analytics replica. "
                 "Schema: customers, invoices, subscriptions, events, employees.",
  "input_schema": {
    "type": "object",
    "properties": {"sql": {"type": "string"}},
    "required": ["sql"]
  }
}

The second has five, and no free-form SQL anywhere in it.

[
 {"name": "revenue_by_period",
  "input_schema": {"start": "date", "end": "date",
                   "granularity": {"enum": ["day", "week", "month", "quarter"]},
                   "segment": {"enum": ["all", "new", "expansion", "self_serve"],
                               "default": "all"}}},
 {"name": "top_customers",
  "input_schema": {"metric": {"enum": ["revenue", "seats", "usage"]},
                   "period": {"enum": ["mtd", "qtd", "ytd", "last_12m"]},
                   "limit": {"type": "integer", "maximum": 50, "default": 10}}},
 {"name": "churn_rate",
  "input_schema": {"period": {"enum": ["mtd", "qtd", "ytd", "last_12m"]},
                   "basis": {"enum": ["logo", "revenue"], "default": "revenue"}}},
 {"name": "invoice_status",
  "input_schema": {"customer": "string", "state":
                   {"enum": ["open", "paid", "overdue", "any"], "default": "any"}}},
 {"name": "headcount",
  "input_schema": {"as_of": "date", "by": {"enum": ["department", "location", "total"]}}}
]

The team's month of Slack history has 340 questions in it. Somebody classified them: 71% are answerable by one of the five typed tools, and 29% are not — questions like what's the revenue trend for accounts that expanded last quarter but haven't renewed yet, which is a join across three tables with a condition nobody anticipated. That 29% is the entire case for the first design, and it is a strong one, because the questions in the tail are the ones people actually get excited about.

Those shares are worth converting into counts before either side is argued, because a percentage of a month hides how thin the tail is. Seven questions in ten of three hundred and forty is a bit over two hundred and forty that the typed tools answer as they stand, and the tail is a little under a hundred questions in a month — two or three working days if a person sat down and did them by hand. That is small enough to be servable by a human and large enough that nobody in the channel would enjoy waiting for one. So the argument cannot be settled on volume, and has to be settled on what each design does when it is wrong.

Expressiveness has a price and the price is the failure mode#

The tail is real and the argument still loses on its own, because it prices the gain and not the cost. Consider what the model must hold in its head to write correct SQL against a warehouse: which of invoices.amount, invoices.amount_due and invoices.total_incl_tax is the one finance means by revenue; that subscriptions has soft deletes so every query needs deleted_at IS NULL; that events is partitioned by day and an unbounded scan takes four minutes; that internal test accounts are flagged is_internal and every published number excludes them; that the replica lags the primary by about ninety seconds.

None of that is in the schema. All of it is required for a correct answer, and every one of those facts is a place where a query can be syntactically perfect, semantically plausible, and wrong. A team that ran this configuration for a quarter audited 120 answered questions by hand and found 9 with wrong numbers, none of which had produced an error of any kind. Three had included internal accounts. Two had used gross rather than net revenue. Two had double-counted rows across a fan-out join, where joining a table that holds several rows per record multiplies the rows on the other side and every sum taken afterwards comes out inflated. Two had silently missed the last day of the month because of a timezone boundary in the date filter. Every one of those answers was posted into a Slack channel where a VP read it.

Nine in a hundred and twenty is roughly one answer in thirteen, and it is a floor rather than an estimate. The audit could only find errors a human re-deriving the number by hand was able to find, so every wrong answer that happened to look wrong is in the count and every wrong answer that looked exactly as plausible as a right one may not be. Two of those four categories — the timezone boundary and the fan-out join — were caught because the person auditing knew the warehouse well enough to go looking for them. Nobody knows the rate for a category nobody thought to check, and the structure of this failure guarantees that the unchecked categories are the quiet ones.

That is the structural point about a broad tool: there is nothing to validate against. Any syntactically valid query is an acceptable call, so the schema validation that catches a bad enum value or a missing argument in the five-tool design has no equivalent. The failure mode moves from an error your harness counts to a number a human believes.

The reason is worth stating in the schema's own terms rather than as a slogan. A JSON Schema constrains the shape of a value: this key is present, this value is a string, this string is one of four listed words. Every check it can perform is one that can be performed on the argument alone, before anything runs, which is what makes validation cheap. Design A declares sql to be a string. Every query the warehouse would happily execute and every query that would return a wrong number are equally strings, so the declaration is already satisfied the instant the model has produced anything at all. To reject a wrong query you would have to know what the right answer was, and the right answer is the thing you asked the tool for. The enum in the five-tool design escapes this only because somebody moved the knowledge out of the query and into the list of permitted values: the check is cheap at call time because the decision was made in advance, in code, once.

The typed tools invert this. revenue_by_period(start, end, granularity, segment) cannot exclude the internal-accounts filter, because the filter is in the implementation. It cannot pick the wrong revenue column, use the wrong timezone, or scan an unpartitioned range, because a human decided all of that once, in code, under review, with a test. When it fails it fails loudly: an unsupported granularity is rejected before execution, and a question outside the five tools produces no call at all rather than a confident wrong one.

The difference is easiest to see as a question of where a wrong call can be stopped, since design A has exactly one gate in it and the gate only checks grammar.

Cardinality is the second place the broad tool fails

Wrong numbers are the failure the hand audit found, and there is a second one that shows up in the harness rather than in a VP's Slack thread. A free-form query has an unbounded result set, and the model has no way to know the cardinality of what it asked for until the rows arrive in its context.

run_query("SELECT customer_id, invoiced_at, amount
           FROM invoices WHERE invoiced_at >= '2026-01-01'")

-> 12,431 rows, 2.1 MB serialised
   harness truncated to the first 200 rows (context guard)
   model reply: "Total invoiced since January: $487,200."

The true figure was $31.4 million. The model summed the two hundred rows it could see, and the truncation note the harness appended read as an aside rather than as an invalidation of the arithmetic. On this team's traffic, 14% of run_query calls returned more than 1,000 rows and 4% returned more than 10,000, so the condition is not exotic; it is a monthly occurrence with a large error attached and no mechanism that would catch it.

The arithmetic in that answer was not wrong, which is the part worth sitting with. Twelve thousand rows became two hundred, so about one row in sixty survived, and scaling the visible sum back up by that same factor lands within a few percent of the true total. The model added correctly over everything it could see. What it could not see was that the denominator had moved underneath it, because a truncated result set is not marked as truncated inside the rows: the two hundred rows look exactly like a complete answer to a narrower question, and there is no field in them that would say otherwise. This is not an arithmetic slip and no amount of care with sums prevents it. It is a missing fact about the provenance of the rows, arriving in a channel — a note appended by the harness — that carries none of the weight the rows themselves do.

Each of the available mitigations gives back some of what made design A attractive. Injecting a mandatory LIMIT inside the tool bounds the payload and reproduces the same silent truncation one layer down. Running EXPLAIN first and refusing anything whose estimated row count exceeds a threshold turns the failure into an error the model can react to, at the cost of a second round trip and an estimate that is unreliable on skewed data. Returning a summary alongside the rows — count, min, max, and the sum of each numeric column — is the cheapest of the three and removes the incentive to count by hand, which is the actual mechanism of the failure. A typed tool has none of this to decide, because revenue_by_period returns one row per period by construction and its cardinality is known before the query runs.

The defence offered for design A at this point is that an analyst writing the same query would make the same mistakes, so holding the model to a higher standard is unfair. It is tempting because it is half true — analysts do forget the internal-accounts filter — and it misses the three places the two situations differ. An analyst who makes that mistake once carries the correction forward, whereas the model re-derives the same five unwritten facts from scratch on every question with no memory of the last time it got them wrong. An analyst's query is read by somebody, usually the analyst, before the number is quoted. And an analyst who is unsure asks. The typed tools are not competing against a careful analyst; they are competing against a component whose confidence is the same whether or not it happens to hold the facts, which is the property that makes an audit necessary rather than optional.

The five things that decide it#

Expressiveness is one dimension out of five, and it is rarely the one that decides.

Permission scope is the row that usually settles it in a company of any size. headcount(as_of, by) can be exposed to the finance lead and withheld from the sales channel; run_query cannot be, because a read replica containing the employees table grants salary access to anyone who can phrase a SELECT. Splitting a broad tool into narrow ones is often the only way to make authorization expressible at all, and that is an argument no prompt can substitute for.

Calls per workflow points the other way in more surfaces than people expect. A broad tool that answers a question in one call beats three narrow ones chained together, and chapter 1 covered what each additional hop costs. Design A only looks good on this row if the model does not have to probe the schema first, which in practice it does, roughly a third of the time.

When the broad tool actually wins#

The condition is specific: a broad tool wins when the split forces the model to choose between tools on a distinction the arguments do not express.

An order lookup service makes this concrete. Orders older than 180 days move to cold storage, so the backend has two paths and the surface inherited two tools, search_recent_orders and search_archived_orders. A user asks about "that order from the conference." The model has to pick a tool, and the fact that decides which one — the order's age — is precisely the thing the user did not say and the model cannot know. There is no argument it can set to express I don't know how old it is, so it guesses, gets an empty result about 40% of the time, and either reports "no orders found" or tries the other tool, at the cost of an extra hop.

Merging them into search_orders(query, date_range=None) fixes it, because the distinction that was previously encoded in the choice of tool becomes an optional argument. When the user knows the date range they supply it; when they do not, the tool searches both stores and merges the results. The hot/cold boundary is an implementation detail of the storage layer, and it never belonged in the surface.

The same test rules out splits that look sensible on an org chart. get_enterprise_customer and get_smb_customer fail it, because the model does not know a customer's tier before it looks them up. create_invoice_draft and create_invoice_final pass it, because the user's request says which one they want, and — more importantly — the two differ in what happens afterwards.

Expressibility decides one direction only#

The rule cuts one way. A distinction the arguments can express is a candidate for merging; it is not automatically a merge. read_report and delete_report are trivially expressible as report_action(id, action), and collapsing them is a mistake, because the two differ in reversibility and in who is allowed to invoke them. Argument expressibility decides when a split is confusing the model. It says nothing about when a split is protecting you, which is a separate criterion and the subject of the next chapter.

Nor is the choice binary in practice. Both designs above are corners of a space with a lot of useful middle, and the middle is where most production analytics agents end up: a constrained query interface that takes a structured filter object over an allowlist of curated views rather than raw SQL, with a row limit and a cost cap enforced before execution. That shape keeps a good part of the tail — the model can compose filters it was never explicitly given — while keeping the internal-accounts filter and the revenue-column decision in the view definitions where they belong.

{"name": "query_metrics",
 "description": "Query a curated analytics view with filters and grouping.",
 "input_schema": {
   "view": {"enum": ["revenue", "subscriptions", "usage", "invoices"]},
   "measures": {"type": "array",
                "items": {"enum": ["net_revenue", "gross_revenue", "seats",
                                   "active_accounts", "events"]}},
   "filters": {"type": "array",
               "items": {"dimension": {"enum": ["segment", "plan", "region",
                                                "channel", "industry"]},
                         "op": {"enum": ["eq", "in", "not_in"]},
                         "value": "string | array"}},
   "group_by": {"type": "array",
                "items": {"enum": ["month", "quarter", "plan", "segment", "region"]}},
   "period": {"start": "date", "end": "date"},
   "limit": {"type": "integer", "maximum": 500, "default": 100}}}

Every value in that schema is drawn from an enum, so a call naming a dimension that does not exist is rejected before it reaches the warehouse, and the four views are SQL a data engineer wrote and reviewed with the internal-accounts filter, the revenue column and the timezone already settled. Classified against the same 340 Slack questions, this shape covered 88% against the five typed tools' 71%, because the compositions the five could not express — revenue by month for the expansion segment in EMEA, say — are ordinary filter-and-group-by calls here. It cost about three engineer-weeks rather than three days, most of that spent agreeing what the four views mean rather than writing them, and it inherits one weakness from design A: a model that groups by the wrong dimension produces a number that is well-formed, cheap, and not the one anybody asked for.

Seventeen points of coverage on this traffic is around sixty questions a month that stop needing a data-team ticket, and the structural reason those questions become reachable is worth naming. The five typed tools each answer one fixed question, so their coverage is the sum of five fixed things and grows only when somebody writes a sixth tool. The curated-view tool answers a question the caller assembles from a fixed vocabulary, so its coverage is the number of assemblies that vocabulary admits, which is far larger than the number of words in it. Composition is where the extra questions come from. Enumeration is why the compositions stay safe: every view, measure, dimension and operator is drawn from a list a person wrote, so a call naming something outside those lists is rejected by the same mechanism that rejects a bad granularity.

Granularity is decided on five criteria and expressiveness is the one that argues loudest and settles least. What a broad tool buys is coverage of the tail; what it costs is every guard that a typed argument makes automatic — the filter, the column, the timezone, the row bound and the permission. The narrow tools win unless splitting would ask the model to know something the request does not contain, and the middle of the space is usually reachable if you are willing to spend weeks rather than days on it.

That leaves the reverse problem. The tools in a surface you inherit were not designed against these five criteria, and the usual pathology is not one tool that is too broad but several that overlap enough to scatter calls between them, which is what the routing table at the end of chapter 2 was showing.