The window is freed by cutting definitions, not deployments

The four servers went out over two sprints and the availability work paid off immediately: a GitHub incident in the second week removed nine tools and left the on-call flow running. The context measurement after the cutover was 13,100 tokens of tool definitions, against 14,200 before, and the 1,100 was the three runbook tools that had become skills. That is the outcome chapter 5 predicted, and the team still had a 32,000-token budget with 41% of it spent before the conversation started.

The work that actually moved the number took three days and touched no deployment topology at all. It is worth going through in the order the team did it, because the cheapest technique came first and the most invasive came last, which is the correct order and not the one people usually pick.

Nine tools nobody had ever called#

The first pass was a query against 90 days of trace data, grouped by tool name, and it returned the sort of distribution that surveys of tool use tend to produce: a short head carrying most invocations and a long tail of definitions that had never been exercised. Nine of the 37 tools had zero calls in 90 days. list_issue_link_types, get_issue_watchers, list_repository_topics, get_branch_protection, and five others existed because somebody had walked the upstream API's endpoint list and wrapped each one, which is a reasonable instinct when building a server and produces an interface sized to the API rather than to any agent's needs.

Deleting them removed 2,800 tokens and cost nothing, because a tool with no calls in a quarter has no user. The only care needed is distinguishing a tool nobody calls from a tool that is called rarely and matters when it is — rollback_deploy had four calls in 90 days and is not a deletion candidate, which is a judgement about consequence rather than about frequency, and it is the reason this pass has to be read by a human rather than run as a script.

Six write operations that were one operation#

Jira's write surface had been modelled as one tool per API verb: create_issue, add_comment, transition_issue, assign_issue, add_label, link_issue. Six definitions, six descriptions with substantially overlapping prose about issue keys and permissions, 2,100 tokens. The model's actual usage was a sequence — file the issue, assign it, transition it to In Progress, comment — which meant four round trips through the loop for one logical act, each one re-sending the full definition set.

{
  "name": "update_issue",
  "description": "Create or modify a Jira issue. `action` selects the operation; the fields required for each action are listed below.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "action": {
        "enum": ["create", "comment", "transition", "assign", "label", "link"]
      },
      "issue_key": {"type": "string", "description": "required except for create"},
      "project": {"type": "string", "description": "create only"},
      "summary": {"type": "string", "description": "create only"},
      "body": {"type": "string", "description": "comment only"},
      "to_status": {"type": "string", "description": "transition only"},
      "assignee": {"type": "string", "description": "assign only"},
      "labels": {"type": "array", "items": {"type": "string"}},
      "link": {
        "type": "object",
        "properties": {"type": {"type": "string"}, "to": {"type": "string"}}
      }
    },
    "required": ["action"]
  }
}

That is 640 tokens where six tools were 2,100, and the same treatment on four GitHub tools took 1,400 down to 520. The saving is real and the trade is real with it, so it should be made deliberately rather than as an obvious optimisation.

Collapsing weakens the schema. Six tools each with their own required list let the transport reject a transition call missing to_status; one tool with a mode argument cannot express that in JSON Schema without oneOf branches that most clients render poorly and some models handle badly, so the enforcement moves inside the handler and arrives as a runtime error after the model has already committed to the call. It also weakens authorization, because permission that was granted per tool — this agent may comment but may not transition — now has to be enforced per argument value inside one tool that the agent either has or does not. The team kept create_issue separate for exactly that reason: it was the one operation with a different approval path, and folding it into the enum would have made the permission model a string comparison in application code.

The rule that came out of it was to collapse operations that share a subject, a permission and a failure mode, and to leave alone anything that differs on any of the three.

The long tail behind a question#

Six of the remaining tools were used in fewer than 2% of conversations and were genuinely needed when they were used: reassigning a deploy lock, querying a specific audit endpoint, forcing a webhook redelivery. Together they were 1,900 tokens resident in every request to serve one conversation in fifty. The alternative is to make them discoverable rather than resident.

DISCOVERY_TOOLS = [
    {
        "name": "list_capabilities",
        "description": (
            "List rarely-used operations available in a domain. Use when no "
            "resident tool covers the task. Domains: jira, github, deploy."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"domain": {"enum": ["jira", "github", "deploy"]}},
            "required": ["domain"],
        },
    },
    {
        "name": "invoke_capability",
        "description": (
            "Invoke an operation returned by list_capabilities. Call "
            "list_capabilities first to learn the argument names."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "name": {"type": "string"},
                "arguments": {"type": "object"},
            },
            "required": ["name", "arguments"],
        },
    },
]

Two definitions at 380 tokens replace six at 1,900. The cost is paid by the conversations that need the tail: two extra model turns and two extra round trips, which measured at 1.4 seconds of added latency and about 900 extra tokens for the listing plus the reasoning around it. Spread over the 2% of conversations that take it, that is a good trade; applied to a tool used in 40% of conversations it would be a bad one, and the crossover is roughly where the resident cost of the definition stops exceeding the amortised cost of discovering it. The failure mode to watch is a model that does not think to ask — discovery only works if the entry point's description tells it clearly when to reach for the list, which is why list_capabilities above says "when no resident tool covers the task" rather than describing what it returns.

Different tools for different jobs#

The last pass was the one with the largest effect and it required no change to any server. The on-call incident flow used 9 of the 15 remaining tools; the weekly release-notes flow used 5, of which 3 overlapped. Nothing required the agent to carry both sets at once, since the client decides which of the configured servers' tools it forwards to the model.

TOOLSETS = {
    "incident":     ["jira.update_issue", "jira.search_tickets",
                     "deploy.status", "deploy.rollback_deploy",
                     "github.search_code", "github.get_commit",
                     "slack.post_message", "list_capabilities",
                     "invoke_capability"],
    "release_notes": ["jira.search_tickets", "github.list_merged_prs",
                      "github.get_commit", "slack.post_message",
                      "list_capabilities"],
}

def tools_for(task: str, catalog: dict[str, Tool]) -> list[Tool]:
    return [catalog[name] for name in TOOLSETS[task]]

The incident set is 3,800 tokens. The release-notes set is 1,600. Neither agent ever sees the other's definitions, and the tail is reachable from both through discovery if a conversation goes somewhere unusual.

The end state is 3,800 tokens against 14,200, and the entire reduction came from the four rows below the split. The split row moved 1,100 tokens, all of it from noticing that three tools were carrying documents.

There is a fifth lever the team did not need and which is worth knowing about, because for some interfaces it is the largest one available. Most of a tool definition's token cost is prose rather than structure: a schema with six properties serialises to a couple of hundred tokens, and the descriptions attached to the tool and to each property are frequently three or four times that. Definitions written by wrapping an upstream API tend to inherit the vendor's documentation, including the paragraph about rate limits and the two sentences about the deprecated expand parameter, none of which changes what the model does. Rewriting descriptions to say only what the model needs in order to choose the tool and fill its arguments routinely halves them. The reason it went unused here is that it is the one pass that can silently make the agent worse — a description trimmed past the point where the model can tell two tools apart produces wrong tool selection rather than an error — so it wants an evaluation set behind it — a fixed collection of tasks with known-good outcomes that you re-run after every change to see whether tool selection got worse — and the team had 40 tools and no evaluation set, which is a common and honest place to be.

Answering the objection about four deployments#

The reviewer who says that splitting means four deployments instead of one is correct on the fact and is comparing it against the wrong thing. The answer is not that reliability is worth the cost; it is that the two sides of the comparison are different kinds of quantity, and only one of them can be planned for.

A deployment is a recurring cost you can price. For this team the four servers were generated from one internal chart, so the marginal deployment was about three hours to set up and roughly an hour a month of patching, dependency bumps and certificate renewals, all of it landing on a rota that already existed. Four deployments is not four times the work of one, since the base image, the CI pipeline, the dashboards and the alert rules are shared, and the honest figure for the split was around 14 hours up front and four hours a month across all four. That is a line in a capacity plan and it can be staffed, deferred, or refused on budget grounds like anything else.

Shared fate is not that kind of quantity. One server fronting four systems means any one of those systems can take the other three down with it, and the cost of that is the product of an incident rate you do not control and a consequence that depends on what the agent happened to be doing at the time. The March outage cost 90 minutes of a team that could not file tickets during an incident, and the next one might cost nothing or might cost a rollback that did not happen. You cannot put a number on next year's version of it, which is exactly why it does not appear in the comparison the reviewer is making, and why the answer has to be to name the axis rather than to argue about hours.

The same framing settles the context half. Four deployments buy availability and buy no tokens; the four passes above buy tokens and buy no availability. Presenting them as one project invites a reviewer to weigh 14 hours of platform work against a context saving that the platform work does not deliver, and the write-up should keep them separate for that reason.

Everything in this chapter and the last assumed the servers are operated by your own platform team, which makes the arguments about deployments, blast radius and interface size internal engineering questions. When the operator is outside your organisation, a further set of questions applies that no amount of interface trimming addresses, and the next chapter is about what a reviewer needs to see before that dependency is approved.