Not yet is defensible if you leave the door open

The team was two engineers and the feature was an internal one: a weekly-report drafter that turned a project's activity log into three paragraphs a manager could edit. One provider, one prompt, one model, about 3,000 requests a day, roughly $220 a month at the small model's rates. They had read the platform team's design doc for the gateway, which was still six weeks from a first deployment, and they came to ask whether they should wait for it and build against it, or ship now and migrate later.

Nothing else at that company called a model. There was no second team to enforce a budget across, no second workload sharing a credential, no compliance question in flight, and no vendor to block in an emergency because there was one vendor with one workload behind it. Every argument in this topic assumed a population, and this was a population of one.

What the dependency costs a team of two#

The honest accounting for that team is not about latency. Their report drafter runs asynchronously and 6 ms is invisible. It is about what a shared runtime dependency does to a system that currently has none.

Their service today depends on their database and the provider's API. Adopting the gateway adds a component owned by a team they are not on, deployed on a schedule they do not control, with 43 minutes a month of expected total unavailability and a failure mode that is total rather than degraded. It adds an onboarding exercise, a declared degradation behaviour, a token whose lifecycle somebody has to own, and a second place to look when a request fails. It adds a queue: when they want a model the gateway has not been configured for, they file a ticket instead of changing a string.

Against that, the central controls buy essentially nothing while they are alone. A budget across services is meaningless with one service. A kill switch that only they would ever use is a deploy they could do themselves. An audit trail spanning the company is an audit trail of their one workload. The one control that genuinely applies is central key management, and that is available to them without a gateway, which is most of the point of this chapter.

So "not yet" is the right answer, and the condition that makes it right is narrow enough to state: the gateway would be the only shared component they depend on, and the controls it provides have no second party to be enforced against. When either half stops being true, the answer changes.

The thresholds in that table are worth deriving rather than copying, because they encode a comparison between two costs. The gateway's cost to a tenant is roughly the expected outage minutes times what a minute of that feature being down is worth, plus the fixed effort of onboarding and maintaining a declared degradation path — call it three engineer-days initially and a day a year. The gateway's value to a tenant is the losses it prevents, and for a single small workload those are close to zero, since the credential exposure is already handled by the vault and a $220 monthly bill cannot run away far enough to matter before somebody notices. The $2k threshold is where a plausible runaway — a retry loop left running over a weekend — costs more than the onboarding does, and the second-workload threshold is where the value terms stop being zero because there is finally something to enforce across.

The cost on the platform side belongs in the same calculation and usually gets left out. A gateway that anyone should depend on needs an on-call rotation, a deploy pipeline with staggered rollouts, a budget store, a usage pipeline, price tables kept current as vendors change them, and someone who reads the release notes when a provider adds a field to the streaming protocol. At the company in this topic that was two engineers' sustained attention. Standing that up to serve one two-person team is a poor trade for everyone involved, and saying so out loud is more persuasive to a team asking to wait than any argument about their own architecture.

The two obligations that keep the door open#

"Not yet" turns into a problem when it becomes "never, because migrating is now a project". The way to prevent that is to make the eventual adoption a configuration change, which takes two commitments at the point where you tell them to wait.

The first is that the provider key lives in the company vault and is fetched at runtime by the service's workload identity, never written into CI variables, a .env file, or the deployment manifest. That gets the key under central rotation and central audit immediately, which is the control that applies to a population of one, and it means that on the day the gateway arrives, the change is that the platform team stops granting the service read access to that vault path. There is no hunt for copies of the key, because there was only ever one copy.

The second is that every call emits a usage record in the platform's schema, to the same sink the gateway will write to. This is the commitment that gets skipped, and it is worth more than it looks, because it means the company's model-usage table has been complete since the first request rather than starting on gateway launch day.

{
  "schema": "ai.usage.v1",
  "ts": "2026-04-02T09:14:22.418Z",
  "principal": "svc:report-drafter",
  "source": "direct",
  "feature": "weekly-report-draft",
  "vendor": "anthropic",
  "model": "claude-haiku-4-5",
  "input_tokens": 1512,
  "output_tokens": 287,
  "cached_input_tokens": 1024,
  "outcome": "complete",
  "duration_ms": 1840,
  "request_id": "req_01HXZ..."
}

The only field that changes when they migrate is source, from direct to gateway. Dashboards, chargeback queries and the coverage calculation from chapter 3 all keep working across the transition, and the coverage number stays honest because a workload that emits records is a workload you can see, whether or not it goes through the proxy.

A third commitment costs almost nothing and is worth asking for anyway: call the provider through a client whose base URL is configuration rather than a constant.

client = Anthropic(
    base_url=os.environ.get("MODEL_BASE_URL", "https://api.anthropic.com"),
    api_key=vault.read("kv/report-drafter/anthropic")["key"],
)

Because most gateways speak the vendors' own wire formats, accepting request and response bodies in exactly the shape the vendor's own SDK already sends and expects, migration for this team is one environment variable and one credential swap, deployed in an afternoon. The interesting consequence is that if adoption is genuinely that cheap, the decision to wait costs the company nothing, which is exactly what makes "not yet" defensible rather than an excuse.

Writing the policy that separates two kinds of non-compliance#

The same reasoning settles the argument chapter 3 left open. A director wanting every model call behind the gateway by quarter end, enforced by revoking direct provider keys on a fixed date, is looking at two non-compliant workloads that are not the same kind of thing.

The nightly enrichment job holds a key issued against the company's provider organisation. Its traffic is in the audit scope everyone else's traffic is in, its key is one of the credentials a company-wide rotation must cover, and the emergency block that the compliance team has been told exists does not apply to it. Every all-or-nothing guarantee is broken by exactly this workload, and revoking its key is an enforcement action that works, because the key is one the company issued.

The team on a separate vendor account funded from their own cost centre breaks none of those guarantees, because it was never inside them. There is no shared credential to rotate. Its absence from your audit trail is not a hole in a claim you made, since your claim covers your organisation's usage. The kill switch never reached it. Revoking their access is not available to you, and telling them to migrate on a date buys no control you did not already lack — while making it likely they route through a personal account, or a vendor SDK embedded in a product you never see, which is strictly worse than a known team with a known contract. That workload is a procurement and data-processing problem with procurement and data-processing answers.

The policy that follows binds on the thing whose justification is real.

POLICY PLAT-7: Model access on company provider accounts

1. Any workload calling a model on a credential issued against a company
   provider account MUST route through the AI gateway.
2. Direct provider credentials on company accounts are revoked on 30 June.
   Egress to provider domains is denied outside the gateway subnet from
   the same date.
3. A workload may hold a time-boxed exception, granted by the platform
   team, valid for at most 90 days, recorded with an owner and an end
   date. Break-glass paths (ch. 6) are exceptions of this kind and are
   granted per route, not per team.
4. Any workload calling a model on a credential NOT issued against a
   company provider account MUST, within 30 days: register the vendor and
   account with procurement, hold a signed data processing agreement
   before any customer data is sent, and emit ai.usage.v1 records to the
   platform sink. It is not required to route through the gateway.
5. New workloads MUST emit ai.usage.v1 records from their first request,
   whether or not they use the gateway.

Deliberately not covered: models embedded in third-party SaaS products
the company buys (covered by vendor review); coding assistants used by
engineers under the IT acceptable-use policy; models running on-device
or on company hardware with no external API call; research notebooks on
the sandbox account, which is provisioned with a $500 monthly hard cap
and is denied access to production data by network policy.

Rule 4 is the one that gets argued about, because it looks like a smaller rule for the team that complied least. It is a rule whose enforcement mechanism you actually possess — procurement, contracts and a reporting obligation — applied to a workload where the gateway's mechanism does not reach. Uniform enforcement of a rule whose justification is not uniform is how a platform team burns the credibility it needs for rule 1, which is the rule that matters.

The exclusions list is doing real work as well. Writing down that a coding assistant is out of scope, and why, stops the policy from being quietly ignored in a category everyone knows it cannot cover, and keeps the coverage number in chapter 3 meaningful by defining its denominator.

What "not yet" is not#

It is not an exemption, and the difference is a review date. The report-drafter team's answer was recorded as a decision with the triggers from the table above and a scheduled revisit, so the day a second team shipped a model-backed feature, the trigger fired without anyone having to notice.

It is also not a durable state without maintenance. Vault paths get copied into CI variables during a migration weekend, usage records stop being emitted when someone refactors the client, and the door quietly closes. A quarterly check that reads the flow logs, samples the usage sink for each registered workload, and confirms each direct key is still vault-resident costs an hour and is the only thing keeping "configuration, not migration" true.

And a gateway is not the destination. It is a control point, and the controls are what you actually wanted: one credential per vendor with a revocation path whose latency you know, complete coverage of the paths that share your account, a meter you have reconciled against an invoice, one retry layer with a budget, a published availability contract, and a tested answer per caller for the day it is not there. A gateway that has all six is worth its 6 ms. A gateway that has the deployment and none of the six has bought a latency cost and a shared outage in exchange for a diagram.

Which makes the first month of running one mostly a matter of working through that list in order: put every path on it and prove it with egress logs, reconcile the meter against the first invoice and record the correction factor, delete the callers' retry code, publish the SLO with real numbers in it, and run the fault-injection exercise with each team before you tell anyone the switch works.