The provider seam

A provider seam is the boundary where code written in terms of your product ends and code written in terms of one company's API begins, and therefore the one place where the question of which company that is gets answered. It exists because vendors' SDKs — a software development kit being the client library a model vendor ships so you can call its API from your own language — disagree with each other about the shape of a request, the names of parameters, the events a stream emits and the class hierarchy of errors, all of which topic 2.2 set out. Without a boundary, each of those disagreements is copied into whichever module needed a model, and the vendor's name ends up in fifty files instead of one. The seam turns up in three forms, and this topic works through all of them: a hand-rolled adapter, a class you write and run inside your own process; a gateway, a separate service you deploy and operate so that keys, budgets and audit logs live in one place; and a router, a hosted third-party service that fronts dozens of vendors behind a schema you did not design.

This chapter is about the first form and about the design question common to all three: what the boundary promises the caller, and where the vendor decision gets resolved. Those two are separable and both are usually got wrong in the same way, by building one object that represents "the provider" and selecting it with one setting. The alternative is a port per capability — an interface named after something your application needs, with the smallest set of methods that capability requires — resolved at the composition root, which is the top of main(), the lifespan handler, the worker's startup function, or whatever your framework calls the entrypoint where concrete things are built and handed to the code that uses them.

One environment variable answers a question with several answers#

Here is the shape that looks like a seam and behaves like a global:

# infra/llm.py
class Provider:
    def __init__(self, name: str) -> None:
        if name == "anthropic":
            self._client = Anthropic()
        elif name == "openai":
            self._client = OpenAI()
        else:
            raise ValueError(f"unknown provider {name}")
        self.name = name

    def chat(self, messages: list[Message], **kw) -> ChatResult: ...
    def embed(self, texts: list[str]) -> list[list[float]]: ...


provider = Provider(os.environ["PROVIDER"])   # module-level singleton

Domain code imports that singleton directly:

from infra.llm import provider

def index_article(article: Article) -> None:
    chunks = chunk(article)
    vectors = provider.embed([c.text for c in chunks])
    store.upsert(article.id, chunks, vectors)

The PROVIDER variable is a single-valued answer to a question that has one answer per capability. Setting it to openai moves embeddings and chat together; there is no value it can take that means "embeddings there, chat here". Every route to a different embedding vendor therefore runs through a change to the shape of Provider — a second constructor argument, or a second singleton, or an EMBEDDING_PROVIDER override read inside embed() — and each of those propagates into every call site, into the test fixtures that patch infra.llm.provider, and into the background workers that build their own instance at startup because they run outside the web process.

The reason no value of PROVIDER can express a split decision is worth stating as a counting argument rather than as a complaint about the code, because the counting argument tells you what to change. A configuration variable holds one value at a time. A value selects one branch of the if chain in that constructor. That branch builds one client, and the one client answers both chat and embed. So the number of independent vendor decisions the design can represent is one, and the number the application has is two — and no amount of care at the call sites alters that, because the shortfall is in the shape of the type, not in how it is used. The Provider class bundles two capabilities into one object, so any mechanism that selects the object selects both capabilities together.

Seeing it that way makes the fix obvious and makes the usual fixes visibly insufficient. Adding an EMBEDDING_PROVIDER override read inside embed() raises the count from one to two, and stops there — the next capability that needs its own vendor gets a third special case inside a class that now branches on vendor names in two different places. What scales is separating the object, so that each independent decision has its own type to be resolved into, and the count of decisions the design can represent is the count of ports rather than a constant somebody has to keep raising.

That matters more than it sounds, because the choice of vendor is not settled once. A model that is the best available in March is mid-priced and second-best by September, and pricing on embeddings in particular moves independently of pricing on chat, so the question of who computes your vectors reopens every few months without the question of who answers your users being reopened at all.

Naming the port after the caller, not the seller#

Keeping vendor identity out of the places where product logic lives, so that a decision about vendors becomes a decision about wiring, is a narrower goal than it sounds, and the narrowness is what makes it achievable. A seam does not give you a uniform model API, and it does not make two providers behave identically. Its whole job is that index_article never learns which company computed its vectors.

The unit that carries this is a port. Embedder is a port. Anthropic is not. The distinction matters because ports are named from the inside — from what the caller wants — while a client class is named from the outside, after whoever sells it. A port named after the caller's need has a natural size, because the caller's need is bounded; a port named after a vendor has no natural size, because the vendor keeps shipping features.

from typing import Protocol

class Embedder(Protocol):
    def embed(self, texts: list[str]) -> list[list[float]]: ...

class ChatClient(Protocol):
    def chat(self, messages: list[Message], *, system: str | None = None) -> ChatResult: ...

Two protocols, two methods between them. index_article now takes an Embedder and knows nothing else:

def index_article(article: Article, embedder: Embedder) -> None:
    chunks = chunk(article)
    vectors = embedder.embed([c.text for c in chunks])
    store.upsert(article.id, chunks, vectors)

Nothing in that function mentions a vendor, an environment variable, or an SDK.

The composition root is where vendor names are allowed to appear#

The composition root is the one place where importing from openai import OpenAI is correct. Everything below it receives what it needs as an argument.

# app/composition.py
def build_embedder() -> Embedder:
    match os.environ["EMBEDDING_PROVIDER"]:
        case "voyage":
            return VoyageEmbedder(model="voyage-3-lite")
        case "openai":
            return OpenAIEmbedder(model="text-embedding-3-small")
    raise ValueError(os.environ["EMBEDDING_PROVIDER"])


def build_chat_client() -> ChatClient:
    match os.environ["CHAT_PROVIDER"]:
        case "anthropic":
            return AnthropicChat(model="claude-sonnet-4-5")
        case "openai":
            return OpenAIChat(model="gpt-4o")
    raise ValueError(os.environ["CHAT_PROVIDER"])

Two builders, two environment variables, independently set. Moving embeddings is now EMBEDDING_PROVIDER=voyage plus one adapter class implementing one method, and CHAT_PROVIDER stays where it is. No domain module changes, because no domain module ever named a vendor, and the move is reversible, since flipping the variable back is not a code deployment.

None of this requires a dependency-injection container — a Spring, or one of the lighter Python equivalents — and reaching for one is a common way to make the idea look heavier than it is. The mechanism is passing arguments; a container is an optional convenience for applications with enough wiring to justify it. What matters is that construction happens once, at a place you can point to, and that everything downstream receives an interface it did not build. The test consequence follows for free: a unit test of index_article passes a fake embedder returning fixed vectors, with no patching of module globals and no environment variable set, which also means two tests can use two different fakes in the same process.

That last clause is the whole of why a seam at the call site is testable and a global one is not, and the reason is about when a name gets bound to an object. from infra.llm import provider binds at import time, once per process, and copies the reference into the importing module's own namespace. A test that wants a different object has to reach into module state and replace it, and module state is shared by every test in that process, so the replacement is visible to tests that did not ask for it and has to be undone afterwards. Two tests wanting two different embedders in the same run are asking one variable to hold two values at the same instant, which is the same shortfall as the PROVIDER variable above, arriving in the test suite instead of in production.

A parameter binds per call. index_article(article, embedder) resolves embedder from the arguments of this invocation and nothing else, so two calls in the same process, in the same test file, microseconds apart, can be given two different objects without either knowing the other exists. Nothing has to be patched, because nothing was ever shared. This is also why the fake needs no relationship to the real adapter beyond having an embed method of the right shape — the protocol is structural, so a class defined in the test file satisfies it without importing anything from the application.

The failure in the singleton design was not that it had an abstraction. It had one. The failure was where the vendor decision was resolved and how many capabilities it resolved at once. Three placements are common, and they differ on both of those counts:

The third row is not more abstract than the second. It has more interfaces and less machinery, because each interface is one method wide and none of them contains a branch on a vendor name.

Reading the table down rather than across shows that the three rows are three placements of one decision rather than three levels of sophistication. In the first row the vendor decision is made at every call site. In the second it is made once, at import, by a constructor that resolves both capabilities together. In the third it is made once per capability, at startup, in a file whose job is to make decisions. Every other column is a consequence of that one difference, including the last: what a test can substitute is exactly what the decision was deferred to, so deferring it to a module global gives a test a module global to patch, and deferring it to a parameter gives a test a parameter to fill.

The port promises less than the provider offers#

ChatClient.chat above takes messages and an optional system prompt. Anthropic's client can also take thinking, tools, tool_choice, cache_control markers on content blocks, a service_tier, and a stop-sequence list. OpenAI's can take response_format, logit_bias, seed, parallel_tool_calls. None of those appear in the port, and their absence is deliberate rather than an oversight to be corrected later.

A port is a promise to the caller that every implementation will honour. Each thing you add to the promise is something every current adapter must implement and every future adapter must implement, and adding it later is a change to the interface, to all existing implementations, and potentially to callers. So the question to ask about a candidate parameter is not whether some provider supports it but whether the caller's logic depends on it and everything behind the port can do it. A summariser that needs deterministic output does depend on temperature, so temperature belongs. A summariser that happens to run against a vendor offering a seed does not thereby need seed in the shared contract; that is a property of one adapter's construction, and it can live in the adapter's constructor where it costs nothing to anyone else.

There is a useful test for the boundary. Ask whether the caller would branch on the thing. If index_article would ever write if provider_supports_batching: ..., then batching is part of the contract and has to be represented; if it would not, the capability belongs inside whichever adapter can exploit it. An adapter that batches transparently because its vendor charges half price for asynchronous batch jobs is free to do so, and the port still says embed(texts) -> vectors, because from the caller's position nothing changed except latency. The port describes the caller's model of the world, and the adapter is allowed to be cleverer than that model as long as it does not leak the cleverness upward.

That test works because of what a port is for. A port exists to keep something out of the caller, and a thing the caller branches on is by definition not being kept out of it — the branch is the caller knowing. So "would there be a branch" is not a heuristic about interface design; it is the definition of the boundary, asked in a form somebody can answer about a specific parameter on a specific afternoon. Applied honestly it removes most candidate parameters, because most of them are things one vendor happens to offer rather than things the application's own logic turns on.

This is where the analogy to a database repository holds and where it stops. A UserRepository with get, save and find_by_email is small for the same reason: it promises what the application uses, and Postgres-specific features live in the Postgres implementation. The difference is that model providers ship user-visible capabilities on a monthly cadence, and each one arrives with product pressure attached, so the force pulling methods into the shared contract is much stronger than it ever was for a database.

Vendor names and vendor behaviour#

A seam gives you independence of vendor names in domain code. It does not give you independence of vendor behaviour, and confusing the two is the most expensive mistake available here. Swapping an embedder means the vectors in your store were produced by a different model with different dimensionality and a different geometry, so every existing vector is incomparable to every new one and the actual migration is a full re-index of 90 million tokens plus a cutover plan for the index. The port made the code change small; it did nothing about the data.

The same holds on the chat side. Two adapters behind one ChatClient will not respond identically to the same prompt, and the prompt that was tuned against one model is an input your seam passes through untouched. Nor does the seam address the differences in error taxonomies, rate-limit headers and stop reasons that topic 2.2 laid out — those still have to be handled, now inside adapters rather than scattered through the application, which is an improvement in locality rather than an elimination of work.

What a correctly placed seam buys, then, is precise and modest: the vendor decision becomes a line of configuration read at startup, per capability, and the blast radius of changing it is one adapter class. That is worth having because it converts a fortnight of chasing a global into a pull request, and it is worth stating narrowly because the same seam will be sold internally as "we can switch providers", which is a claim about data and behaviour that no interface can make.

What remains open is the size of the promise. Two one-method ports are easy to defend on the day you write them, and the ticket that arrives three weeks later asks for structured outputs, and the one after that asks for tool calling, and each request has a reasonable case for living in the shared shape. The next chapter follows an interface that said yes fourteen times and looks at what each yes cost.