Integration tests for tenant isolation

An integration test here is a test that drives the running server through its real transport, against a real database, as an authenticated user, rather than calling a handler function directly. That is the right level for this question, because tenant isolation is a property of the whole path — token validation, identity propagation, the repository, the search index, the cache — and no unit of it can be tested in isolation without assuming the part you are trying to check.

The set that answers the question runs as tenant A and reaches for tenant B. Building it is not hard; the work is in choosing the categories, because a test set that is huge along one dimension and empty along another is more convincing than a small honest one and tells you less. What follows is the categorisation, the assertion that makes each case mean something, and the two categories that go green whether or not the server is enforcing anything.

Six categories that reach across the boundary#

Each category corresponds to a different route into another tenant's data, which is why coverage of one says nothing about the others.

The first three are what most teams build. The last three are where the leaks that survive a review actually live, and they are the ones a per-tool matrix structurally cannot contain, because they are not properties of a tool — they are properties of a path.

That coverage of one category says nothing about another is a fact about the code rather than a convention about testing. Each category exercises a different piece of machinery. Id substitution exercises the repository's predicate. Alternate lookups exercise the search index. Forged claims exercise the derivation of a tenant from a token. Oracles exercise the error path. Replay exercises the idempotency and cursor stores. Ordering exercises the cache. Those are six separate pieces of code, written by different people at different times, with no shared enforcement point between them, so a demonstration about one of them is a demonstration about one of them and nothing more. What makes a per-tool matrix feel like coverage is that it varies the tool while holding all six pieces of machinery fixed — and the tool is the one dimension along which nothing interesting differs.

Two categories are worth spelling out because their concrete cases are less obvious than the first. Forged claims are not an attempt to defeat signature verification, which belongs to topic 6.12, on authenticating callers to an MCP server; the case that matters is a validly signed token whose tenant claim disagrees with the tenant the server would derive some other way, plus every side channel through which a tenant can be suggested — an X-Tenant-Id header that an internal gateway is trusted to set, a tenant field a tool schema still accepts because it was there before the token carried one, a workspace id in a resource URI. The assertion is that the derived tenant wins and the suggestion is ignored, and the interesting failure is a server that logs a mismatch and proceeds. Replay is the same idea applied to state: take a call that B made and that failed, and re-issue it as A with B's idempotency key, B's request id, B's pagination cursor. Cursors are the most productive of those, because an opaque cursor is usually a base64 blob containing an offset and a filter, and a server that decodes it and trusts what it finds has accepted a filter from the caller.

Asserting that nothing of B's appears anywhere#

The assertion is a design decision rather than a detail. Asserting status == 403 checks that the call was refused, and a refusal that names the thing it refused has already leaked. What the assertion should say is that no identifier, name or fragment belonging to B appears anywhere in what A received, error text included:

def assert_no_trace_of(other: TenantFixture, response: ToolResult) -> None:
    haystack = json.dumps(response.payload) + (response.error_message or "")
    for needle in other.canaries():        # ids, names, titles, emails, file keys
        assert needle not in haystack, f"{needle!r} leaked to {response.tenant_id}"

canaries() returns the strings that exist nowhere except in B's fixtures — a document titled Zarquon Merger Memo, a customer named Blicero GmbH, a storage key t_bravo/2026/07/8841.pdf. Fixtures full of Test Document 1 and user@example.com cannot support this assertion, since those strings collide across tenants by construction, and a suite built on them will be quietly unable to tell a leak from a coincidence.

Follow what happens to a suite built on colliding fixtures, because the failure is not the one people expect. If both tenants own a document called Test Document 1, then A's own perfectly correct response contains that string, and the assertion "nothing of B's appears in what A received" fires on a server that is behaving exactly as intended. The test is now red for a reason nobody can fix, so it gets weakened — the needle list is trimmed, or the assertion moves from the whole payload to a single field, or the case is skipped. Each of those makes the suite green again by removing its ability to fail, which is the same defect the happy-path suite has, arrived at from the opposite direction. A canary is a string chosen so that its appearance has exactly one possible cause. That is not a preference about fixture hygiene; it is the condition under which the assertion is capable of meaning what it says.

The harness that produces the cases should generate them rather than have them written by hand, because a hand-written matrix goes stale the day a tool is added:

@pytest.mark.parametrize("tool", ALL_TOOLS)
@pytest.mark.parametrize("arg", ID_SHAPED_ARGS)
async def test_cross_tenant_id(server, tenant_a, tenant_b, tool, arg):
    victim_id = tenant_b.real_id_for(arg)
    baseline = await server.call(tool, {**tool.valid_args(tenant_a), arg: tenant_a.real_id_for(arg)},
                                 as_=tenant_a)
    assert baseline.ok, "positive control failed: the call does not work at all"
    attack = await server.call(tool, {**tool.valid_args(tenant_a), arg: victim_id}, as_=tenant_a)
    assert_no_trace_of(tenant_b, attack)

The baseline call is the part that is easy to leave out and expensive to leave out.

Two categories that pass for the wrong reason#

A green test tells you the assertion held. It does not tell you the code under test was exercised, and two of the six categories fail that way routinely.

The property that separates a test which measures something from a test which does not is worth stating exactly, because it is the same property in all six categories and it is what the rest of this chapter is applying. A test is evidence about a claim only if its result would have been different had the claim been false. Take a suite of 340 cases in which every tenant successfully reads its own records, delete every tenant predicate in the repository, and run it again: all 340 still pass, because a server that returns everything to everyone also returns A's records to A. The outcome does not depend on the claim, so observing the outcome carries no information about it, however many times it is observed. A cross-tenant case has the property by construction — delete the predicate and it goes red — which is the entire reason for building one. It is also the reason a cross-tenant case that passes for an accidental reason is worse than having no case at all, because it has the appearance of the good kind.

Direct id substitution passes when the id was never real. If tenant B's fixtures are created in a separate transaction that the test rolls back, or if the test author writes id="doc_00000000" because a real id was inconvenient to obtain, the server answers "not found" — and it would answer "not found" with no tenant enforcement at all, because there is nothing there. The whole matrix goes green in eleven seconds and means nothing. The way to tell is the positive control in the harness above, a positive control being a companion case whose success proves the machinery under test was actually reached: the same call, with the same argument, made by a user who is entitled to that id, must succeed within the same test run. If the baseline fails, the attack case proves only that the tool is broken. Running the two calls adjacently, against the same seeded data, is what turns "the server said no" into "the server said no to A and yes to B".

Alternate lookup paths pass when the corpora do not overlap. A test that authenticates as A, searches for invoice, and asserts none of B's documents appear is a test that will pass on a completely unfiltered index if B's fixtures happen not to contain the word invoice. Search fixtures tend to be generated per tenant from separate templates, so non-overlap is the default rather than the exception. The way to tell is to plant the collision deliberately: write a canary string into B's corpus that appears nowhere else, wait for the index to catch up, and search for that exact string as A. If the search stack is asynchronous, the test also has to wait for the document to become searchable — a test that queries 50 ms after the write and finds nothing has measured indexing lag, not isolation.

The general principle covers both cases and applies to categories you invent later. A negative assertion is only meaningful when you have shown that the thing you asserted the absence of was reachable in the first place. Every negative test needs a companion that demonstrates the data exists, is findable, and would have been returned had the boundary not held. Without it, the suite is a machine for producing green.

Written out as an elimination, the pairing is a single step. An empty response has exactly two explanations: the boundary refused the record, or the record was never reachable. The negative case on its own cannot separate them, because it produces the same output under both. The positive control makes the same call against the same seeded row as a user who is entitled to it, and that call has different outputs under the two explanations — it succeeds if the row is there, and fails if it is not. Observing that it succeeds strikes out the second explanation and leaves the first, which is why the pair is evidence and neither half of it is. Running the two adjacently, inside one test function, against one seeding, is what keeps the elimination valid: two calls made against two different database states are two experiments, and two experiments eliminate nothing about each other.

Mutation testing, which means deliberately introducing a defect into the implementation to see whether the suite turns red, is the cheap systematic version. Delete the tenant predicate from one repository method, or make Identity.tenant_id return a constant, and run the isolation suite. If it stays green, the suite is not testing what its name says. Doing that once per quarter against three or four deliberately broken builds costs an afternoon and is the only routine that catches a suite rotting into decoration.

What that routine does is ask the discriminating-power question mechanically rather than by inspection. Reasoning about whether a case would have failed under a false claim is hard, because it means holding the whole path in your head at once; manufacturing the world in which the claim is false and running the suite inside it means holding nothing at all. A suite that stays green against a deliberately broken build has told you, without anyone needing to argue about what the tests mean, that its outcome does not depend on the property it is named after.

Fixtures, coverage and today's build#

When the customer's engineer asks what the suite does not prove — and a good one always does — the answer that keeps the conversation honest has four parts.

It covers the paths that were thought of. Every category above came from someone's model of how data escapes, and the categories the reader has not thought of are exactly the ones with no tests. The audit-hook recording from the chapter on ambient authority is a partial defence, since it enumerates outbound calls from the running process rather than from anyone's memory, and partial is the accurate word.

It covers a fixture, not a corpus. Isolation tests run against tens of rows per tenant. Behaviour that only appears at scale — a query planner choosing a different index, a search engine's cross-shard scoring, a cache that only evicts when it is actually full, a pagination cursor that encodes an offset into a shared result set — is invisible at fixture size. A leak that requires 40 million rows to surface will not appear in a suite that finishes in four minutes.

It covers today's build. The suite is a regression barrier, so it says that the paths it knows about are still closed at this commit, and it says nothing about the tool merged next Tuesday that reaches a new service. That is an argument for the suite running in CI on every merge rather than quarterly, and it is still not an argument that the next path will be covered.

And it covers the application's behaviour while the application is behaving. Nothing in the suite constrains what happens when the process is running code that is not yours. A customer who accepts the suite as proof of isolation has been given more assurance than the artefact supports, and saying that first is cheaper than having it discovered.

A suite that reaches across the boundary is the only artefact in this topic that a customer's security team can check for themselves, which is why it is worth building carefully rather than counting. Three properties do the work: the cases are generated from the tool registry so they cannot go stale, the assertion looks for B's canaries in everything A received rather than for a status code, and every negative case is paired with a positive control that proves the data was there to leak.

There is a fifth thing the suite does not cover, and it is nearer than the others: some of these paths do not run inside a tool handler at all. A retry that fires after the response has been sent, a cache populated by one request and read by the next, a log line written on the way out and an error string assembled by a framework layer above the handler — the tests above can only observe those where they happen to surface in a response. When they surface somewhere else, the first report tends to arrive from a support engineer rather than from CI.