Model-specific prompting differences

Read these first

• Anatomy of a prompt
• Model landscape literacy

What you will know

• Port a working prompt to a second model and measure what broke

  A model swap can change tool-call compliance, schema adherence, refusal behavior, or answer calibration even when the prompt is unchanged.

• Run the same prompt against two model families and record one formatting or instruction-following divergence

  Delimiters, examples, role handling, and output constraints may carry different authority across model families.

• Patch the prompt for each target model and show the quality change on a fixed evaluation set

  Retuning without a held-out comparison can improve a few examples while degrading the cases that motivated the original prompt.

• Maintain per-model variants without duplicating the whole prompt

  A shared core with isolated adapters limits drift while accommodating differences in roles, tool schemas, and control parameters.

• Set and defend a boundary beyond which a prompt should not be abstracted

  Forcing incompatible capabilities behind one template often produces lowest-common-denominator behavior and hides model-specific assumptions.

Chapters

1. 1

   Prompt portability across models

   Moving a working prompt to a second model changes more than wording tolerance: tool-call compliance, schema adherence, refusal thresholds and answer calibration can each shift while the prompt stays byte-identical. These are the axes worth measuring on both sides of a swap, and what has to be held constant for the two sides to be a comparison at all — the same frozen set, decoding settings matched across two providers whose parameters are not equivalent, and enough repeats to know each model's own spread.

   Read chapter
2. 2

   Per-model instruction authority

   Five hundred clean JSON responses on one model and 6% fenced-with-preamble on another does not make the second model worse at following instructions. Each family's post-training conventions decide how much weight a delimiter, a worked example or an output-format sentence carries. Read such a divergence as something to record and patch per model, and move the constraint onto a mechanism the model cannot route around rather than reaching for a firmer sentence.

   Read chapter
3. 3

   Chat templates and rule placement

   Seven operating rules obeyed and the eighth violated in 4% of conversations looks arbitrary until you know the system turn is flattened using each family's own chat template, and each was trained on its own convention for how far that turn outranks later content. Diagnosing it properly means restating the rule near the end of the input, comparing placement across families, and measuring rather than calling it randomness.

   Read chapter
4. 4

   Decoding guarantees and firm instructions

   Strict schema-constrained decoding cannot emit invalid JSON. \"Return only the JSON object\" is a probability, and its failures land in exactly the tail a 300-case eval is least likely to contain. What changes in the prompt and in the surrounding code when the target model offers no schema, and why a rule the business treats as absolute cannot be implemented by wording on any model — getting a never-quote-prices rule to zero rather than to 1% means a deterministic check on the response before it leaves your process, or removing the capability that makes the violation expressible.

   Read chapter
5. 5

   Chain-of-thought prompting

   \"Think step by step: first list the key claims, then weigh them\" was tuned on a general-purpose model. Ported to a model that reasons internally before answering, the same opening imposes a decomposition the model did not choose, bills you in tokens and latency for the visible steps, and on some tasks lowers quality outright. What to strip, what to keep, and how to tell the two cases apart before rewriting anything.

   Read chapter
6. 6

   Regression testing a prompt patch

   A port leaves twelve failures on a 400-case eval and you rewrite until all twelve pass. Shipping there is the trap: wording written for twelve cases routinely regresses the cases the original phrasing was quietly protecting. The comparison that catches it scores the patched prompt against the unpatched run over the whole set, paired case by case and broken out per error type rather than as one headline number.

   Read chapter
7. 7

   Pricing a model migration

   Migrations are attempted for a cheaper per-call price, and the port is worth that price minus what the divergence costs: validation and repair calls, retries, the p95 an extra round trip adds, the quality of the failures that still slip through, and the standing cost of maintaining a second variant of every prompt. Price those against each other on your own traffic mix and fix the threshold in advance, so a measured schema-violation rate or a blown latency budget becomes a written reason to abandon the migration rather than an argument at review time.

   Read chapter
8. 8

   A shared core with per-model adapters

   Three models, three 90-line descendants of what began as one prompt, a policy change applied to two and missed on the third for three weeks: that drift is structural, not careless. The split that prevents it puts task, policy and domain vocabulary in a single core and isolates roles, tool schemas and control parameters in adapters small enough to read whole in review.

   Read chapter
9. 9

   The model abstraction boundary

   One template, one parameter set, no model-specific text anywhere is a tempting proposal after a three-week migration, and it is wrong at the edges. Native tool calling versus a text protocol, constrained decoding versus an instruction, reasoning-effort controls only some models expose: hiding these turns a visible incompatibility into an invisible quality loss. Where to place the boundary, and how to measure whether it sits correctly.

   Read chapter
10. 10

    Silent updates behind a model alias

    A provider rolling an alias from one dated snapshot to the next is the same mechanism as a cross-provider port, with the timing chosen by somebody else and no baseline on the old side unless you already had one. Log the served model identity, run the two-arm comparison against the pinned snapshot, and treat the pin as a dated loan: on one triage service the roll moved in-policy refusals twenty points and added $945 a month of output tokens nobody attributed.

    Read chapter