Image inputs

Read these first

• Chat API shape

What you will know

• Send an image to a model and get a useful answer about it

  Send a screenshot or a chart and ask a question about it. The capability is better than most people expect.

• Measure how image resolution and tiling change token cost for a chosen provider

  A resize can cross tile boundaries, causing stepwise cost jumps rather than smooth scaling.

• Calculate the token cost of an image before sending it

  Pricing and vision-token formulas vary by model, and rounding or minimum charges can invalidate rough pixel-based guesses.

• Run the same image at two downscaled resolutions and record which answer-critical details disappear

  Tiny text usually fails before broad scene or chart structure, making aggressive compression especially risky for document tasks.

• Compare a vision model with a dedicated parser on one chart or scanned document using accuracy and latency

  Vision can preserve layout and semantics, while parsers often win on exact text extraction, repeatability, and predictable cost.

Chapters

1. 1

   Image content blocks

   An image reaches a model as another typed block in a message's content list, beside the text rather than in an attachment field, and it is charged as input on every request that carries it. This chapter covers what a vision model can actually read, how the request is built in JSON and in Python, how to ask a question the image can answer, and why several images in one message need a text label each.

   Read chapter
2. 2

   Image token cost

   An image is priced on its pixel dimensions after a server-side resize you did not ask for, not on the bytes you uploaded, and the formula differs by provider: some divide area by a constant, some count fixed 512-pixel tiles, and all of them have a floor and a ceiling. This chapter shows the order of the steps, an estimator you can run before sending, measured per-image figures, and why a model upgrade can triple the bill with no change to the request.

   Read chapter
3. 3

   Resolution and per-field legibility

   Halving an image's resolution does not degrade every extracted field equally: layout, chart shape and large type survive it untouched while 8-point footnotes collapse, because a glyph stroke narrower than a pixel becomes a smear. This chapter gives the two-resolution measurement protocol, the ground truth it needs, the cap-height thresholds that explain the ordering, and the traps that make the experiment measure nothing.

   Read chapter
4. 4

   Tile-based image pricing

   Where a provider bills images by 512-pixel tiles, cost is a step function: 1024x1024 and 800x800 both normalise to the same 2x2 grid, so a resize between them removes 39% of the pixels and none of the bill. This chapter works the tiling arithmetic, tabulates the staircase by aspect ratio, gives a sweep that measures the real boundaries for your model, and shows how to read a small saving as evidence that most images never moved.

   Read chapter
5. 5

   The image detail parameter

   Where a provider exposes a detail control, it selects between two processing paths rather than sliding a quality dial: low sends one heavily downscaled view at a flat token cost, roughly nine times cheaper, and is right for large high-contrast questions. On a document it puts the text below the resolution the model can resolve and the model answers anyway, with plausible values that appear on no original. This chapter covers the arithmetic, the per-task binding, and escalation.

   Read chapter
6. 6

   OCR pipelines and vision models

   On a document extractor at volume, accuracy is the least decisive input: determinism, per-page cost multiplied by reprocessing, and whether errors carry a signal decide it. OCR plus rules is exact, repeatable, cheap to re-run and flags its own low-confidence fields; a vision model reads what no rule was written for and returns well-formed wrong numbers. This chapter runs the comparison and derives the split it implies.

   Read chapter
7. 7

   Error independence in repeated runs

   Accepting a field only when two runs agree multiplies error rates together only if the two runs fail for different reasons. Two resolutions of one image are nested rather than independent: the smaller run sees a strict subset, so disagreements are mostly it being wrong about cases the larger run had right, and a faint digit misread identically at both sizes agrees. This chapter measures the correlation and ranks checks that do carry signal.

   Read chapter