Skip to content

8 min read

Lead with the answer and keep the reasoning one action away

On this page

The second version of Atlas shipped with a layout the team called show your work. The answer pane opened with the search plan the model had written, then the four retrieved passages with their relevance scores, then a short paragraph of reasoning about which passage governed, and finally the answer. On a 900-pixel-tall viewport at the default zoom the first word of the actual answer rendered at pixel 1,340. Session replay over the first fortnight showed that 41% of sessions never scrolled the pane at all, and among those that did, the median time from request to the answer being visible on screen was 9.2 seconds against a total generation time of 5.8 — users were spending longer finding the answer than the system spent producing it.

Handlers did not complain that the answer was hidden. They complained that Atlas had got slower and that it "waffled", which is what a product looks like from the outside when the thing you asked for is the last thing on the page. The team reversed the order in a two-week experiment: answer first, everything else behind a control labelled How Atlas got this. Total generation time did not change. Median time to the answer being visible fell to 4.4 seconds, the rate at which handlers opened the reasoning panel settled at 12% of answers, and the rate at which they opened the sources settled at 31%. The 12% is the interesting number, because the entire original layout had been justified by the belief that handlers wanted to see the reasoning.

The layers, and the cost of moving between them

Progressive disclosure means the interface answers the question in the first sentence or two and keeps everything a sceptical reader might want exactly one action away. It is an ordering discipline with a click budget attached, and the discipline is what stops the phrase from degenerating into "put the long stuff in an accordion".

For an assistant answering from documents, four layers are usually enough.

LayerContentReached byWho opens it
0The answer in one or two sentences, plus the support label and any caveat that changes what the reader does nextnothing — it is on screeneveryone
1The full answer with per-span marks, the caveats, and the list of sources with their titles and locationsone click, in placethe reader deciding whether to trust it
2The retrieved passages as text, the search queries that produced them, tool steps and their argumentsa second click, usually a side panelthe reader auditing a specific claim, and support staff
3The prompt, the model version, token counts, the raw envelopenot exposed in product; available in the trace viewerengineers and incident response

The rule that keeps this honest is that layer 0 must be actionable on its own. A reader who stops there and acts on what they read should not be acting on a misunderstanding that layer 1 would have corrected. That constraint decides most of the arguments about what goes where, because it converts is this important? into would knowing this change what they do? — a question with an answer.

Applied to the subsidence example from chapter 1, the 60-day window belongs in layer 0 because it is the answer. The requirement that the appeal be in writing belongs in layer 0 too, because a handler who tells a policyholder "you have 60 days" without that has given advice that will produce a rejected appeal. The identity of the clause it came from belongs in layer 1, because knowing it is clause 14.2 rather than 14.3 changes nothing about what the handler does next. The fact that the manual was last revised eleven months ago belongs in layer 1 as well. The fact that this policy was superseded by an endorsement three weeks ago belongs in layer 0, and putting it behind a click is the kind of mistake that produces the incident in chapter 9.

What does not qualify as progressive disclosure

Collapsing a three-sentence answer behind a Show answer control is not disclosure, it is a toll booth. The test is whether the hidden content is optional for the reader's task; a 4,000-token retrieval dump is optional for nearly everyone, a two-line answer is optional for nobody.

Hiding a caveat because it hurts the numbers is a different failure with the same shape. A team that finds its conversion rate improves when the not found in your documents label moves into layer 1 has learned something true about conversion and nothing about whether the change is defensible. The uncertainty machinery of chapter 1 lives in layer 0 or it does not exist, because its entire function is to be seen before the reader acts.

There is also a version of this that hides too much. Sources at 31% open rate are being opened by a third of readers, which means the summary line about them — 3 sources, Household Claims Handling Manual and 2 others — is doing real work in layer 0 even though the sources themselves are not. Disclosure works when each layer advertises what is underneath it accurately enough that the reader can decide whether to spend the click. A control labelled Details advertises nothing; How Atlas got this (4 passages) tells a reader what they are buying.

Getting the answer first out of a model that reasons first

The layout above conflicts with how generation works. Models produce better answers on multi-hop questions — ones that cannot be answered from any single retrieved passage, so the model has to combine two or three of them and carry an intermediate conclusion — when they work through the problem before committing, and a model that has been told to state its conclusion in the first sentence commits before it has done the work. On Atlas's internal evaluation set of 340 policy questions, forcing answer-first phrasing dropped exact-match accuracy on the multi-clause subset from 0.79 to 0.71. That is a real cost and it is why show your work got built in the first place.

There are four ways out of it, and they trade different things.

ApproachAnswer qualityTime to answer on screenExtra tokensNotes
Prompt for answer-first prose−8 points on multi-hop0.4snonecheapest, and the accuracy loss lands on exactly the hard questions
Reason into a hidden field, then an answer field, one structured streamunchanged3.9snone beyond the reasoning it was doing anywaythe reasoning field is generated but never rendered as the primary surface
Two calls: reason, then a second call that writes the answerunchanged4.6s+31%simplest to reason about, worst latency, second call can drift from the first
Stream reasoning into a secondary rail while the answer streams into the main columnunchanged3.9snonerequires the model to emit both channels; the rail must be visually subordinate

Atlas went with the second, and the shape of it is a structured stream where the client renders fields as they complete rather than rendering the token flow directly.

ts
import { streamObject } from "ai";
import { z } from "zod";

const AnswerSchema = z.object({
  // generated first, never rendered as the primary surface
  working: z.string().describe("Reason about which clauses govern, then stop."),
  // layer 0
  headline: z.string().describe("The answer in one or two sentences."),
  blocking_caveat: z.string().nullable()
    .describe("Only if a reader acting on the headline alone would be wrong."),
  // layer 1
  detail: z.string(),
  sources: z.array(z.object({
    chunk_id: z.string(),
    doc_title: z.string(),
    locator: z.string(),
  })),
});

const { partialObjectStream } = streamObject({
  model: "claude-sonnet-4-5",
  schema: AnswerSchema,
  prompt: buildPrompt(question, retrievedChunks),
});

for await (const partial of partialObjectStream) {
  // headline arrives after `working` completes; render it the moment it exists
  ui.update({
    headline: partial.headline,
    caveat: partial.blocking_caveat,
    detail: partial.detail,
    sources: partial.sources ?? [],
  });
}

The client-side counterpart is unremarkable, and that is the point — once the layers exist as fields, the layout is a matter of which ones are mounted open.

tsx
<Answer>
  <SupportLabel status={envelope.label} />
  <p className="headline">{a.headline}</p>
  {a.caveat && <Callout tone="warning">{a.caveat}</Callout>}

  <details>
    <summary>How Atlas got this ({a.sources.length} passages)</summary>
    <MarkedAnswer sentences={envelope.sentences} />
    <SourceList sources={a.sources} />
  </details>
</Answer>

The AI SDK's own material on multi-step generation and generative UI is the best worked reference for the streaming half of this, particularly for interfaces where tool steps are part of what gets disclosed rather than a single answer object.

The reasoning trace is not documentation

Teams that ship reasoning-first usually believe they are being transparent, and there is a second, quieter motive underneath: a streaming chain of thought fills the wait. Something is moving on screen for four seconds instead of a spinner, and the wait feels productive. It does work in that narrow sense, and it is a bad trade, because the filler is made of the one piece of content guaranteed to be irrelevant to the 88% of readers who never open it. Chapter 7 deals with what to put in that four seconds instead.

There is a real audience for the trace, and it is not the handler answering a policyholder. It is the compliance reviewer reconstructing a decision six weeks later, the support engineer working a ticket, and the engineer debugging a retrieval regression. Those readers arrive with a specific question, they arrive after the fact, and they are willing to spend more than one click. Designing layer 2 for them rather than for the primary user changes what it should contain: full chunk text rather than a summary, the query strings, timestamps, document versions, and a stable link that can be pasted into a ticket. A reasoning panel designed to impress the primary user tends to contain a fluent narrative of the model's thinking, which is the least verifiable artifact in the entire response and cannot be used as evidence of anything.

Disclosure also does not survive being copied. When a handler copies the answer into an email to a policyholder, the layers collapse to layer 0 plus whatever they highlighted, the marks vanish, and the caveat goes with them unless it was inline text. Products where copying is the dominant exit — which is most assistants embedded in a console — need layer 0 to be self-contained as plain text, not merely self-contained on screen. Atlas eventually made the blocking caveat part of the copied string and left everything else out, which handlers accepted because the alternative they had been living with was a citation marker [1] pasted into a customer email.

The click that gets spent

Sources open on 31% of answers, which makes them the most-used element in layer 1 by a wide margin, and it means a third of the trust the interface is buying rests on what happens after that click. A reader who spends an action on a source and lands on the first page of a forty-page PDF has learned something about the product, and it is not what the design intended. The next chapter takes that click apart: what a citation has to resolve to before following it is cheaper than re-answering the question, how staleness silently invalidates one that used to work, and which metric tells you whether any of it is being used.

AI product UX — read the current version at https://daviddominguez.dev/learn/en/ai-product-ux/lead-with-the-answer-and-keep-the-reasoning-one-action-away

Back to top