Conversation earns its place when users cannot state what they want

A fintech platform ran two projects in the same half-year, both approved on the same argument, which was that the task was too complex for a form. The first was an investigation assistant for the fraud team: an analyst opens an alert, asks why it fired, gets an answer, notices something odd in the merchant history, asks about that, and follows the thread. The second was an assistant for the quarterly VAT return, a forty-step process that took the finance team eleven days each quarter and was universally described as the worst thing about the job.

The investigation assistant is still running and averages 7.4 turns per session. The VAT assistant was replaced after two quarters by a wizard, which is a fixed sequence of screens that shows which step you are on, what has been completed and how much is left. The complexity argument was true of both, and it predicted the wrong outcome for one of them, which makes it a bad separator.

The separator is whether the next request depends on the last answer#

Look at what happens across turns rather than at how many there are. In the fraud investigation, turn three cannot be written before turn two has been answered, because the analyst's next question is a function of the content of the previous response. They ask why the alert fired; the answer mentions that the card was used at a merchant the customer has never transacted with; the analyst asks about that merchant; the answer shows nineteen other flagged transactions in the last hour; the analyst asks who else was hit. Nobody could have drawn that path in advance, because the branching at each step depends on facts that arrive during the session.

The VAT return is a long path with no such property. Step 14 is the same step every quarter, it comes after step 13 every quarter, and the only thing that varies between quarters is the values. The whole tree is drawable at design time, which means it can be rendered: as a wizard that shows where you are and how far is left, as a checklist that survives being closed and reopened over eleven days, as a background job that pre-fetches the ledger extracts before anyone sits down. What the conversational version added was turn-taking overhead on a process that already knew its own shape, plus the loss of the one thing an eleven-day process most needs, which is a visible record of what has been completed.

So the question to ask is not "how many steps" but: can I write down the set of requests in advance? Where you can enumerate them — even a large set, even a hundred — you have a menu, and a menu renders better as a menu than as a text box. Where the set is genuinely open, and where the user's ability to name what they want depends on having seen a response, the turns are doing real work and conversation is the surface that supports them.

The two on the right of that split are worth stating plainly as the answer to the interview version of this question, because "chat suits complex tasks" is what most people say and it puts the VAT return on the wrong side. Chat is right for fraud investigation and for exploratory analysis over a warehouse. Chat is wrong for the VAT return and for the equipment order. The property that divides them is whether the user can state the request before seeing a response — not how many steps the work takes, and not how hard it is.

Multi-step is the tempting proxy, and it is the wrong one#

It is worth being concrete about what the wizard gave the VAT team back, because "use a wizard" sounds like a downgrade and was not. Eleven days of work by four people is not one sitting, so the state has to be durable and inspectable: who completed step 19, when, and with what value. The wizard rendered that as a checklist the whole team could see, which meant two people could work different branches without colliding and a returning user could tell at a glance where the work stood. It also produced the artefact the auditors asked for at year end, which was a record of who signed off on each figure. The conversational version had all of that information somewhere in a transcript and none of it in a form anyone could act on, and the reason is structural: a conversation's state is a history, and an eleven-day process needs a status.

The reason the complexity heuristic is so attractive is that it correlates with something real. Open-ended tasks are usually also complex, so anyone who has only seen a handful of examples will find complexity predicts the answer well enough. It fails on exactly the cases that cost the most, which are the long, painful, expensive processes with a fixed shape — tax filing, compliance reviews, onboarding a corporate customer, migrating a data set. Those are the processes with the biggest pain and the loudest stakeholders, which is why they attract assistant proposals, and they are precisely where a conversational surface removes the two things a long process depends on: state you can see, and a place to stop and come back.

The inverse error also happens and is quieter. A request can be trivially simple per turn and still be open-ended in aggregate. "What does this error mean" against a log platform is one short question, and the set of possible next questions after any given answer is unbounded. Nobody would call it complex; it belongs in a conversation anyway.

There is a second common substitute for the real property, which is unstructured input. Someone points out that users type free text — a description, a paste, a photo caption — and concludes that the surface must be conversational. Free-text input is an input-parsing question and it is answerable inside a form, as the expense report in the earlier chapter showed. What makes a surface conversational is turns that carry information forward, not a text field.

Measure open-endedness rather than asserting it#

Open-endedness is a distribution, not a boolean, and the distribution is measurable before you build anything. If some version of the workflow already exists — a support queue, a search box, an internal Slack channel where people ask the team — sample a few hundred real requests and count how much of the mass falls into a small number of intents — distinct things people are trying to get done, grouped regardless of how each person phrased it, so that "revenue last week", "how did we do" and "weekly numbers please" are one intent and not three.

# 200 requests sampled from six weeks of the #data-questions channel.
# Cluster, then ask how much mass the head holds.
labels = classify(requests, taxonomy=DRAFT_INTENTS, model="claude-opus-5")
counts = Counter(labels)

head = counts.most_common(10)
head_share = sum(n for _, n in head) / len(requests)

print(f"top-10 intents cover {head_share:.0%} of requests")
for intent, n in head:
    print(f"  {n/len(requests):5.1%}  {intent}")

top-10 intents cover 85% of requests
  22.0%  weekly revenue summary
  11.5%  why did metric X move
   9.0%  list top accounts by <dimension>
   8.5%  export <table> for date range
   7.0%  which customers churned last <period>
   6.5%  status of pipeline run
   6.0%  compare this <period> to last
   5.5%  who owns <dataset>
   4.5%  definition of <metric>
   4.5%  is <dashboard> stale
  15.0%  (tail: 71 distinct requests, none above 1%)

An 85 percent head — the share of all requests that the ten commonest intents account for, with everything rarer sitting in the tail — is not an argument for chat, and it is not an argument against it either. It is an argument for a menu covering ten intents and a text box for the fifteen percent that does not fit — which is a different product from a chat assistant, because the ten most common jobs become one-click things with a stable, linkable, shareable result, and the conversation stops carrying traffic it was never good at. The fraud investigation tool run through the same exercise came back with a head share of 31 percent and a tail of 340 distinct question shapes, which is what a genuinely open request space looks like.

If nothing exists to sample, the cheap substitute is to write the menu yourself and see how hard it is. A team that can list the twelve things users will want in twenty minutes has just discovered they have twelve things, and a team that fills a whiteboard and keeps going has learned something too.

Open-endedness does not reopen a question the gate already settled#

One boundary matters enough to state explicitly. The paraphrase question from the earlier grid runs first, and a high open-endedness score does not overturn it. A lawyer researching whether a particular indemnity survives termination is in an unambiguously open request space — they cannot state what they are looking for until they have read something, and their next question is entirely a function of the last answer. Every property in this chapter says conversation. The gate still holds, because the answer they need is the clause, and a fluent restatement of the clause is not a substitute for it no matter how the navigation is arranged.

What that leaves is a legitimate hybrid rather than a contradiction: conversation as the navigation, and the governing text as the answer. The turns find the passage; the passage is what gets shown. Teams that miss this build a research assistant that summarises beautifully and quietly becomes the thing people cite, and the failure mode of that arrangement is severe enough to deserve its own treatment.

Two honest limitations remain on the line this chapter draws. The first is that open-endedness is a property of a user population and not of a task, so the same feature can sit on different sides for a specialist and for an occasional user. A fraud analyst investigating an alert needs open exploration; a customer-service agent looking at the same alert wants three buttons, because their question really is one of three. Segment before you decide, and be prepared to ship both.

The second is that request distributions move. A tool that launches with a 31 percent head share often has a 70 percent head share a year later, once users have learned what it is good at and stopped asking it the other things. That is a surface change waiting to happen — the moment the head gets fat, the top intents deserve to be first-class controls, and the conversation goes back to being the escape hatch. Re-running the intent-coverage count quarterly is what turns that from a hunch into a scheduled decision.

The next chapter takes the gate case seriously, in the setting where getting it wrong has consequences that no completion-rate dashboard will show you.