Chat hands the user the job of discovering what the system can do

An internal tools team at a 900-person logistics company replaced their purchase-request form with a chat assistant, and six weeks later the finance lead asked why her queue had gone quiet. The form had been universally disliked: eleven fields, four of them optional, a cost-centre dropdown nobody understood, and a validation step that rejected the whole submission if the vendor tax ID was malformed. The assistant was a clear improvement on every dimension the team measured while building it. It parsed free text well, it handled misspelled vendor names, it asked sensible follow-up questions, and in a manual review of 200 conversations the model gave a correct answer in 194 of them.

The numbers that mattered were elsewhere. In the quarter before the change, 94 percent of started purchase requests were submitted, at a median of 41 seconds from open to submit. In the six weeks after, 61 percent of started conversations ended in a submitted request, at a median of 3 minutes 12 seconds. Eighteen percent of the requests that did get submitted bounced back from finance for a missing or wrong cost centre, against 2 percent before. The model was answering well and the job was completing worse.

The thing they had built is what everybody at the company called a chatbot: a feature whose entire interface is a text box, where a person types a request in ordinary language, a language model answers, and the exchange continues in turns until the job is done or the person gives up. Underneath it sits a model that is genuinely good at reading a badly phrased sentence. Around it sits nothing — no fields, no labels, no list of the things it can be asked for. The eleven-field form it replaced was the other kind of interface, the kind that shows its whole structure on screen before anyone types a character.

Those two are surfaces, and which surface a feature gets is a decision separate from what the model behind it can do. A surface is the shape an interaction takes: who starts it, what the user can see before they act, when the result appears, and where it lands. The same extraction model can be reached through a text box, through a set of fields it has already filled in from a photo, through a suggestion that appears while somebody is typing, or through a job that runs when a record is created and writes its answer where the user was going to look anyway. Choosing among those is what this topic is about, and it is a live question because the text box has been the default since the first widely used chat assistants shipped — a conversation is the surface a language model arrives with, so it is the one people ask for by name, usually in a sentence that starts "can we add a chatbot for".

What the choice governs is interaction cost: the work a person has to do to get a result, counted in the things they must remember, look up somewhere else, type, re-read and correct, rather than only in the seconds they spend waiting for a response. The logistics team moved a job from one surface to another, the model answered well on the new one, and the interaction cost went up by enough to lose a third of the requests. Where exactly it went is visible in a single session.

The model answered every question correctly and the request still did not get filed#

Here is one of the abandoned sessions, reconstructed from the event log. Nothing the assistant says in it is wrong.

14:02:11  user  I need to order two monitors
14:02:13  bot   Happy to help with that. Roughly what will they cost?
14:02:31  user  about 300 each I think
14:02:33  bot   Got it — $600 total. Which budget should this come out of?
14:02:58  user  the design team budget
14:03:02  bot   I couldn't find a budget called "design team budget".
                Do you have the cost centre code?
14:03:40  user  no
14:03:43  bot   No problem — it's on your profile page in Workday, under
                Organization. Once you have it we can carry on.
14:05:20  user  CC-4471
14:05:24  bot   Thanks. Who is the vendor, and do you have their tax ID?
14:05:51  user  probably dell? do I need the tax id
14:05:54  bot   For purchases over $500 the tax ID is required. If Dell is
                already an approved vendor I can look it up — is this a
                UK or US entity?
14:06:38  [session ends]

That session is not an outlier, and the shape of the drop-off says where the cost sits. Plotting abandonment against turn number across the six weeks, 9 percent of sessions ended after the assistant's first reply, 22 percent ended on the turn where the cost centre was requested, and a further 19 percent ended on the turn that asked for the tax ID. Both spikes land on a question the user could not answer from where they were sitting. Almost nothing was abandoned during the parts the model was doing well — parsing quantities, matching vendor names, computing totals — which is the clearest available evidence that the losses were not about answer quality.

The old form did four things in that transcript's place, and it did all four before the user typed anything. It listed every field the request needed, so the user knew up front that a cost centre and a tax ID were coming. It rendered the cost centre as a select containing the twelve centres this user had spend authority over, defaulted to the one on their profile, so "the design team budget" was never a phrase anyone had to produce. It marked the tax ID as conditionally required with the threshold written next to it. And when validation failed, it failed on the field, in place, with the rest of the entered data still on screen.

The assistant reproduced none of that, and it was not because the model was weak. A text box publishes nothing about itself. Whatever the system knows about required fields, permitted values, thresholds and approval chains is on the server, and the interface's only channel for revealing it is a reply to something the user has already guessed at. That is a structural property of the surface rather than a quality of the model behind it.

A form publishes its contract; a text box publishes a blinking cursor#

Interface designers have a name for the general principle: recognition over recall. A control that displays its options asks the user to recognise the right one; a blank field asks them to retrieve it from memory. The purchase form was an unpleasant instance of recognition — too many fields, badly grouped — and the chat assistant replaced it with pure recall. Comparing the two along the work each one demands makes the trade visible.

Four distinct kinds of work moved from the system to the person. The first is capability discovery: whether the assistant can do this at all, and how much of it. The second is vocabulary — the user says "design team budget" and the system knows CC-4471, and nothing on screen bridges the two. The third is field completeness, which the form held for the user and the conversation makes the user hold in their head; the eighteen percent bounce-back rate is that work being done badly under pressure. The fourth is error localisation. A form says which field is wrong. A conversation says something is missing, and the user reconstructs the state of the request by scrolling.

There is a fifth that is easy to miss because it has no failure message attached. In a form, the state of the request is visible: filled fields are filled and empty ones are empty. In a conversation, the state lives in the assistant's memory and in the scroll-back, and the user has no reliable way to check what the system currently believes. That is why chat surfaces so often grow a summary card before the confirm step — the card is a form, arriving late, doing the job the form would have done at the start.

Prompt starters publish six capabilities out of forty#

The standard patch for discoverability is a row of suggestion chips: three or four example prompts under the input box, sometimes refreshed per context. They help, and the size of the help is worth being honest about. A row of chips publishes three to six capabilities of a system that typically has dozens, so it fixes the first session and leaves the long tail exactly where it was. It also has a shape problem, in that a small unordered set of examples is a menu with no grouping, no labels and no indication of what lies beyond it. Once you find yourself designing a good set of chips, you are designing a menu inside a text box, and a menu rendered as a menu is easier to scan.

None of this argues that the model was the problem or that a better model would fix it. Run the same trace with a model that never misunderstands anything, and the user still does not know the tax-ID threshold until they are asked, still cannot name their cost centre, and still leaves at 14:06:38. The discoverability cost is paid by the surface, not by the inference. This is worth separating cleanly, because "the completion rate is low, let us improve the prompt" is the most common wrong turn taken after a launch like this one, and it can absorb a quarter without moving the number.

Nor is the cost about typing speed. Voice input removes the typing and leaves every one of the four transfers intact; the user still has to know what to say. And nothing here says conversation is the wrong surface everywhere — a later chapter draws that line properly. The claim is narrower: where the required inputs, the permitted values and the approval steps are knowable in advance, a conversation charges the user to rediscover them, one turn at a time, every session.

The tell in a request that arrives as "add a chatbot"#

Requests rarely arrive as "we need a surface for structured intake". They arrive as "add a chatbot for expenses", or "can we put an assistant on the policy docs", and the sentence carries an implementation with it. The useful move is to ignore the noun and ask what the completed task looks like, then check one thing: can you write down the fields?

If you can enumerate the inputs the system needs — and for purchase requests, expenses, leave, access provisioning, refunds under a threshold and most internal intake, you can — then the interface can enumerate them too, and a conversation is spending turns to collect what a form could display. If you can also name the approval steps in advance, the case gets stronger, because approval routing is exactly the kind of thing a text box cannot show and a form can. The purchase request had eleven fields and a two-step approval chain, both known at design time, both of which the chat assistant re-derived from scratch in every session.

The same question also identifies the users for whom chat genuinely was an improvement, and there were some. Eleven people in the logistics company filed more than twenty purchase requests a month, knew their cost centre from memory, and could produce a complete request in one sentence. For them the assistant was faster than the form, because everything the form displays is something they already know and did not need shown. That is a real segment with a real preference, and it is roughly 3 percent of the user base — which tells you what to build for it, namely a fast path for experts rather than the default surface for everyone. Surface decisions made from the behaviour of the most fluent users are a recurring way to arrive at chat by accident.

The counter-question keeps you honest: what does the user get from the conversation that the form withheld? Sometimes the answer is real. In the logistics team's case, the assistant's genuine advantage was that it accepted "two monitors, about 300 each, Dell" as a single sentence rather than five fields, which is a parsing win and not a conversation win — and a parsing win can be delivered inside a form, as a paste-anything box above the fields that fills them in. That is a different surface again, and it is one of five worth having names for.

Naming the cost is only half the job. A team that concludes "not chat" still has to decide what to build instead, and "a form" is not always the answer — the enrichment job that runs on every new lead is not a form, and neither is the suggestion that appears while someone types. The next chapter lays out the five surfaces actually available and the questions that pick between them, so the decision stops being a matter of taste.