03 · What I do
Intent models, confidence policy, repair, handover and the evals that prove it.
Four AI products from problem definition to production, one of them a customer facing agent inside an enterprise. Voice and chat, multilingual, on real telephony, with the diary and business constraints wired in underneath.
The control on the right is the technical core of the discipline: one utterance, three confidences, three different permitted actions. Everything else on this page exists to make that decision reliable and measurable.
Conversation design is the design of what a chatbot or a voice agent says and does. Not the AI model itself, which is bought or hired: what it is allowed to do, how certain it has to be before it acts, what it says when it is unsure, and how somebody reaches a human.
Three words the page uses. An intent is what the person is trying to do, as the system classifies it. Confidence is how sure the model is that it read them correctly, on a scale. An eval is a fixed set of test conversations run after every change, so a fix in one place cannot quietly break something three steps away.
What I actually produce
Nine artefacts, and an engineer can build from every one of them.
If your spec names deliverables, these are the ones it probably means. None of them is a script, and the wording of what the product says is the last thing written rather than the first.
Speech, not chat
Six constraints that only exist once it is a phone call.
A chat product can show a list, undo a message and take its time. On a call none of that is true, and a design that ignores it fails in the first thirty seconds.
How it gets measured
A behaviour is not shipped because it demos well.
Every behaviour has a case set with the expected outcome written down, run before and after a change. Two of ParrotB's, with their real counts:
caller states a problem, agent extracts the job
slot held across turns, diary checked before commit
above threshold the agent stops being useful and transfers
out-of-scope requests handed over rather than attempted
What the numbers are and are not. These are pass rates against written cases, not satisfaction scores and not a claim about every call. A case set is a regression suite: its job is to tell you a change made things better rather than only different.
Why 86% ships and 97% ships. Because the two behaviours carry different costs when they are wrong. A missed detail in intake is recovered by one question. A booking committed against a diary that cannot take it is somebody's afternoon, so its gate is higher and the agent is not permitted to commit until the constraint check returns.
The gates. Emergency detection and refusal scope are pass or fail rather than a percentage. A behaviour whose failure is unbounded does not get a tolerance.
What has actually shipped
One inside an enterprise, three of my own, all in production.
Not prototypes, and not a proof of concept that never left a laptop.
Customer facing agent. Sole designer, problem definition through to production. The brief was support load, not technology: a large share of contacts were the same small set of questions.
- Wizard of Oz testing with real users before anything existed that could hold a conversation
- Traffic analysis to pick the flows worth automating, rather than the interesting ones
- Prototype of the main use flow, taken through to production
- A written refusal scope: money, contract and account consequences always escalate
Voice receptionist for one-person trades, on real telephony, in four languages. The failure was never the model misunderstanding the caller. It was bookings that fit the sentence but not the working day.
- Intent model with an explicit emergency gate and transfer rules
- Availability spoken as hour windows, exact start time stored underneath
- Commit blocked until job length plus travel plus buffer are checked against the diary
- Business vocabulary, services, pricing, service area and booking rules as configuration
- Eval set per behaviour, run before and after every change
Receipt extraction where every line carries its own confidence and only the lines the model is genuinely unsure about are ever returned to the person.
- Per-field confidence rather than per-document, so review is targeted
- Arithmetic used as a check: if the lines do not sum, the total is not trusted
- Correction memory, so the same mistake is not made twice
- The pantry layer underneath it is shared by two of my other products
A health product where the design problem was restraint: how much the system should infer, and the point at which inference becomes presumption.
- The interesting constraint was what not to personalise. A product that adapts to a bad week teaches you that bad weeks are the norm
- No grading, no streak punishment, no inferred judgement about the person
What I have worked on and with
Named, because a spec usually names some of it.
I am a designer, not the engineer. What this list means is that I have designed against these constraints and can hold a technical conversation about them without a translator in the room.
The interactive version of the intent model.
The conversation design page carries a live intent model from ParrotB: five things a caller might say, every reading the system considered with its confidence, and what it does with the ones it rejected. The fourth is the one to look at, where nothing clears the threshold and the winning action is to ask a question.
The other five