Status: experimental — a working decision aid, not validated guidance
How to use · Threshold v0.1
Deciding whether (and how) to ship conversational AI
Threshold: An AI-chatbot feature gate.
Threshold is an interactive walk-through of a five-stage decision — from “should this exist at all?” to “what must it pass before launch?” For designers and PMs evaluating a request to add a chatbot or conversational agent to a product.
Audience · designers & PMsFirst pass · 20–40 minBring · an SME for high-stakes calls
A conversational surface is a cost, not a default. Most of the gate is about refusing or narrowing the request.
So treat a “no” or a “narrow it” as a successful outcome, not a failure. Your answers save in your browser as you go — you can leave and come back.
When to reach for it
A request has come in to add chat, a copilot, an assistant, or a companion to a product.
You’re scoping such a feature and want the risk surface on the table before build, not after.
A model, prompt, or persona change is planned for a chatbot you already ship — jump to Stage 5.
How to move through it
Work the stages in order the first time — each one feeds the next. Use the Reference button any time for the 12 attack patterns and the SweetPause worked example.
0
Does it need chat?
List the real jobs users would do; tag each structured-UI or needs-conversation.
You getA verdict. If every job is structured-UI, stop — you don’t need a chatbot.
1
Scope
Choose utility or support / companion, on purpose.
You getA recorded scope and boundary. Default to the narrowest that does the job.
2
Which harms are live?
Answer five yes/no questions and watch the harm board light up and prune.
You getThe harm directions to design against and test for.
3
Pass behaviors
Edit the seeded behavior for each live harm.
You getYour spec: the observable behaviors the agent must exhibit.
4
Launch gate
Set the pass criteria and the over-correction ceiling.
You getThe acceptance test the feature must clear to ship.
5
After launch
Read the standing obligations.
You getThe re-run and monitoring discipline.
6
Decision brief
Copy or print.
You getA record to paste into a spec, ticket, or launch doc.
What to do with the output
The deliverable
The decision brief from Stage 6 is what you hand off. Paste it into the feature’s spec or ticket so the scope decision, the live harms, the pass behaviors, and the launch gate travel with the work. The Stage 4 gate is not a report you file after — it’s the condition for shipping. If the feature can’t clear it, it doesn’t ship, or it ships with the failing capability disabled.
Three things to get right
Write the pass behaviors before you build
An unbuilt feature can be designed around them instead of retrofitted. This is the advantage most teams don’t get — use it.
Bring the right people to the high-stakes rows
Anything marked high-stakes — crisis, vulnerability, safety handoff, false certainty — needs clinical and lived-experience review to set the pass criteria. Not designer intuition, and not an LLM judge.
Passing ≠ safe
It means no known, scoped structural failures. Out-of-scope classes — deception, prompt injection, data security, bias, and anything involving minors — remain untested and need their own safeguards.