How to evaluate an AI agent before it talks to a customer
A test set of real conversations, scored on accuracy, tone, scope and hand-off — the evaluation loop that tells you whether an agent is ready, and catches regressions when you change the prompt.
27 Nov 2025 · 2 min read · by HeiSyn
Most agents are launched after someone chats with them for twenty minutes and says 'seems fine'. That is not evaluation; it is hope. An agent that will have thousands of conversations needs a test set — real questions, expected behaviour, a score — run before launch and after every change, the way any other software is tested.
Build the test set from reality
Pull two hundred real customer messages from the channel the agent will serve: support tickets, chat transcripts, call summaries, emails. Include the easy ones, the ambiguous ones, the angry ones, the out-of-scope ones and the ones that should escalate. For each, write what a good response does — the fact it must include, the action it must take, whether it must hand off.
Score on four axes
Accuracy: did it say true things and take the right action. Tone: does it sound like your business. Scope: did it stay inside what it is allowed to do. Hand-off: did it escalate when it should and not when it should not. A model can grade the first three against your rubric at scale; a person reads the failures and a sample of the passes.
Set a bar and hold it
Decide the pass rates before you look at results — say ninety-five percent on accuracy and scope, ninety on tone, one hundred on mandatory escalations. An agent that misses the bar is not launched, whatever the demo felt like. Every prompt or model change reruns the set; a change that drops a score does not ship.
Keep growing the set
Every escalation and every poor rating in production becomes a new test case with the correct behaviour written down. In six months the test set is the most accurate description of your customers' questions that exists anywhere in the company.
Questions people ask next
- How many test cases are enough?
- Two hundred real examples cover the common cases and enough of the rare ones to catch regressions. Grow it from production escalations; a test set that is a year old and unchanged no longer describes your customers.
- Who should grade the outputs?
- A model grades against the rubric at scale; a person reads every failure and a random sample of passes each run. Model grading catches the volume; human reading catches the rubric being wrong.
- How often should we rerun the evaluation?
- After every prompt or model change, and on a schedule — monthly at least — even without changes, because model vendors update behaviour. A quiet regression from a vendor update is caught by the scheduled run, not by customers.
Want this built for your team?
A free scoping call: we map your motion, name the number that should move, and quote a fixed-scope build.
