Do not start a voice-agent test with, “Can it have a natural conversation?” Start with, “What happens when the agent cannot finish the work?”
Take a caller reporting that a prescription refill has not arrived. The agent needs to check the request, explain what it finds, and either solve the problem or place it with a person who can. A smooth greeting is not the test. The test is whether the caller has a refill, a credible next step, or a reason to call again.
Test one complete call state
For each important call type, define a scenario with four parts:
- what the caller needs
- the relevant account and system state
- what the agent may say
- what must be true when the call ends
For the refill scenario, test a request that is ready, one that needs pharmacist review, one that has no matching prescription, and one where the pharmacy service is unavailable. The expected language must change with the state.
An agent may confirm a refill only when the pharmacy system confirms it. If review is required, the caller needs an honest answer and a real handoff or callback plan.
Include the moments that make voice hard
Voice adds conditions that text tests often ignore:
- the caller interrupts the agent or changes the request
- a name, number, or date is heard incorrectly
- the caller waits through silence while a tool responds
- a transfer target is unavailable
- an important detail appears early in a long call
Use these conditions against a real customer goal. They are not theatrical edge cases. They change whether a caller trusts the next step.
Test the recovery, not only the agent
When the pharmacy service is unavailable, the test should check more than the fallback sentence. It should prove that the follow-up has an owner, a deadline, and the context the receiving person needs.
Have an operations lead review the scenario. Could they use the record to call the customer back without asking the same questions again? Then have an engineer run the same scenario. Can the test fail if the agent claims the refill is ready when the tool state is unknown?
That is the useful pairing: one test for the person who owns the customer, and one for the system that speaks to them.
Keep the failures that taught you something
The best test cases come from calls the team had to recover. Preserve the caller goal, the tool state, the words the agent used, and the action that fixed the problem. Do not keep a vague ticket such as “refill workflow failed.” It will not tell the next engineer what the agent must do differently.
The management question
Which real call from last month would you be least comfortable replaying today?
Make that the next test. It is probably closer to the customer risk than another happy-path demo.