The agent I’ve been building talks to customers on phone and chat and takes real actions on their behalf: reboot a device, open a ticket, reschedule a dispatch, run a speed test. It’s barred from one thing: changing contact info on the account. That goes through the self-service portal, not the model.
The problem
Each turn produces two outputs: the reply text, and a log of what actions were attempted and whether they succeeded. Nothing keeps them in sync. The model generates the next plausible line of dialogue without checking the log first. “I’ve opened a ticket for you” is a plausible thing to say whether or not the ticket actually got created.
This showed up in the first few turns after a new action tool was wired in, not after an extended testing phase. The agent claimed success on an action the log showed had failed to fire.
Example
Customer asks to move an appointment. The scheduling action fails, timeout or a silent step failure. The model still says “you’re all set, moved to next week,” because that’s the plausible reply to that request. The customer hangs up believing it’s done. Nobody shows up to the old slot, and the next contact is a missed-appointment notice for an appointment the agent said it had already moved.
Why voice makes it worse
There’s no slack in a phone call for a pause while something double-checks itself; a pause reads as broken and people talk over it or hang up. A false claim that’s merely annoying in a chat window becomes something a person hears in a confident voice and believes outright.
The fix
A pass runs on the reply text after generation, before the customer sees it:
- Scan for language claiming a concrete action completed: ticket opened, appointment moved, reboot sent, speed test run.
- Check each claim against the action log for that turn. Strip any claim without a matching success.
- If the customer explicitly asked to change contact info, redirect to the portal link instead of a vague reply that reads as a yes.
- If stripping leaves nothing useful, fall back to a plain line: can help with that, don’t have it available right now.
- Log every strip. A repeated pattern on one action shows up later instead of staying buried in a transcript.
Where that leaves things
A narrow filter for one kind of confident overreach, not a smarter model. Its only job is making sure the agent doesn’t say “done” unless the log agrees.