The risk with an AI phone agent is simple: it may say something false, such as a price, a policy, or a promise, with full confidence. Before you trust one with your line, find out how often that happens and what the system does when it doesn't know the answer.
Answering Agent reviewed 17,724 inbound calls and flagged 12 as hallucinations, a 0.07% rate (99.93% accuracy). This post covers how those calls were scored, where the errors cluster, and what to check before you buy.
Study Overview: 17,724 Calls Analyzed
The calls came in around the clock across multiple business locations. The agent was connected to POS and CRM platforms like NXT Wash and Sonny's, so it could pull customer details, vehicle information, and membership status during the call instead of guessing.
The dataset covered home services, medical practices, law firms, and car washes. Call types included new-customer inquiries, existing-customer questions, appointment bookings, pricing requests, and availability checks.
How Calls Were Scored and Measured
Reviewers checked every call using transcript and audio analysis. They compared each answer against a Knowledge Base of business rules, pricing, service areas, and FAQs.
A call counted as a hallucination when the AI stated wrong or invented information, such as a made-up policy or price, instead of admitting it didn't know. Responses with confidence scores below 60–70% went to human review.
Each new deployment ran through a 14-day optimization period. When reviewers found an error, the team updated the Knowledge Base right away so the same mistake couldn't repeat.
Hallucination Rate Findings
Hallucination Rates by Call Type
Routine calls rarely go wrong. Hours, pricing, and service availability are lookups, and a well-connected agent either has the answer or doesn't.
Edge cases are where errors happen: referral discounts, holiday or emergency policies, and multi-step requests that depend on an earlier conversation. Your Knowledge Base review should focus on these. They are the questions your staff answer from memory and nobody ever wrote down.
Why a Wrong Answer Costs More Than "Let Me Check"
A fabricated answer can bind you. Air Canada's chatbot invented a bereavement fare policy, and the airline ended up in a legal dispute over honoring it. An agent that says "I'll have someone confirm that" costs you a callback. One that invents a refund policy can cost you the refund and the customer.
Performance Across Different Industries
Error Rates by Industry
Each industry breaks in a different place:
- Home services: the key job is catching urgency. Calls mentioning something like "burst pipe" were routed straight to staff. In this dataset, 15.9% of calls were flagged that way.
- Medical: most volume is scheduling, so accuracy depends on reading the calendar correctly.
- Legal: AI-handled intake saved 10–15 minutes per call. The agent still has to stay away from anything resembling legal advice.
- Car washes: most calls are about memberships and services. Pricing calls converted to unlimited memberships at 31%, and 23% of members calling to cancel stayed after a win-back offer.
Answering Agent Performance Advantages
Integrations are the main reason accuracy holds up. With OptSpot, NXT Wash, and Sonny's connected, the agent reads live membership and vehicle data instead of relying on a static script. Jacksons Car Wash uses it to promote offers and sign up SMS subscribers on inbound calls.
What This Means for Your Business
Reliability and Cost Comparison
Humans are accurate when they pick up, but they often can't pick up. Busy staff, lunch breaks, and after-hours calls all turn into missed calls and lost revenue. An AI agent answers every call, around the clock, and can handle many calls at once.
On cost, a full-time receptionist typically runs $35,000–$45,000 a year with benefits. Answering Agent is $199 a month. For the full math, see the cost-benefit breakdown and the AI vs. human receptionist comparison.
What to Look for in AI Answering Systems
- Accuracy measured on real calls: ask for the call count, how hallucinations were defined, and who scored them. A percentage with no method behind it tells you nothing.
- Escalation below a confidence threshold: the agent should hand off to a person when it isn't sure, instead of filling the gap with an invented answer.
- Integrations with your systems: live POS, CRM, and membership data is what keeps answers grounded.
- A correction loop: errors should become Knowledge Base fixes within days, so the same mistake doesn't repeat.
- Clear pricing and an exit: Answering Agent offers a 30-day money-back guarantee.
For rollout steps, use the 24/7 AI phone answering checklist.
Answering Agent vs. Typical AI Performance
| Metric | Answering Agent | Typical AI Systems | Human Receptionist |
|---|---|---|---|
| Accuracy Rate | 99.93% | 90–95% | High (when answering) |
| Answer Rate | 100% | 100% | ~25.9% |
| Availability | 24/7/365 | 24/7/365 | ~40 hours/week |
| Annual Cost | Less than 1 employee | ~$588–$2,388/year | ~$35,000+ |
| Call Capacity | Unlimited simultaneous | Varies | 1 call at a time |
Conclusion
No AI agent is perfect, so compare it against what actually happens to your calls today. Many go unanswered, and some get answered from memory. Pick a system with measured accuracy, hard escalation rules, and live data access. Then track its errors the same way this study did. To weigh the numbers for your own line, start with missed calls vs. AI answering and how to calculate ROI.
FAQs
What counts as an AI “hallucination” on a phone call?
A hallucination is when the agent states wrong or invented information as fact, such as a price, policy, or availability it has no source for. Misreading what the caller wants and answering confidently anyway counts too.
How can a business verify an AI phone agent’s accuracy before buying?
Run a pilot on your real calls. Review transcripts against your own pricing and policies, and track the error rate and how often calls escalate. Ask the vendor how they define and score hallucinations. Answering Agent's 99.93% figure comes from 17,724 reviewed calls.
When should an AI phone agent transfer a call to a human?
It should transfer when the call is urgent, when the request falls outside what it knows, or when its confidence drops below a set threshold. A handoff is always better than a guess.
Related Blog Posts
Book a walkthrough
See it handle your calls.
Book 20 minutes, or hear a sample call first.
