How to Keep AI Support Assistants Accurate and Prevent Hallucinations

You keep AI customer support accurate by limiting what the assistant can say to what your sources support, and by sending everything else to a person.

Hallucinations are part of how language models work. OpenAI’s 2025 research paper argues that models guess because standard training and evaluation reward a confident guess over admitting uncertainty. In support, that guess sounds like an official policy, a refund promise or a delivery date nobody agreed to.

Key takeaways

• Hallucinations come from how models are trained. OpenAI’s 2025 research says training and evaluation reward guessing over admitting uncertainty.
• Grounding answers in your own documents (RAG) reduces errors but does not remove them. Stanford found RAG-based legal tools still hallucinated on 17–33% of queries.
• You are responsible for what your bot says. A 2024 tribunal ordered Air Canada to pay CA$812.02 after its chatbot invented a refund rule.
• The controls that matter most: answer only from sources, fetch live data with tools, allow “I don’t know”, and keep people on money decisions.
• Measure accuracy every week with a fixed test set and a sample of real conversations, not just deflection rate.

A hallucination is an answer that sounds confident but is false or has no basis in your sources. NIST’s 2024 Generative AI Profile calls this “confabulation” and lists it as a core risk of generative AI. In a support queue, it usually takes one of four forms.

• Invented policy: states a rule your company never made. Cursor’s support bot told users a one-device limit was a security feature (2025).
• Wrong fact: gets a price, date or limit wrong, such as quoting a 60-day return window when yours is 30.
• Outdated answer: repeats a policy you have replaced, such as last year’s shipping page that was never archived.
• Misgrounded answer: cites a real source that says something else, such as linking the refund article when the article does not support the claim.

The fourth type is the hardest to catch. The answer links to a real help article, so it looks checked, but the article says something different. A reviewer who only checks that a citation exists will miss it.

Most wrong answers in production trace back to one of four causes. Only the first one is about the model itself.

The model guesses when it does not know

A language model predicts likely text. When it lacks the fact, a plausible answer is often more likely than “I don’t know”. OpenAI’s paper ties this to benchmarks that score a wrong guess and a refusal the same, so models learn to guess.

Retrieval finds the wrong passage, or nothing

Retrieval-augmented generation (RAG) means the system searches your documents first and gives the model the best-matching passages. If the search returns an old FAQ or an unrelated page, the model writes a fluent answer from bad material.

Your content is missing, conflicting or out of date

If two help articles give different refund windows, the assistant will quote one of them. It cannot know which policy your team actually follows.

The prompt pushes it to always answer

Instructions such as “always be helpful” or “never leave the customer without an answer” make guessing more likely. The assistant needs explicit permission to say it does not know.

Note: RAG reduces hallucinations but does not remove them. Stanford RegLab’s 2024 study found that RAG-based legal tools from LexisNexis and Thomson Reuters still hallucinated on 17% to 33% of test queries.

Two public cases show the cost. Neither involved a hack or unusual question. Each bot answered an ordinary customer and made something up.

• Air Canada, 2024: chatbot invented a refund rule. The website chatbot told a grieving customer he could claim a bereavement fare within 90 days after flying. The airline’s policy did not allow that. The tribunal ordered Air Canada to pay CA$812.02 (Moffatt v. Air Canada, CanLII).
• Cursor, 2025: support bot made up a login policy. Users logged out by a bug were told by an AI agent named “Sam” that one device per subscription was intended. No such policy existed. Users cancelled, and a co-founder corrected it publicly (Ars Technica).

The tribunal in the Air Canada case rejected the argument that the chatbot was responsible for its own words. The company is accountable for everything on its website, whether a page or a bot says it.

Wrong answers also cost you the channel. A 2026 Gartner survey found only 27% of customers would try a chatbot again after a negative experience. Gartner’s Eric Keller advises that GenAI should attempt resolution “only when confidence is high, with a clear path to human support.”

Grounding means the assistant may only state facts it found in your approved sources for this conversation. Everything it says should trace back to a passage or a system record.

1. Customer: “Can I return a sale item?” It arrives on store chat, email or WhatsApp.
2. Retrieve: the assistant finds the returns policy, sale items section. It uses approved, current documents only.
3. Draft: it writes the answer and cites the section. If no source is found, it says so and offers a person.
4. Check: does every claim match the passage? If yes, send. If no, hold the reply or hand off.

Use tools for anything about one customer

Order status, plan details, balances and stock change every minute. Do not paste them into the knowledge base. Give the assistant a tool, such as a read-only API call, that fetches the record after it verifies who the customer is.

Make “I don’t know” a valid answer

Tell the assistant to reply only from the passages provided, and to say so and offer a person when the passages do not cover the question.

Tip: If you are choosing between fine-tuning and retrieval for accuracy, start with retrieval. Fine-tuning changes tone and format well but is a poor way to teach facts that change. We compare the two in LLM fine-tuning vs RAG.

No single control is enough. Each one catches a different kind of error and adds its own cost, so pick them by risk.

• Retrieval grounding (RAG): catches answers built from the model’s general knowledge. Trade-off: only as good as your documents and search. Use it for every policy and product question.
• “No source, no answer” rule: catches guesses when search returns nothing useful. Trade-off: more handoffs in the first weeks. Use it for every assistant.
• Tools for live data: catch stale or invented order, account and stock details. Trade-off: API work, identity checks, access control. Use them for anything about one customer’s record.
• Cited sources: catch claims nobody can trace or review. Trade-off: longer replies, and clean article titles are needed. Use them for policy, pricing and eligibility.
• Groundedness check: catches claims the retrieved text does not support. Trade-off: an extra model call adds delay and cost. Use it for high volume or regulated topics.
• Human approval: catches wrong refunds, exceptions and promises. Trade-off: slower replies and staff time. Use it for money, legal and policy exceptions.
• Regression test set: catches accuracy drops after model, prompt or content changes. Trade-off: upfront work writing real Q&A pairs. Use it before every release.

Our view: every support assistant needs the first three rows and a test set. Add the groundedness check and human approval where a wrong answer costs money or breaks a rule.

Want a second opinion on how your support bot handles accuracy? Talk it through with our team: Book a free AI consultation.

1. Write down what the assistant may answer. List the topics it handles alone, the ones it drafts for approval, and the ones it always hands off.
2. Clean the knowledge base. Keep one current version of each policy, give each article an owner, and archive anything replaced.
3. Ground every answer. Retrieve approved passages, instruct the model to answer only from them, and show the source.
4. Add a refusal and handoff path. When no passage covers the question, the assistant says so and passes the chat to a person with the context.
5. Use tools, not text, for live data. Fetch orders and accounts through read-only API calls after checking who the customer is.
6. Require approval for money and exceptions. Refunds, credits, waivers and anything outside policy wait for a person.
7. Test before every change, then review weekly. Run your test questions after each model, prompt or content update, and read a sample of real conversations every week.

Note: Tell customers they are talking to AI. Under Article 50 of the EU AI Act, which has applied since 2 August 2026, people must be told when they interact with an AI system unless it is obvious. The Cursor bot signed its emails with a human-sounding name, which made its invented policy more believable.

Deflection rate alone hides hallucinations, because a confident wrong answer still counts as deflected. Track these alongside it.

• Answer accuracy on a test set. The share of your fixed test questions answered correctly. Rerun it after every change.
• Groundedness rate. The share of answers where every claim is supported by the cited source.
• Handoff rate. How often the assistant passes to a person. A sudden drop can mean it has started guessing.
• Re-contact rate. Customers who come back about the same issue within a few days, often a sign the first answer was wrong.
• Customer-reported errors. Complaints that quote the bot. Log each one against the source that caused it.

A small error rate adds up at volume. The calculator below estimates how many wrong answers reach customers, and what human approval on high-risk topics removes.

Example: 2,000 AI conversations a week at a 3% error rate gives 60 wrong answers a week. If 20% of those are on money or policy topics, that is 12. With human approval on, those 12 are stopped, so 48 reach customers each week, about 208 a month (a month is counted as 52 ÷ 12 weeks). This is an illustrative estimate. Use the error rate from your own weekly review of real conversations.

To weigh the cost of reviews and approvals against the hours the assistant saves, try our free AI ROI Calculator.

Our best example is Zenva, an AI assistant we built for ecommerce merchants. Zenva drafts each reply from live order data, and a support agent reviews and approves it before it goes out.

Two design choices keep its replies accurate. Order facts come from the store’s system at the moment of the reply, not from the model’s memory or a copied FAQ. And agents review and approve every draft, so an unsupported promise is caught before the customer sees it.

• Zenva, ecommerce replies: an estimate for one store with 2,000 customer emails a month is about 50 hours of support effort saved each month, with drafts built from order data and approved by an agent. Read the Zenva case study.
• Paymate, omnichannel support: for high-confidence email queries, first response time went from hours to under one minute, with other cases handed to human support. Read the Paymate case study.

The trade-off is speed. Approval adds a step, so we use it where errors are costly and let the assistant send routine, well-sourced answers on its own.

• Launching without a test set. You cannot tell whether a model or prompt change made answers better or worse.
• Letting the assistant fall back on general knowledge. When your documents don’t cover a question, it should say so, not fill the gap.
• Leaving old policies searchable. One stale article is enough for the assistant to quote the wrong rule.
• Hiding the handoff. Customers who cannot reach a person stop trusting the channel, and wrong answers go uncorrected.
• Judging success by deflection alone. A confident wrong answer counts as deflected. Pair it with accuracy and re-contact rates.

Can you stop an AI chatbot from hallucinating completely?

No. OpenAI’s 2025 research explains that language models guess when uncertain because training rewards it. You can make wrong answers rare and catch most of them by grounding answers in your sources, letting the assistant say it does not know, and reviewing samples every week.

What is the best way to reduce hallucinations in a customer support chatbot?

Combine four controls: retrieve answers only from approved, current content; refuse and hand off when no source covers the question; fetch live order or account data through tools; and require human approval for refunds and exceptions. Test against a fixed question set after every change.

Does RAG eliminate hallucinations?

No. Retrieval-augmented generation reduces them but does not remove them. Stanford RegLab’s 2024 study found RAG-based legal research tools from LexisNexis and Thomson Reuters still hallucinated on 17% to 33% of test queries. Retrieval quality, content quality and a refusal rule all matter.

Who is liable when an AI customer service bot gives wrong information?

In the cases so far, the company. In Moffatt v. Air Canada (2024), a British Columbia tribunal rejected the airline’s argument that its chatbot was responsible for its own statements and ordered it to pay CA$812.02. Treat your bot’s answers like any other page on your website.

How do you measure AI customer support accuracy?

Track answer accuracy on a fixed test set, the groundedness rate of real answers, handoff rate, re-contact rate and customer-reported errors. Review a sample of real conversations each week. Deflection rate alone is misleading because a confident wrong answer still counts as deflected.

Do I have to tell customers they are talking to AI?

In the EU, yes. Article 50 of the EU AI Act has required it since 2 August 2026, unless it is obvious. It also helps trust elsewhere: Gartner’s 2026 survey found 87% of customers say access to a human agent is essential when companies use GenAI in service.

Accurate AI customer support comes from limits, not from a smarter model. Ground every answer in approved sources, fetch live data through tools, let the assistant say it does not know, and keep people on decisions that cost money. Start this week by writing 50 real customer questions with their correct answers, and run your assistant against them.

See our AI customer support automation services, read how to deploy agentic RAG for customer service, or start with how customer support automation works.