Which Customer Questions Should AI Never Handle? A Simple Decision Framework

Let AI customer service answer questions that have one correct, written answer. Keep a person on anything where the answer depends on judgment, money, law or someone’s wellbeing. Most queues sit somewhere in between, and that middle lane is where AI drafts and a person sends.

Customers draw the same line. In a Gartner survey of 3,566 customers published in August 2026, 50% said service is easier when companies use generative AI. Yet 87% said an option to reach a human agent is essential.

Key takeaways

• AI customer service is safe to automate when a question has one correct, written answer and a wrong reply costs little.
• Four checks decide the lane: costly if wrong, legal right, vulnerable customer, needs judgment.
• Air Canada, Cursor and New York City all had bots answer questions a person should have owned.
• If the AI cannot cite an approved source, it should hand over, whatever the topic.
• 87% of customers in Gartner’s 2026 survey said a route to a human is essential.

The public failures share a pattern. The bot was not asked a hard technical question. It was asked something where the honest answer was “a person needs to decide this,” and it answered anyway.

Air Canada, 2024 (failed check: vulnerable customer)

A customer asked the website chatbot about bereavement fares after a death in the family. The bot described a refund rule that did not match the airline’s policy. A British Columbia tribunal held Air Canada liable and awarded about CA$812 (Civil Resolution Tribunal decision). The lesson: grief plus money is a person’s job, and the company owns what the bot says.

Cursor, 2025 (failed check: needs judgment)

Users were being logged out when switching devices. The AI support agent, signing emails as “Sam”, said this was a one-device policy. No such policy existed. Users canceled subscriptions before a co-founder apologized (Ars Technica, April 2025). The lesson: when no document has the answer, the bot must say so and hand over.

NYC MyCity, 2024 (failed check: legal right)

New York City’s business-advice chatbot told owners they could take a cut of workers’ tips and that landlords could refuse housing vouchers. Both answers contradicted the law (The Markup, March 2024). The lesson: legal and regulatory advice needs a qualified person, even when the bot draws on official pages.

Note: A disclaimer did not save any of them. The NYC bot carried a warning that it might produce incorrect content, and the Air Canada tribunal still held the airline responsible for what its chatbot said.

These seven categories are our view of where a person should make the final call. Each one fails at least one of the four checks in the next section.

1. Bereavement and hardship. “My husband died last week. What happens to his plan?” The customer needs care and often an exception. A policy quote reads as cold, and a wrong one is costly.
2. Disputes and formal complaints. “I never authorized this charge.” These can trigger legal duties and deadlines. A missed dispute is a compliance failure, not just a bad chat.
3. Legal, tax or medical advice. “Can I deduct this on my taxes?” The answer depends on facts the bot cannot verify, and a confident wrong answer causes real harm.
4. Safety and threats. “The heater you sold me is sparking.” Product safety, self-harm or threats need a trained person and often a reporting process.
5. Account security and identity changes. “Change the bank account for my payouts.” These are common fraud routes. Verification should follow a fixed process with a human check.
6. Exceptions to policy. “I am two days past the return window. Can you help?” Goodwill decisions need judgment and budget authority. A bot that improvises creates a precedent.
7. Requests for a person. “Let me talk to someone.” Gartner’s 2026 survey found 87% of customers see a human option as essential. Honor the request at once.

Tip: “Never handle alone” does not mean AI stays out. It can still detect the category, collect the order number and pass a summary to the agent, so the customer does not repeat themselves.

Run every question type through four yes-or-no checks. One “yes” moves it out of the fully automated lane. A “yes” on legal rights or vulnerability moves it straight to a person.

1. Costly if wrong. Would a wrong answer move money, lock an account or expose personal data? If yes: AI drafts, human sends.
2. Legal right. Is the customer raising a dispute, a formal complaint, a cancellation right or a data request? If yes: human owns.
3. Vulnerable customer. Does the message mention bereavement, illness, financial hardship, abuse or a safety risk? If yes: human owns.
4. Needs judgment. Does the right answer depend on an exception, a negotiation or a policy that is not written down? If yes: AI drafts, human sends.

A fifth condition sits on top of all four: the AI must be able to cite an approved source for its answer. If it cannot, it should hand over, whatever the category. That rule alone would have stopped the Cursor incident, where the bot explained a bug with a policy no document contained.

Definition: Grounding means the model answers only from documents you supply, such as your policy pages or order records, rather than from what it learned in training. A grounded answer can be traced to its source.

Here is how six customer messages score on the four checks and which lane each lands in. The examples are illustrative.

• “What time does your Bandra store close on Sundays?” All four checks no. Lane: AI handles. Answer from the store-hours page and link to it. One correct, written answer, nothing to decide.
• “Where is my order 4471?” All four checks no. Lane: AI handles. Look up the order status by API and share the tracking link; hand over if the order is lost. A read-only lookup with a clear source of truth.
• “My parcel arrived broken. I want a refund.” Costly if wrong: yes. Lane: AI drafts, human sends. Collect photos and the order number, check the refund policy, draft a reply; an agent approves the refund. Money moves, but the policy is written down, so a draft saves most of the effort.
• “My father passed away. Can I close his account and get the balance?” Legal right and vulnerable customer: yes. Lane: human owns. Acknowledge, explain that a specialist will help, and route with context; do not quote policy. The Air Canada case started with a question like this.
• “I was charged twice. I am disputing this with my bank.” Costly if wrong and legal right: yes. Lane: human owns. Flag it as a dispute, log it with a timestamp, and route it to the billing team. The CFPB warned in 2023 that bots may miss when a customer is raising a dispute.
• “Your app is slow and I am thinking of canceling.” Needs judgment: yes. Lane: AI drafts, human sends. Gather device and plan details and draft troubleshooting steps; a person decides on any retention offer, because retention offers are judgment calls outside written policy.

The middle lane is the one most teams skip. It is also where most of the time savings in a regulated queue come from, because the agent edits a draft instead of writing from scratch.

• AI handles: the AI sends, answering from a cited source. Typical questions are hours, order status and how-to. The main risk is outdated source documents. Log the answer and the source used, and watch repeat contacts within 7 days.
• AI drafts, human sends: an agent sends after review; the AI collects details, checks policy and drafts. Typical questions are refunds, claims and cancellations. The main risk is agents approving without reading. Log the draft, edits and approver, and watch the edit rate on drafts.
• Human owns: an agent or specialist sends; the AI detects, summarizes and routes. Typical questions are disputes, grief, safety and legal. The main risk is slow or lost hand-offs. Log the category, summary and hand-off time, and watch the time to reach a person.

This comparison is our assessment of typical setups, not a survey.

In the EU, Article 50 of the AI Act has applied since August 2, 2026. Chatbots must tell people they are talking to an AI unless that is obvious. It applies to companies outside the EU when their system is used by people in the EU. Our guide to Article 50 for chatbots covers what that means in practice.

In the US, the Consumer Financial Protection Bureau’s 2023 report on chatbots in consumer finance warned that bots may fail to recognize when a customer is raising a dispute, and that firms risk breaking consumer finance law when that happens. India’s Digital Personal Data Protection Rules, notified in November 2025, add duties on how personal data in chats is handled.

Do

• Tell customers at the start that they are talking to an AI.
• Show a visible route to a person in every conversation.
• Detect dispute and complaint language and route it with a timestamp.
• Log every answer with the source it used.

Don’t

• Give a bot a human name without saying it is an AI.
• Make customers repeat a phrase to reach an agent.
• Let the bot state a policy it cannot cite.
• Rely on a disclaimer to cover wrong answers.

Warning: This is general information, not legal advice. Check sector rules with counsel, especially in finance, insurance, healthcare and telecoms.

1. Export three months of tickets. Group them into your top 20 question types by volume.
2. Run each type through the four checks. Do it with a support lead and someone from legal or compliance in the room.
3. Assign a lane to each type. Write it down as a routing policy, with an owner for each lane.
4. List the trigger phrases. For the human lane, note words like “dispute”, “passed away” or “unsafe” that should force a hand-off.
5. Ground the first lane. Connect the AI only to approved documents and systems, and block answers without a source.
6. Design the hand-off. Pass the conversation summary and collected details so the customer does not repeat themselves.
7. Review transcripts every week. Read a sample from each lane. Move a question type to more automation only when the transcripts support it.

Step 5 depends on good retrieval. Our guide to agentic RAG for customer service automation covers how to ground answers in your own documents.

Want a second opinion on where your lanes should sit? Book a free AI consultation.

1. Measuring containment only. A customer who gives up looks like a contained chat. Track repeat contacts and time to reach a person.
2. Hiding the human option. Gartner found in 2026 that only 27% of customers would try a chatbot again after a bad experience.
3. Routing by topic, not by message. A refund question can turn into a bereavement case mid-chat. Check every message, not just the first.
4. Setting the lanes once. Policies and products change. Review the routing policy whenever a policy page changes.

New to the topic? Start with what customer support automation is and how it works.

What customer service tasks should not be automated?

In our view, anything involving bereavement or hardship, payment disputes and formal complaints, legal or medical advice, safety issues, account security changes, policy exceptions and direct requests for a person. AI can still detect these, collect details and route them, but a person should make the final call.

What are the pros and cons of automated customer service?

Pros: instant answers at any hour and lower cost for routine questions. Cons: wrong answers carry legal risk, and customers dislike being blocked from a person. Gartner’s 2024 survey of 5,728 customers found 64% would prefer companies did not use AI for service.

Do AI chatbots have to say they are AI?

In the EU, yes. Article 50 of the AI Act has applied since August 2, 2026 and requires chatbots to inform people they are interacting with AI unless it is obvious. Outside the EU, disclosing is still good practice. Cursor’s 2025 incident showed what happens when users assume a bot is a person.

Is a company liable for what its AI chatbot says?

In the leading case, yes. In Moffatt v. Air Canada (2024), a British Columbia tribunal held the airline responsible for its chatbot’s wrong bereavement-fare advice. It rejected the argument that the chatbot was a separate entity responsible for its own actions.

When should an AI agent hand over to a human?

When any of the four checks is a yes, when the customer asks for a person, or when the AI cannot cite an approved source for its answer. Gartner advises that generative AI should attempt resolution only when confidence is high, with a clear path to human support.

AI customer service works best when it knows what it should not answer. Run your top 20 question types through the four checks this week, and write the result down as a routing policy your team and your bot both follow. Start automation in the first lane only, and move a question type up only after you have read its transcripts.

To build escalation rules into a support agent, see our AI customer support automation and AI agent development services. Or book a free AI consultation.