What is it called when AI gives wrong information?

It's called a hallucination, or a confabulation: the model states something false as if it were fact. It happens because the model predicts likely text and, without a confidence gate or good source retrieval, a confident guess ships exactly like a verified fact.

What shouldn't you tell ChatGPT or a public AI tool?

Don't paste customer data, passwords, trade secrets, or anything regulated into a public consumer AI tool, because inputs may be retained or used for training depending on the provider's terms. For business use, build on an arrangement where you control data retention and the system runs on data you own.

How can a business prevent its AI from giving wrong answers?

Use deterministic guardrails, confidence scoring, automated fallback to a human for low-confidence or sensitive questions, human approval on customer-facing output, and a feedback loop that logs and corrects every decision. Structure matters more than which model you pick.

When AI Gives a Customer Wrong Information: Who Pays

Q: Who is liable when AI makes a mistake?

The business that deployed the AI is almost always accountable. Model providers disclaim accuracy in their terms, and courts have treated a customer-facing bot as the brand's own voice, including a widely reported case where an airline was ordered to honor what its chatbot wrongly promised.

Q: What should I do if an AI gives me incorrect answers?

Screenshot the answer with its timestamp, ask a human to confirm or correct it in writing, and escalate. If you run the business, treat it as a signal your system needs a confidence threshold and a human approval gate before customer-facing answers ship.

What happens when AI gives a customer wrong information?

When an AI gives a customer wrong information, the business that deployed it is held accountable, legally and financially, not the model vendor. Customers treat the bot as the brand's voice, so a false price, policy, or promise becomes a commitment the business may have to honor, plus lost trust and cleanup cost. Good engineering stops it before the customer ever sees it.

You didn't add AI to your business to inherit a new way of making promises you never approved. Below is what actually goes wrong inside these systems, who ends up on the hook, the build-level safeguard that keeps a wrong answer from reaching a customer at all, and the exact next step to pressure-test your own setup.

A concrete example: the wrong answer that cost a company

There's a widely reported case where an airline's website chatbot told a grieving passenger he could claim a bereavement discount retroactively. That was false. The airline argued the chatbot was responsible for its own statements. A civil tribunal rejected that argument and ordered the airline to pay. If you want to cite a case like this, confirm the exact name, year, and tribunal citation from the public record before you publish it.

That is the whole pattern in one case. The AI stated something wrong. The customer relied on it. The business owned the outcome. "The AI said it" is not a defense.

Why does AI give false information in the first place?

Wrong answers are not random. In production systems they come from a few specific failures:

Hallucination. The model generates a fluent, confident answer that has no basis in your data. It is predicting likely text, not checking facts.
Bad retrieval (RAG). Most business bots use retrieval-augmented generation: they pull your documents, then answer from them. If your content is chunked poorly, the model grabs half a policy or an outdated page and answers from the wrong slice.
Out-of-context retrieval. The system fetches a passage that looks relevant by keyword but answers a different question, so the reply is confidently off-topic.
Stale data. The price changed, the policy changed, the source did not. The bot repeats what it has.
No confidence gate. The model has no built-in sense of "I'm not sure." Without a layer that measures certainty, a guess ships exactly like a fact.

The term for an AI stating false information as fact is a hallucination or, more broadly, a model confabulation.

Who is liable when AI makes a mistake?

There is a liability chain, and it almost always lands on the deploying business. Here is how the parties sort out.

Party	Role	Typical liability for a wrong customer answer
Foundation model provider (OpenAI, Anthropic)	Builds the underlying model	Low. Their terms of service disclaim accuracy and fitness for purpose; you accept that risk by building on them
Vendor / integrator	Builds the bot on top of the model	Shared, by contract. Depends on your agreement and what they promised
Deploying business (you)	Puts the bot in front of customers	High. You present it as your voice, so courts and customers treat its answers as yours

The short version: the model providers write their terms to push accuracy risk downstream, and the business that shows the bot to customers tends to own what it says. Read your vendor contract and the provider's terms with that in mind before you launch anything customer-facing.

How do you stop a wrong answer before it reaches a customer?

You design the system so a low-confidence or customer-facing output cannot ship on its own. This is where the build matters more than the model. The safeguards that work:

Deterministic guardrails. Hard rules the AI cannot override: never quote a price, never confirm a refund, never state a policy outside an approved list of sources.
Confidence scoring thresholds. Score each answer. Below the threshold, the system does not guess, it escalates.
Automated fallback to a human. When confidence is low or the topic is sensitive, the bot hands off to a person instead of inventing an answer.
Human approval on customer-facing output. The thing a customer actually reads gets a human sign-off, or it is confined to answers the business pre-approved.
A feedback loop. Every decision the AI makes is logged and reviewed, so the system gets corrected instead of repeating the same miss.

That last pair is Akira's doctrine, and it is how we run our own systems. Anything a customer will see is gated behind human approval, and anything the AI decides runs on a feedback loop. The point of an AI build is to take real work off your plate, not to hand a stranger a megaphone with your name on it. If you're weighing where to let AI act versus only draft, our guide on where to start with AI and what an AI agent actually can and can't do both walk through the draft-before-it-acts line.

How we build this

Take a customer support bot answering over a business's own policy and pricing documents. You set it up so no answer touching price, refunds, or policy ships without clearing a confidence threshold first. Anything under that line, or any question flagged sensitive, hands off to a person instead of guessing. You scope the retrieval so the bot can only answer from an approved set of current documents, not the whole internet. Every answer gets logged with its source. The point you can state plainly: a wrong price or policy commitment cannot leave the system unseen, because the build does not allow it to.

The safest AI system is the one where a wrong answer gets caught before a customer ever sees it, not apologized for after.

What should you log so you can prove what happened?

When a customer disputes what the bot told them, you need a record. Build the audit trail in from day one:

The full prompt and response for every conversation, timestamped.
The retrieval trace: which documents or vector-database chunks the system pulled to build that answer.
The confidence score and which guardrail or fallback fired, if any.
The model and version used, so you can attribute behavior to a specific release.

With that trace, root-cause attribution is a quick job instead of a guessing game. You can show whether the bot answered from an approved source, and whether a human approved what shipped.

What should I do if an AI gives me incorrect answers?

As a customer: screenshot it with the timestamp, ask for the same answer in writing from a human, and escalate to a person. As a business owner: that same expectation is why your bot needs a human gate and a log. Being the business that caught the mistake first is the version worth being.

Your next step

AI will get things wrong. The question is whether your system is built so the customer never pays for it. Here's the concrete move: pull up your own bot and ask it a question about price, refunds, or policy. If it answers confidently without citing an approved source, and without a human able to see what it said, that is the gap. Want a second set of eyes on where your AI can commit your business without approval? Get in touch with Akira and we'll walk your setup for the exact points a wrong answer could ship. Custom AI that does the work, with you still in control.