← Course Home

Practical Challenges in Generative AI

Hallucinations and Inconsistency: When AI Gets It Wrong

5 min read

ChatGPT is brilliant — but it isn't infallible

It's easy to give a tool like ChatGPT more credit than it deserves. It writes fluently, answers instantly, and sounds confident about almost everything. That combination can lull you into trusting it more than you should.

OpenAI itself displays a quiet warning at the bottom of the ChatGPT screen: "ChatGPT can make mistakes. Check important info." It's easy to scroll past. It's also one of the most important sentences on the page.

If you don't fact-check an AI's output, you can make big mistakes — and the more confidently the AI states something, the easier it is to let your guard down.

A good rule of thumb: an expert in their field uses AI as a thinking partner, reasoning within the conversation, rather than copying and pasting answers and trusting them blindly.

Two different problems: hallucination and inconsistency

When generative AI tools misbehave, it's usually in one of two distinct ways.

Hallucination is when the AI confidently produces output that is false. It doesn't say "I'm not sure" — it states the wrong answer as if it were fact.

Inconsistency is when the AI gives meaningfully different answers to the same question, asked at different times.

Picture asking an AI for a source on some claim, and it hands you back a confident, specific answer: "See Chen & Alvarez, 'Cognitive Load in Remote Teams,' Journal of Workplace Psychology, 2019." It reads like a real citation — title, authors, journal, year. It also doesn't exist. Nobody wrote that paper. The AI didn't dig it up and misreport it; it generated something that sounds exactly like what a real citation sounds like. That's hallucination in a single concrete moment.

These look similar from the outside — "the AI got it wrong" — but they come from different causes, and that matters for how you deal with them.

Diagram — Two Different Failure Modes

BOTH LOOK LIKE “THE AI GOT IT WRONG” — BUT THE CAUSES DIFFERHallucinationOne bad answer, stated with confidenceWhat it looks likeStates a false “fact” as if it were trueWhy it happensPredicting plausible-sounding text —or learning from incorrect training dataEases with: “only answer if you know”InconsistencyDifferent answers, asked at different timesWhat it looks likeSame question, meaningfully different repliesWhy it happensShared, externally-hosted infrastructure —hardware, load, and conditions vary session to sessionEases with: “take your time” (unproven, low-cost)

Why hallucinations happen

To understand hallucination, it helps to remember how these models actually operate: they predict the next word in a sequence, using surrounding context to make that prediction as accurate as possible.

That's a probabilistic process, not a lookup in a database of facts. Even a powerful, well-trained model will occasionally predict a plausible-sounding sequence of words that doesn't correspond to anything true. The model isn't "lying" — it's doing exactly what it was built to do (predict likely text), and sometimes the most likely-sounding text is wrong.

There's a second cause, too: the training data itself can be wrong. If a model is trained on factually incorrect material, it will naturally tend to reproduce those errors. A model can't output knowledge it was never given correctly in the first place.

One practical mitigation is prompt engineering: explicitly instructing the model to "provide an answer only if you know the answer." This kind of instruction won't eliminate hallucination, but in some cases it nudges the model away from confidently guessing when it's actually uncertain.

Why inconsistency happens

Inconsistency is a different animal. It's most noticeable when you ask an AI the same question on separate occasions and get answers that don't match — sometimes not even close.

Many early ChatGPT users noticed this directly: the model seemed sharper in some sessions and noticeably weaker in others. This isn't your imagination. When a model is hosted externally — running on shared infrastructure that the provider manages — variations in the underlying hardware, system load, and other factors outside your control can subtly change how a response gets generated from one session to the next.

One unproven but commonly suggested tactic is to instruct the AI to "take its time." The idea is that this discourages the system from rushing to produce a response, potentially allowing more computational resources to be allocated to your particular prompt. It's not a guaranteed fix, but it costs nothing to try.

Why this matters going forward

Detecting hallucinations and inconsistencies — and building reliable ways to reduce them — is one of the most active areas of AI research today. Solving this problem well isn't just an academic exercise; it's expected to become a market worth billions, because every business that wants to build trustworthy AI products has to grapple with it.

For now, the most reliable safeguard isn't a clever prompt — it's you. Treat AI output as a draft from a fast, well-read, occasionally-wrong collaborator, and verify anything that matters before you act on it.

Key takeaway

Generative AI can fail in two distinct ways: hallucination, where it confidently states something false (often because it's predicting plausible-sounding text or was trained on bad data), and inconsistency, where it gives different answers to the same question (often due to variability in externally-hosted infrastructure). Prompt engineering can help at the margins, but the dependable safeguard is human fact-checking — especially when the stakes are high.

What's next?

Reliability is one practical challenge of working with generative AI. The next lesson tackles a very different one: the financial side. We'll look at why building and running AI models is so expensive, and how organizations decide how much to invest.