AI Hallucinations: Why AI Makes Things Up and How to Stop It

You ask an AI chatbot for three research papers on a topic. It gives you titles, authors, journals and years, all formatted perfectly. You search for them, and one does not exist. The AI did not lie on purpose. It hallucinated.

AI hallucinations are one of the biggest problems with tools like ChatGPT, Claude, Gemini and every other language model, including the assistants on Red Drogon. They have led to fake legal citations in court filings, invented quotes in articles and wrong numbers in business reports. This guide explains what AI hallucinations are, why they happen, the warning signs, and practical techniques to reduce them, whether you are a student, a writer or a business building AI products.

What Is an AI Hallucination?

An AI hallucination is when an AI model produces information that is false, made up or not supported by its sources, while presenting it as if it were true. The term is borrowed from psychology, but the AI is not “seeing things”. It is generating text that sounds plausible without being grounded in fact.

Common types include:

• Fabricated facts: wrong dates, numbers, statistics or events.
• Fake sources: citations, papers, books, court cases or URLs that do not exist.
• Invented quotes: words attributed to real people who never said them.
• Wrong reasoning: math or logic steps that look correct but are not.
• Unfaithful summaries: summaries that add details not present in the original document.
• Made-up features: describing software settings, functions or product features that do not exist.

Why Do AI Models Hallucinate?

To understand hallucinations, it helps to know what a language model actually does.

1. Language models predict text, they do not look up facts

A large language model generates an answer one piece at a time, each time predicting what is most likely to come next based on patterns learned during training. It does not consult a database of verified facts. When the patterns strongly point to the right answer, it is correct. When they do not, it still produces fluent, confident text, because fluent text is what it was trained to produce.

2. Rare facts are hard to learn

Well-known facts appear many times in training data, so models learn them reliably. Rare, specific facts, such as a minor researcher’s birthday or the exact title of an obscure paper, may appear once or not at all. The model fills the gap with something that fits the pattern.

3. Training and testing reward guessing

In September 2025, OpenAI researchers published a paper and summary titled Why language models hallucinate. Their core argument is that standard training and evaluation reward guessing over admitting uncertainty. On an accuracy-only test, a guess sometimes scores points while “I don’t know” always scores zero, so models learn to guess. They suggest penalizing confident errors more than expressions of uncertainty.

4. Missing or outdated information

Models have a knowledge cutoff and know nothing about your private documents. When asked about recent events or internal company details, they may fill in the blanks instead of saying they do not know.

5. Pressure from the prompt

Prompts that assume something false (“Why did Einstein win two Nobel Prizes?”) or demand a specific number of items (“Give me ten studies proving this”) push the model to produce an answer that fits the request, even if the facts do not exist.

Warning Signs of an AI Hallucination

• Very specific details you did not ask for, such as exact percentages, page numbers or dates with no source.
• Citations you cannot find with a quick search, or links that lead nowhere.
• Quotes from real people that you cannot find in any reliable source.
• Answers that change when you ask the same question again in a new chat.
• Too-perfect matches to what you hoped to hear.
• Niche topics, recent events, local information and specialized technical details.
• Confident answers to questions that do not have a clear answer.

How to Reduce AI Hallucinations: Tips for Everyday Users

Give the AI the facts

The single most effective fix is to provide source material. Paste the article, upload the document or include the data, and tell the model: “Answer only using the text below. If the answer is not in the text, say so.”

Allow “I don’t know”

Explicitly tell the model that saying “I’m not sure” or “I could not find that” is a good answer. This counters the guessing habit described above.

Ask for sources, then check them

Ask the AI to cite where each claim comes from, and use tools with web search when facts matter. Then open the sources. A citation is only useful if it exists and actually says what the AI claims.

Break questions into smaller steps

Complex questions invite errors. Ask for one thing at a time and check each step, especially for calculations and multi-part research.

Avoid leading questions

Ask “Did Einstein win more than one Nobel Prize?” instead of “Why did Einstein win two Nobel Prizes?” Neutral questions do not push the model toward a false premise.

Ask the model to double-check

After an answer, ask: “Review your answer. Which claims are you least confident about, and why?” Models often flag their own weak points when asked directly.

Cross-check important answers

For anything that matters, such as health, legal, money or published work, compare with an official source or a second tool, and never rely on AI alone.

Many of these habits are part of good prompting. Our prompt engineering guide covers them in more depth.

How Businesses Reduce Hallucinations in AI Products

When AI answers customers or employees at scale, a single hallucination can cause real damage. Teams building AI products use several layers of protection.

Technique	How it helps	Effort
Retrieval-augmented generation	Grounds answers in approved documents	Medium
Strict system instructions	Requires answers from context and “I don’t know” when unsure	Low
Citations and quote checking	Shows the source passage so users and checkers can verify	Medium
Structured outputs	Limits answers to set fields and choices	Low
Tool use for facts	Calls a calculator, database or API instead of guessing numbers	Medium
Automated evaluation	Tests the system on known questions before release	Medium
Human review for high-stakes output	Catches errors before they reach customers	High

Retrieval is the foundation for most reliable business assistants. Our guide to retrieval augmented generation explains how it works and why it reduces, but does not eliminate, hallucinations. The OWASP Top 10 for LLM Applications lists misinformation as one of the main risks of AI systems, alongside risks such as excessive agency, which matters more as AI starts taking actions on its own.

Hallucinations in AI agents

Hallucinations are more dangerous when AI can act. An agent that invents a file name, a command or a customer’s order number can take a wrong action that builds on itself. That is why agent systems use tool results to check facts, require approval for important actions and keep logs. Read more in our guide to agentic ai.

Are Some AI Models Better Than Others?

Yes. Newer models and models with web search or retrieval generally hallucinate less on factual questions than older or smaller ones, and models that are trained to express uncertainty make fewer confident errors. But no current model is hallucination-free. OpenAI’s own research summary says hallucinations persist even in its latest models.

Smaller models that run on a laptop tend to make more factual mistakes than the largest cloud models, which is worth remembering if you run an llm locally. Coding assistants can also hallucinate functions or library features that do not exist, a risk we cover in our guide to ai for coding. Image generators have their own version of the problem, such as extra fingers or garbled text, explained in how ai image generators work.

Where Hallucinations Are Most Dangerous

• Low risk: brainstorming, creative writing, first drafts, rewording your own text.
• Medium risk: learning a new topic, summaries, code suggestions, marketing claims.
• High risk: medical, legal, tax and financial guidance, academic citations, news, anything published under your name.

The rule is simple: the higher the stakes, the more you verify. For high-risk topics, treat AI as a starting point for research, never as the final answer, and consult a qualified professional.

Famous Types of Hallucination Mistakes

Hallucinations have caused real problems when people trusted AI output without checking it. These patterns appear again and again:

• Fake legal cases. Lawyers in several countries have been sanctioned or embarrassed after filing documents that cited court cases an AI tool invented.
• Invented academic references. Students and researchers have submitted work with references to papers that never existed.
• Made-up quotes in articles. Writers have published quotes attributed to real people that the people never said.
• Wrong product or policy details. Customer support bots have promised refunds or features that the company never offered.
• Imaginary software features. Developers have spent hours searching for settings and functions that an AI described with confidence.

In almost every case, the fix would have been the same: check the source before relying on it. A one-minute search is far cheaper than a public correction.

A Simple Fact-Checking Routine for AI Answers

1. Highlight the claims. Mark every number, date, name, quote and citation in the answer.
2. Rank by importance. Decide which claims your decision or article really depends on.
3. Verify the important ones. Check each against an official site, the original document or a trusted reference.
4. Open every citation. Make sure it exists and says what the AI claims it says.
5. Remove or flag the rest. Delete claims you cannot verify, or clearly mark them as unconfirmed.

Teaching Students and Teams About Hallucinations

The best protection is awareness. Show people a real example of an AI inventing a citation, then ask them to find the error themselves. Once someone has caught a hallucination with their own eyes, they rarely trust AI output blindly again. For teams, add verification to the workflow itself: a checklist before publishing, a second reviewer for customer-facing AI content, and clear rules about which topics always need a human expert.

Copy-and-Paste Prompts That Reduce Hallucinations

• For documents: “Answer only using the text between the tags. Quote the sentence you used for each point. If the answer is not in the text, reply ‘Not found in the document.'”
• For research: “List your sources with links for every factual claim. If you are not sure a source exists, say so instead of guessing.”
• For numbers: “Show the calculation step by step and state where each input number comes from.”
• For self-checking: “Review your previous answer. Mark each claim as verified from the provided material, general knowledge, or uncertain.”
• For honesty: “It is completely fine to say ‘I don’t know’. A confident wrong answer is worse than no answer.”

These prompts do not make a model perfect, but they change what it is rewarded for in the conversation: being faithful and honest instead of sounding complete.

Key Terms Explained

• Hallucination: an AI output that is false or unsupported but presented as fact.
• Grounding: connecting an AI’s answer to real source material, such as documents, search results or databases.
• Retrieval: searching a set of documents for passages relevant to a question before the AI answers.
• Citation: a reference to the source behind a claim; only useful if it exists and supports the claim.
• Knowledge cutoff: the date after which a model has no training data, so it does not know later events.
• Abstention: when a model declines to answer or says it does not know, which is better than a confident wrong answer.
• Faithfulness: how closely a summary or answer sticks to its source material without adding new claims.
• Calibration: how well a model’s confidence matches how often it is actually right.

Key Takeaways

• Hallucinations come from how language models work, not from random bugs.
• Giving the AI source material is the single best way to reduce them.
• Let the AI say “I don’t know” and avoid leading questions.
• Open and check every citation and important number.
• The higher the stakes, the more verification and human review you need.
• No current model is hallucination-free, so build checking into your workflow.

Remember that hallucinations are a reason to check AI, not a reason to avoid it. Used with source material, clear instructions and a quick verification habit, AI remains a fast and valuable research and writing partner.

Frequently Asked Questions

What is an AI hallucination?

An AI hallucination is when an AI model produces false or made-up information, such as wrong facts, fake citations or invented quotes, while presenting it confidently as if it were true.

Why does ChatGPT make things up?

Language models predict likely text rather than look up verified facts. When they lack information, they can still produce fluent answers, and training that rewards correct guesses over admitting uncertainty makes guessing more likely.

Can AI hallucinations be completely eliminated?

Not with current technology. They can be greatly reduced with source documents, retrieval, clear instructions, citations, tool use and human review, but every current model still makes some mistakes.

How can I tell if AI is hallucinating?

Watch for very specific unsupported details, citations you cannot find, quotes you cannot verify and answers that change when you ask again. Check important claims against reliable sources.

Does RAG stop AI hallucinations?

RAG reduces hallucinations by grounding answers in real documents, but it does not stop them completely. Poor retrieval or unclear instructions can still lead to wrong answers.

Why does AI make up citations?

The model has learned what citations look like and can generate realistic titles, authors and journals that fit the pattern, even when no such source exists. Always open and check any citation an AI gives you.

Do newer AI models hallucinate less?

Generally yes, especially models with web search or retrieval and models trained to express uncertainty, but no model is hallucination-free, so verification is still necessary.

Is it safe to use AI for medical or legal questions?

Use AI only as a starting point to understand terms and prepare questions. Always confirm with official sources and a qualified professional before making medical, legal or financial decisions.

What prompt reduces hallucinations?

Provide the source text and say: answer only from this text, cite the part you used, and say you do not know if the answer is not there. Avoid leading questions and ask for one thing at a time.

Final Thoughts

AI hallucinations are not random glitches. They come from how language models work: they generate likely text, they struggle with rare facts, and they have been rewarded for guessing. You cannot switch hallucinations off, but you can make them rare and catch the rest. Give AI your sources, let it say “I don’t know”, ask for citations and check them, and keep a human in charge of anything that matters. Research in this area moves quickly; this guide reflects what is known as of November 2026.