Introduction

Artificial intelligence (AI) is rapidly transforming our world, with large language models (LLMs) like ChatGPT becoming increasingly sophisticated. However, these advanced systems aren’t without their quirks. One such issue is what’s often called “AI hallucination,” a phenomenon that has become the focus of much discussion and research. But what exactly is it, and why should we care?

What is AI Hallucination?

The term “hallucination” in the context of AI refers to instances when an AI model, especially an LLM, generates content that is incorrect, nonsensical, or fabricated. It’s not about the AI “seeing” things; instead, it’s about producing outputs that are not grounded in fact or the provided context. Definitions of AI hallucination include:

• Output that is fluent but unrelated to the input.
• Content that is incorrect and not supported by any information.
• AI-generated text that lacks fidelity to factual accuracy or reality.
• The generation of information that deviates from established knowledge.
• The confident assertion of a factual inaccuracy.
• The creation of new data or information that does not exist.

This phenomenon occurs across various AI applications, such as:

• Text Translation: Outputs that are fluent but completely unrelated to the original text.
• Text Summarization: Generated content that is inconsistent with the source document, further categorized into intrinsic and extrinsic hallucination.
• Chatbots: Responses that deviate from factual correctness, presenting fictional or erroneous information.
• Healthcare: AI systems generating inaccurate or contextually irrelevant medical information, raising concerns due to the sensitive nature of health data.
• Academia: Fabrication of information and references, compromising research integrity.
• Other Domains: Including investment portfolios, journalism, retail, and sports, where accuracy, coherence, and trustworthiness in AI outputs are crucial.

Timeline of Main Events Related to AI Hallucination:

• 2000: The term “hallucination” is first used in the field of Artificial Intelligence (AI), specifically in computer vision. In this context, it refers to the constructive generation of additional image data, seen as a beneficial process for tasks like super-resolution, image inpainting, and image synthesis.
• Pre-2018: Various applications of AI, particularly in fields like Natural Language Generation (NLG) and Neural Machine Translation (NMT), start to exhibit behavior where generated content is fluent but factually incorrect, often referred to as a form of hallucination. This is seen as a negative trait, and differs from the early use of “hallucination” in computer vision.
• 2018-2021: Research begins to address the issue of ‘hallucination’ in AI models, particularly with the advent of more advanced Natural Language Generation. More specific definitions of AI hallucination start to emerge, focusing on inaccurate or non-factual outputs.
• 2021: Researchers like Dziri et al. , Liu et al. and Huang et al. begin to define AI hallucination in areas of Knowledge Graphs, Hallucination Detection and Text Summarization.
• 2022: Papers discuss the causes of hallucination, exploring whether it stems from datasets or the models themselves (Ref: [74]). Research continues on controlling hallucinations at the word level and fact-aware techniques in data to text generation .
• Early 2023: The rise of Large Language Models (LLMs) like ChatGPT brings the issue of AI hallucination to the forefront. The term becomes widely used, and the inconsistency and imprecision of the definition becomes a concern.
• 2023: Numerous studies and articles explore and define “AI hallucination” in various contexts, including healthcare, academia, software engineering, climate change, retail, and more. Many studies note that AI outputs can seem plausible but contain factual inaccuracies, fabricated references, or information not based in the input data or training.
• 2023 (Specific):Researchers identify extrinsic and intrinsic hallucination.
• Some authors call for replacing the term “hallucination” with terms like “AI misinformation,” and raise concerns about its stigmatizing effects in health contexts.
• Several papers focus on identifying and mitigating hallucination in large language models and generative AI
• Systematic reviews, such as the one in the given source, are conducted to analyze the various definitions and usage of the term “AI hallucination.” The review finds a lack of a universally accepted definition for the term.
• Late 2023: Continued discussion and research aim to establish a consistent and universally applicable terminology, with an emphasis on mitigating confusion.

Why is AI Hallucination a Problem?

AI hallucination is a significant issue because it erodes trust in AI systems. When an AI confidently presents false information, users may struggle to discern fact from fiction. This is especially problematic in high-stakes areas like:

• Healthcare: Inaccurate AI-generated medical information could lead to misdiagnoses, inappropriate treatments, or harmful outcomes. ChatGPT’s suggestions might not adhere to evidence-based guidelines or best practices.
• Legal Settings: Fabricated information generated by AI could have serious legal ramifications, such as incorrect legal briefs.
• Academia: Fabricated findings and fake bibliographic references compromise research integrity and mislead students.

The consequences of AI hallucination underscore the urgent need for robust solutions and critical evaluation of AI outputs.

The Path Forward

The fact that “hallucinate” was named the word of the year in 2023 highlights the significance of this issue. Popular media has extensively covered the phenomenon, illustrating how AI chatbots sometimes make things up. The research community recognizes the need for unified efforts to address this challenge by:

• Developing a Robust Taxonomy: Establishing consistent and universally accepted definitions of AI hallucination.
• Preventing Hallucination: Although this is a key research goal, few solutions have emerged thus far.
• Critical Thinking: Encouraging users to approach AI outputs critically rather than accepting them blindly.

Historical Context: The Origin of “Hallucination” in AI

The term “hallucination” first appeared in AI within computer vision around 2000, where it was seen as a positive attribute, referring to the ability to enhance images through techniques like super-resolution. However, its meaning shifted when applied to NLP systems and LLMs, where it now signifies a flaw: the generation of factually incorrect or nonsensical content.

Lack of a Unified Definition

There is no universally agreed-upon definition of “AI hallucination,” leading to confusion and varied interpretations. For instance:

• In medical contexts, hallucination refers to sensory perceptions in the absence of stimuli, while in AI, it denotes data and prompt-related errors.
• The term’s association with mental illness, particularly schizophrenia, makes it stigmatizing and potentially misleading.

Key Themes and Definitions of “AI Hallucination”

From a systematic review of 14 databases, several themes and definitions have emerged:

• Fabrication & Inaccuracy: AI hallucination involves the creation of elements that do not actually exist, such as fictitious bibliographic references.
• Non-Factual Statements: AI systems generate confident yet incorrect outputs, undermining trust.
• Inconsistency with Training Data: Hallucinations often result from outputs unfaithful to the training data.
• Plausibility vs. Truth: Outputs may appear plausible but are ultimately false, distorting information from the source.
• Specific Types of Hallucination:
  • Extrinsic Hallucination: Introducing information not present in the source data.
  • Intrinsic Hallucination: Distorting source data into factually incorrect representations.
• Novelty vs. Usefulness: AI may prioritize novelty over accuracy, leading to random inaccuracies.
• Misinformation: Some researchers advocate for the term “AI misinformation” to avoid anthropomorphism.

Applications and Domains Affected

AI hallucination impacts numerous fields, including:

• Healthcare: Incorrect diagnoses, fabricated medical literature, and inaccurate treatment options.
• Academia: Falsified research results, nonsensical text, and fake references.
• Legal/Tax: Incorrect legal advice and fabricated cases.
• Software Engineering: Non-existent software elements.
• Climate Science: Inaccurate data provision.
• Retail: Misleading information.
• Database Management: Non-factual statements.

Recommendations

To mitigate the issue of AI hallucination, the research community recommends:

1. Consistent Terminology: Establish universally applicable terms across AI-impacted domains.
2. More Specific Terms: Encourage the use of nuanced terms to accurately describe the issues.
3. Formal Definition: Develop robust definitions to standardize discussions and solutions.
4. Critical User Practices: Promote skepticism and verification when interacting with AI-generated content.

Conclusion

AI hallucination is not merely a quirky glitch; it’s a fundamental challenge that must be addressed as AI becomes increasingly integrated into daily life. Ensuring accuracy, coherence, and trustworthiness in AI outputs is vital to maintaining public trust and leveraging AI’s transformative potential. By fostering a deeper understanding of this issue and advocating for unified solutions, we can navigate the complexities of AI more responsibly and effectively.

FAQ’s- AI Hallucination: Misinformation, Mitigation, and Meaning

1. What is “AI hallucination” and why is it a problematic term?

“AI hallucination” generally refers to instances where AI models, especially large language models (LLMs), generate outputs that are incorrect, nonsensical, or factually inaccurate, despite often being presented confidently and with seeming plausibility. The term is considered problematic for several reasons. First, it’s a misnomer because AI models do not have sensory perceptions, meaning that these inaccuracies stem from data and prompts rather than from the absence of real-world stimuli, as with human hallucinations. Second, using the term “hallucination” can be stigmatizing as it associates AI errors with mental illness, specifically schizophrenia, which can undermine efforts to destigmatize mental health issues. There is also a lack of consistent definition of the term, which further contributes to the problem.

2. How has the term “hallucination” been used in the past within the field of AI?

The term “hallucination” was initially used in computer vision around 2000 with a positive connotation. It was associated with constructive applications like super-resolution, image inpainting, and image synthesis, where additional pixels or details were generated to enhance the usefulness of an image, which was a desirable outcome and not a problem to be avoided. The meaning of the term shifted as AI moved into text and language.

3. Is there a universally accepted definition of “AI hallucination”?

No, there is not a universally accepted definition of “AI hallucination.” The concept is interpreted differently across various fields and applications, including computer science, healthcare, law, and academia. These interpretations are often based on the specific challenges encountered within each field. The lack of a unified definition leads to confusion and hinders effective communication about this important AI issue. Many researchers also include the idea that the AI is ‘confident’ about the inaccurate or fabricated information, which is a key aspect of the problem.

4. What are some common characteristics of “AI hallucination” as identified in the literature?

Based on the review, “AI hallucination” often involves AI models:

• Generating content that is factually incorrect or non-existent.
• Producing responses that sound plausible but are not supported by facts or the provided context.
• Fabricating information, such as nonexistent references or data.
• Creating text that is semantically or syntactically plausible, yet wrong.
• Presenting inaccurate information with confidence.
• Generating text based on internal logic rather than the correct context.
• Providing responses that are not derived from a real-world source but from statistical predictions.
• Distorting information from a source into a factually incorrect representation (intrinsic hallucination)
• Introducing information not present in the source data (extrinsic hallucination)

5. How is “AI hallucination” particularly problematic in healthcare?

In healthcare, “AI hallucination” can lead to the generation of inaccurate medical information which could have serious consequences if relied upon. AI systems in healthcare might generate false diagnoses, incorrect treatment plans, or misleading research summaries. The concern is especially high in medical applications because it could directly affect patient safety. Furthermore, the use of the term “hallucination” in this context is particularly concerning as it carries the risk of stigmatizing both AI systems and individuals who experience hallucinations due to mental health conditions.

6. What alternatives have been suggested to the term “AI hallucination”?

Several alternatives to “AI hallucination” have been proposed, aiming for more accurate and less stigmatizing descriptions. One frequently suggested alternative is “AI misinformation”, as it accurately describes the phenomenon of AI generating incorrect information without attributing human-like characteristics to the system. Other alternatives include “fabrication”, “falsification” or simply “incorrect output,” depending on the specific nature of the problem. A call for more precise and nuanced terms is a focus within the literature on this topic.

7. What research is being done to address the issue of “AI hallucination”?

Research efforts are focusing on several areas to address the problem of “AI hallucination.” Some of these include:

• Developing methods for detecting hallucinated content using benchmarks, fact-checking models and better metrics.
• Improving the training data and processes for LLMs to reduce inaccuracies.
• Exploring techniques to control the generation process and force it to be factually grounded.
• Creating systems that provide context and provenance information, which allows users to verify outputs.
• Developing methods to identify and reduce biases that lead to inaccurate outputs
• Using retrieval augmentation techniques to ensure responses are based on verifiable sources.
• Improving decoding strategies to create more faithful and accurate text.

8. What steps are needed to bring consistency to the discussion of “AI hallucination”?

To bring consistency to discussions about “AI hallucination,” it’s essential to establish universally agreed-upon definitions and terminologies across all impacted domains. This would involve a unified effort among AI researchers, healthcare professionals, ethicists, and legal experts to adopt more precise and nuanced terms that accurately reflect the specific issues being addressed. Furthermore, research is needed to formulate a robust and formal definition of the phenomenon in the context of AI. Finally, a focus on mitigation strategies and development of tools that help reduce these occurrences will be crucial for the effective and responsible deployment of LLMs in various fields.

Glossary

• AI (Artificial Intelligence): The development of computer systems that can perform tasks that typically require human intelligence, such as learning, problem-solving, and decision-making.
• Hallucination (in AI): The phenomenon where AI systems, particularly large language models, generate incorrect, nonsensical, or fabricated information that is often presented with confidence.
• LLM (Large Language Model): A type of AI model that is trained on massive amounts of text data to understand, generate, and manipulate human language.
• Computer Vision: A field of AI that focuses on enabling computers to “see” and interpret images and videos.
• Natural Language Generation (NLG): A subfield of AI that focuses on generating human-like text from structured or unstructured data.
• Extrinsic Hallucination: In the context of AI, this refers to the generation of information that cannot be verified or contradicted by existing knowledge sources or the model’s input data.
• Intrinsic Hallucination: In the context of AI, this refers to the distortion of information from the source data into a factually incorrect representation.
• Knowledge Graph: A structured representation of facts and information that AI models can use to enhance their understanding and generation of text.
• Super-Resolution: A technique in computer vision that enhances the resolution of an image, often by adding new pixels to the original image.
• Image Inpainting: A technique in computer vision used to fill in missing or damaged parts of an image.
• Text Summarization: The process of creating a shorter version of a longer text while retaining the essential information.
• Stigmatization: The act of labeling someone or something as disgraceful or shameful, often leading to social exclusion.

AI Hallucination: A Review

Quiz

1. How was the term “hallucination” initially used in the field of AI, specifically within computer vision?
2. According to the source material, what are the two main reasons that the medical field raises concerns about using the term “hallucination” for AI errors?
3. What are the two paths of action proposed in the article to address the issue of AI hallucination?
4. What motivated the authors to conduct a systematic review of the use of “AI hallucination” across various domains?
5. What was the study period for all the databases reviewed in the article?
6. How did the researchers narrow their search results in Google Scholar to make their review feasible?
7. According to the provided definitions, when does “AI hallucination” occur in dialogue models interacting with a knowledge graph?
8. How does the definition of “AI hallucination” differ when used in the context of text summarization, as opposed to when describing errors in Large Language Models (LLMs)?
9. What is the difference between “extrinsic hallucination” and “intrinsic hallucination” as described by Daull et al.?
10. According to multiple definitions, what is a common characteristic of AI hallucinations regarding how they are presented to the user?

Answer Key

1. Initially in computer vision, “hallucination” was used to describe constructive processes that enhanced images through super-resolution, image inpainting, or image synthesis, adding pixels to improve image quality or fill in missing parts. This was considered a positive attribute, not a problem to be avoided.
2. The medical field is concerned because AI lacks sensory perception, so errors arise from data and prompts, not an absence of external stimuli. Secondly, the term is stigmatizing because it creates a link between AI errors and a mental health issue, potentially undermining efforts to reduce stigma in mental illness, such as schizophrenia.
3. The two proposed paths are: 1) to establish consistent and universally accepted terminologies within AI, and 2) to formulate a robust and formal definition of “AI hallucination” within the AI context.
4. The authors were motivated by the inconsistent use of the term “AI hallucination” across diverse fields, the potentially inappropriate application of the term, and the need for clarity and coherence in discussions and research related to this issue.
5. The study period for all databases began on 01/01/2013 and varied in end dates for each database, between 09/27/2023 and 10/01/2023.
6. The researchers narrowed their search in Google Scholar by using the advanced search feature, looking for records containing the exact phrases “AI hallucination” AND “hallucination in AI,” rather than a broader search.
7. In dialogue models interacting with a knowledge graph, “AI hallucination” occurs when the models generate factually invalid information, despite maintaining plausible general linguistic abilities, by failing to fully discern facts from the graph.
8. In text summarization, “AI hallucination” refers to factual inconsistencies, while in the context of LLMs, it refers to the generation of inaccurate information, often disguised as factual and based on internal logic rather than true context.
9. “Extrinsic hallucination” refers to the model introducing information that is not present in the source data and cannot be verified. “Intrinsic hallucination” occurs when the model distorts information from the source data into a factually incorrect representation.
10. A common characteristic of AI hallucinations is that they are presented with confidence and often appear plausible, despite being incorrect or nonsensical.

Reference

Maleki, N., Padmanabhan, B., & Dutta, K. (2024, June). AI hallucinations: a misnomer worth clarifying. In 2024 IEEE Conference on Artificial Intelligence (CAI) (pp. 133-138). IEEE.