## Hallucination Examples: Factuality vs. Faithfulness ### Overview The image presents two examples of hallucinations in language models (LLMs), contrasting "Factuality Hallucination" with "Faithfulness Hallucination." Each example includes an instruction, the LLM's response, and an explanation of why the response constitutes a hallucination. ### Components/Axes The image is divided into two columns, each representing a type of hallucination. **Left Column: Factuality Hallucination** * **Instruction:** "What are Thomas Edison's main contributions to science and technology?" (Contained in a yellow rounded rectangle) * **Response:** "Thomas Edison developed the first practical telephone and invented the light bulb." (Contained in a blue rounded rectangle) * "first" is in red text. * **Explanation:** "The generated response is factually incorrect because Edison improved the light bulb, building on earlier designs by others, and Alexander Graham Bell invented the telephone." (Contained in a green rounded rectangle) * **Title:** "Factuality Hallucination" **Right Column: Faithfulness Hallucination** * **Instruction:** "Translate the English question into Spanish: 'What is the capital of France?'" (Contained in a yellow rounded rectangle) * **Response:** "The capital of France is Paris." (Contained in a blue rounded rectangle) * "Paris" is in red text. * **Explanation:** "The LLM ignored the user's explicit instruction to translate an English question into Spanish and instead generated an answer to the question." (Contained in a green rounded rectangle) * **Title:** "Faithfulness Hallucination" ### Detailed Analysis or ### Content Details **Factuality Hallucination:** The instruction asks for Edison's contributions. The response incorrectly attributes the invention of the telephone to Edison, when it was Alexander Graham Bell. The light bulb claim is partially correct, but Edison improved it, rather than inventing it. **Faithfulness Hallucination:** The instruction asks for a translation of the question "What is the capital of France?" into Spanish. The response provides the answer to the question in English, ignoring the translation request. ### Key Observations * The "Factuality Hallucination" example demonstrates the LLM providing incorrect information about real-world facts. * The "Faithfulness Hallucination" example demonstrates the LLM failing to follow the specific instructions given by the user. * The use of color (red) highlights the incorrect or problematic parts of the LLM's responses. ### Interpretation The image illustrates two distinct types of errors that LLMs can make. "Factuality Hallucination" refers to the generation of false or inaccurate information, while "Faithfulness Hallucination" refers to the LLM's failure to adhere to the user's instructions or constraints. These examples highlight the challenges in ensuring that LLMs are both knowledgeable and reliable in their responses. The image suggests that LLMs can sometimes prioritize providing an answer, even if it means being factually incorrect or ignoring the specific requirements of the prompt.