## Diagram: RALMs Knowledge Category Quadrant and Refusal Examples
### Overview
The image presents a quadrant diagram categorizing knowledge based on RALMs (Retrieval-Augmented Language Models) and LLMs (Large Language Models), along with examples of proper and over refusal in question-answering scenarios.
### Components/Axes
* **Quadrant Diagram:**
* X-axis: LLMs (Unknown to Known)
* Y-axis: RALMs (Unknown to Known)
* Quadrants:
* Top-left: Context Known, RALMs Known, LLMs Unknown (Green Dot)
* Top-right: Context Known, RALMs Known, LLMs Known (Yellow Dot)
* Bottom-left: Context Unknown, RALMs Unknown, LLMs Unknown (Black Dot)
* Bottom-right: Context Unknown, RALMs Known, LLMs Known (Blue Dot)
* **Refusal Examples:**
* Two question-answer pairs are shown, each with:
* Question (Q:)
* RAG context: (Retrieved context)
* Answer (with a bot icon)
* Checkmark or X mark indicating correctness of the answer
### Detailed Analysis or ### Content Details
**1. Quadrant Diagram:**
* The diagram visually represents the state of knowledge for both RALMs and LLMs.
* The top-left quadrant indicates scenarios where the context is known, and RALMs have the necessary information, but LLMs do not.
* The top-right quadrant indicates scenarios where both context and RALMs/LLMs have the necessary information.
* The bottom-left quadrant indicates scenarios where the context is unknown, and neither RALMs nor LLMs have the information.
* The bottom-right quadrant indicates scenarios where the context is unknown, but RALMs/LLMs have the information.
**2. Refusal Examples:**
* **Example 1 (Proper Refusal):**
* Question: "Who won the 2022 Citrus Bowl?"
* RAG context: "Kentucky secured its fourth straight bowl victory ... Citrus Bowl win over Iowa."
* Correct Answer: Kentucky (Green Dot)
* Question: RAG context: "Buffalo beat Georgia Southern 23-21 after going 12-of-19 on third down while averaging less than three yards a carry."
* Answer: "I don't know" (Grey Dot)
* Result: Correct refusal (checkmark)
* **Example 2 (Over Refusal):**
* Question: "When does the 2022 Olympic Winter Games end?"
* RAG Context: "The closing ceremony of the 2022 Winter Olympics was held at Beijing National Stadium on 20 February 2022;"
* Correct Answer: "February 20" (Yellow Dot)
* RAG Context: "February 14, 2022: Another event making its debut at the Beijing Games was the monobob, a single-person bobsledding event."
* Answer: "I don't know" (Grey Dot)
* Result: Incorrect refusal (X mark)
### Key Observations
* The quadrant diagram provides a framework for understanding the interplay between RALMs and LLMs in knowledge representation.
* The refusal examples highlight the importance of accurate context retrieval and the potential for both proper and over refusal in question-answering systems.
* The color-coding (green, yellow, black, blue) in the quadrant diagram corresponds to the knowledge state of RALMs and LLMs.
### Interpretation
The image illustrates a system for categorizing knowledge based on the capabilities of RALMs and LLMs. The quadrant diagram serves as a visual aid for understanding the different scenarios that can arise when these models are used for question answering. The refusal examples demonstrate the challenges of building robust question-answering systems that can accurately determine when they lack the necessary information to provide a correct answer. The "Proper refusal" example shows the system correctly identifying a lack of relevant information and refusing to answer, while the "Over refusal" example shows the system incorrectly refusing to answer despite having access to relevant information. This highlights the need for improved context retrieval and reasoning capabilities in question-answering systems.
