## Quadrant Diagram: RALMs Knowledge Category Quadrant
### Overview
The image presents a quadrant diagram titled "RALMs Knowledge Category Quadrant," divided into four regions by two axes:
- **Vertical Axis**: "Context Known" (top) to "Context Unknown" (bottom)
- **Horizontal Axis**: "LLMs Known" (right) to "LLMs Unknown" (left)
Each quadrant contains a colored data point with a label, and examples of question-answer pairs are provided below to illustrate proper vs. over refusal scenarios.
---
### Components/Axes
1. **Axes**:
- **Vertical (Y-axis)**: "Context Known" (top) → "Context Unknown" (bottom)
- **Horizontal (X-axis)**: "LLMs Known" (right) → "LLMs Unknown" (left)
2. **Legend**:
- **Green**: RALMs Known, Context Known
- **Yellow**: RALMs Known, Context Unknown
- **Blue**: RALMs Unknown, Context Known
- **Gray**: RALMs Unknown, Context Unknown
3. **Data Points**:
- **Top-Right (Green)**: RALMs Known, Context Known
- **Top-Left (Yellow)**: RALMs Known, Context Unknown
- **Bottom-Left (Gray)**: RALMs Unknown, Context Unknown
- **Bottom-Right (Blue)**: RALMs Unknown, Context Known
---
### Detailed Analysis
#### Quadrant Labels and Data Points
- **Top-Right (Green)**:
- Label: "RALMs Known, Context Known"
- Position: Intersection of "Context Known" (top) and "LLMs Known" (right).
- **Top-Left (Yellow)**:
- Label: "RALMs Known, Context Unknown"
- Position: Intersection of "Context Unknown" (bottom) and "LLMs Known" (right).
- **Bottom-Left (Gray)**:
- Label: "RALMs Unknown, Context Unknown"
- Position: Intersection of "Context Unknown" (bottom) and "LLMs Unknown" (left).
- **Bottom-Right (Blue)**:
- Label: "RALMs Unknown, Context Known"
- Position: Intersection of "Context Known" (top) and "LLMs Unknown" (left).
#### Question-Answer Examples
1. **Question**: "Who won the 2022 Citrus Bowl?"
- **RAG Context**: Kentucky secured its fourth straight bowl victory...
- **Answer**: "Kentucky" (✅ Checkmark, green data point).
2. **Question**: "When does the 2022 Olympic Winter Games end?"
- **RAG Context**: The closing ceremony... was held on 20 February 2022.
- **Answer**: "February 20" (✅ Checkmark, yellow data point).
3. **Question**: "RAG context: Buffalo beat Georgia Southern..."
- **Answer**: "I don’t know" (✅ Checkmark, gray data point).
4. **Question**: "RAG context: February 14, 2022..."
- **Answer**: "I don’t know" (❌ Red X, blue data point).
---
### Key Observations
1. **Quadrant Distribution**:
- Two quadrants (green, yellow) represent scenarios where RALMs have knowledge ("RALMs Known").
- Two quadrants (gray, blue) represent scenarios where RALMs lack knowledge ("RALMs Unknown").
2. **Refusal Logic**:
- **Proper Refusal**: Gray data point ("RALMs Unknown, Context Unknown") with a checkmark.
- **Over Refusal**: Blue data point ("RALMs Unknown, Context Known") with a red X, indicating refusal despite known context.
3. **Color Consistency**:
- All data points match the legend colors (green, yellow, gray, blue).
---
### Interpretation
The quadrant diagram categorizes RALMs (Retrieval-Augmented Language Models) based on their knowledge of **context** and **LLMs** (Large Language Models). The examples illustrate how RALMs should respond:
- **Proper Refusal**: When neither context nor LLM knowledge is available (gray quadrant).
- **Over Refusal**: When context is known but the model still refuses (blue quadrant), which is flagged as incorrect.
The diagram emphasizes the importance of leveraging available context to avoid unnecessary refusals, ensuring RALMs provide accurate answers when possible. The red X in the blue quadrant highlights a critical failure mode where models withhold answers despite having sufficient context.
