## Line Graph: I-Don't-Know Rate Across Mistral-7B Model Versions and Anchoring Methods
### Overview
The image contains two side-by-side line graphs comparing the "I-Don't-Know Rate" across 30 layers of the Mistral-7B model (versions v0.1 and v0.3). Each graph tracks multiple data series representing different anchoring methods (Q-Anchored and A-Anchored) and datasets (PopQA, TriviaQA, HotpotQA, NQ). The y-axis measures the I-Don't-Know Rate (0-100%), while the x-axis represents model layers (0-30). Shaded regions around lines indicate variability/confidence intervals.
### Components/Axes
- **X-axis (Layer)**: 0 to 30 (integer increments)
- **Y-axis (I-Don't-Know Rate)**: 0% to 100% (linear scale)
- **Legends**:
- **Left Graph (v0.1)**:
- Solid blue: Q-Anchored (PopQA)
- Dashed green: Q-Anchored (TriviaQA)
- Dotted orange: A-Anchored (PopQA)
- Dashed red: A-Anchored (TriviaQA)
- Gray shaded area: Overall variability
- **Right Graph (v0.3)**:
- Solid purple: Q-Anchored (HotpotQA)
- Dashed pink: Q-Anchored (NQ)
- Dotted orange: A-Anchored (HotpotQA)
- Gray shaded area: Overall variability
### Detailed Analysis
#### Left Graph (Mistral-7B-v0.1)
1. **Q-Anchored (PopQA)** (blue):
- Starts at ~80% at layer 0, drops sharply to ~20% by layer 10, then fluctuates between 30-60% with peaks at layers 15 (~50%) and 25 (~70%).
2. **Q-Anchored (TriviaQA)** (green):
- Begins at ~60%, dips to ~10% at layer 10, then oscillates between 20-50% with a peak at layer 20 (~60%).
3. **A-Anchored (PopQA)** (orange):
- Starts at ~50%, rises to ~90% at layer 5, then declines to ~30% by layer 30 with minor fluctuations.
4. **A-Anchored (TriviaQA)** (red):
- Begins at ~70%, drops to ~20% at layer 10, then rises to ~80% at layer 20 before stabilizing near 60%.
#### Right Graph (Mistral-7B-v0.3)
1. **Q-Anchored (HotpotQA)** (purple):
- Starts at ~90%, plunges to ~10% at layer 10, then fluctuates between 30-70% with a peak at layer 25 (~80%).
2. **Q-Anchored (NQ)** (pink):
- Begins at ~70%, drops to ~20% at layer 10, then rises to ~60% at layer 20 before declining to ~40%.
3. **A-Anchored (HotpotQA)** (orange):
- Starts at ~60%, peaks at ~95% at layer 5, then declines to ~40% by layer 30 with sharp dips at layers 15 (~20%) and 25 (~30%).
### Key Observations
1. **Version Differences**:
- v0.3 shows higher variability in Q-Anchored (HotpotQA) and A-Anchored (HotpotQA) compared to v0.1.
- v0.1's A-Anchored (PopQA) has more extreme peaks (90%) than v0.3's equivalent (~95%).
2. **Dataset Impact**:
- HotpotQA and NQ datasets exhibit sharper drops in I-Don't-Know rates at early layers (layers 5-10).
- TriviaQA and PopQA show more gradual declines.
3. **Anchoring Method Trends**:
- Q-Anchored methods generally show steeper initial declines than A-Anchored methods.
- A-Anchored methods (e.g., PopQA, TriviaQA) maintain higher rates in later layers (20-30).
### Interpretation
The data suggests that anchoring methods (Q vs. A) and datasets significantly influence the model's uncertainty distribution across layers. Q-Anchored methods (e.g., HotpotQA in v0.3) demonstrate more pronounced early-layer drops in I-Don't-Know rates, potentially indicating stronger initial confidence. A-Anchored methods (e.g., PopQA in v0.1) exhibit higher variability in later layers, suggesting persistent uncertainty. The shaded regions highlight model instability, with v0.3 showing broader confidence intervals than v0.1. These patterns may reflect architectural changes between versions or dataset-specific challenges in knowledge representation.
