## Bar Charts: Δ Saliency Metrics and Accuracy Gains Across Models
### Overview
The image contains two grouped bar charts comparing saliency metric changes (Δ metric) and accuracy gains (ΔPass@1) for three AI models: R1-32B, GPT-OSS-20B, and QwQ-32B. The charts are divided into:
1. **AIME25 (math)**: Δ saliency metrics and accuracy gains.
2. **GPQA-D (science)**: Δ saliency metrics and accuracy gains.
Each chart uses a legend with three categories: Δ shallow self (thinking), Δ Q→thinking (shallow), and ΔPass@1.
---
### Components/Axes
#### Legends
- **Δ shallow self (thinking)**: Dark red (negative values).
- **Δ Q→thinking (shallow)**: Light orange (positive values).
- **ΔPass@1**: Light beige (positive values).
#### Axes
- **X-axis**: Model names (R1-32B, GPT-OSS-20B, QwQ-32B).
- **Y-axis (top charts)**: Δ metric (after – before), ranging from -0.10 to 0.10 (AIME25) and -0.10 to 0.075 (GPQA-D).
- **Y-axis (bottom charts)**: ΔPass@1 (points), ranging from 0 to 15 (AIME25) and 0 to 10 (GPQA-D).
---
### Detailed Analysis
#### AIME25 (Math) Δ Saliency Metrics
- **R1-32B**:
- Δ shallow self: -0.08 (dark red).
- Δ Q→thinking: +0.07 (light orange).
- ΔPass@1: +11.8 (light beige).
- **GPT-OSS-20B**:
- Δ shallow self: -0.09 (dark red).
- Δ Q→thinking: +0.06 (light orange).
- ΔPass@1: +7.4 (light beige).
- **QwQ-32B**:
- Δ shallow self: -0.05 (dark red).
- Δ Q→thinking: +0.05 (light orange).
- ΔPass@1: +8.0 (light beige).
#### GPQA-D (Science) Δ Saliency Metrics
- **R1-32B**:
- Δ shallow self: -0.05 (dark red).
- Δ Q→thinking: +0.04 (light orange).
- ΔPass@1: +2.5 (light beige).
- **GPT-OSS-20B**:
- Δ shallow self: -0.07 (dark red).
- Δ Q→thinking: +0.06 (light orange).
- ΔPass@1: +6.9 (light beige).
- **QwQ-32B**:
- Δ shallow self: -0.03 (dark red).
- Δ Q→thinking: +0.05 (light orange).
- ΔPass@1: +5.2 (light beige).
---
### Key Observations
1. **Δ Q→thinking (shallow)** is consistently positive across all models and tasks, indicating a beneficial impact on saliency.
2. **Δ shallow self (thinking)** is negative in all cases, suggesting a detrimental effect on saliency.
3. **ΔPass@1** (accuracy gains) is highest for R1-32B in AIME25 (+11.8) and GPT-OSS-20B in GPQA-D (+6.9).
4. QwQ-32B shows the smallest Δ shallow self (-0.03 to -0.05) but moderate accuracy gains.
---
### Interpretation
- **Saliency Metrics**: The negative Δ shallow self (thinking) implies that shallow self-reflection reduces model performance, while Q→thinking (shallow) enhances it. This suggests that models benefit from structured, shallow reasoning over unguided self-reflection.
- **Accuracy Gains**: R1-32B outperforms others in math (AIME25), while GPT-OSS-20B excels in science (GPQA-D). QwQ-32B shows balanced performance but lower gains.
- **Model Differences**: The disparity in ΔPass@1 between AIME25 and GPQA-D highlights task-specific model strengths. Math models (AIME25) achieve higher gains, possibly due to structured problem-solving requirements.
---
### Spatial Grounding
- **Legends**: Positioned at the bottom center, with clear color coding.
- **Charts**: Top charts (AIME25/GPQA-D) are stacked vertically, with bottom charts (accuracy gains) aligned horizontally. Bars are grouped by model, with consistent spacing.
---
### Trend Verification
- **AIME25 ΔPass@1**: R1-32B (+11.8) > QwQ-32B (+8.0) > GPT-OSS-20B (+7.4). No strict linear trend, but R1-32B dominates.
- **GPQA-D ΔPass@1**: GPT-OSS-20B (+6.9) > QwQ-32B (+5.2) > R1-32B (+2.5). GPT-OSS-20B leads in science.
---
### Final Notes
The data underscores the importance of Q→thinking (shallow) in improving model performance, while shallow self-reflection hinders it. Task-specific model architectures (e.g., R1-32B for math, GPT-OSS-20B for science) drive accuracy gains, highlighting the need for tailored training strategies.
