## Diagram: Technical Reasoning Process and Model Evaluation ### Overview The image depicts a technical workflow for evaluating reasoning steps in a problem-solving model. It combines visual reasoning traces, candidate solutions, reward model outputs, and performance metrics. Key elements include: - A reasoning trace with color-coded steps - A reward model evaluation system - Mathematical problem-solving examples - Performance comparison charts ### Components/Axes 1. **Left Panel: Reasoning Trace** - Vertical axis: "Reasoning steps" (t₁ to tₜ) - Horizontal axis: "Final conclusion" (cₜ) - Color-coded steps: Blue (t₁), Purple (t₂), Pink (t₃), Yellow (tₜ) - Final conclusion: Green square 2. **Center Panel: Reward Model** - Speech bubble: "Aya walks 9 km each morning. [...] If she walks at 1 km/h, how many minutes will the total be?" - Gavel icon: "Reward model" - Pink box: Contains red X (incorrect) and green checkmark (correct) - Candidate reasoning steps with arrows to evaluation outcomes 3. **Right Panel: Complete Reasoning Trace** - Text box with multi-colored text (blue, purple, green) - Mathematical equations: "9/(s+3)=2.5" leading to "t=1h" - Final answer: "195 minutes" in boxed notation 4. **Bottom Charts** - **Similarity Graph** - X-axis: "Reasoning step tᵢ" - Y-axis: "Similarity(cₜ, tᵢ)" - Curve: Blue line showing decreasing similarity - **Bar Chart** - Y-axis: "Accuracy" and "#Tokens" - Categories: "Correct t₁" (blue), "Incorrect t₁" (orange) - Legend: Blue = Correct, Orange = Incorrect - Subcategories: "Maj@N" and "Pruned" ### Detailed Analysis 1. **Reasoning Trace Flow** - Steps progress from t₁ (blue) to tₜ (yellow) - Similarity decreases exponentially with each step - Final conclusion (cₜ) is isolated in green 2. **Reward Model Evaluation** - Three candidate reasoning steps: - First: Incorrect (red X) - Ignores café stop - Second: Correct (green check) - Identifies s=3, t=60min - Third: Incorrect (red X) - Misses café stop again 3. **Mathematical Solution** - Equations show: - 9/(s+4) = 2.5 → s=3 - 9/(s+3) = 2.5 → t=1h - Final answer: 195 minutes (1h 35min) 4. **Performance Metrics** - Accuracy: - Maj@N: 100% (blue bar) - Pruned: 100% (green bar) - Token Usage: - Maj@N: Full length (blue bar) - Pruned: 70% reduction (green bar) ### Key Observations 1. **Step Similarity Pattern** - Similarity decreases by ~30% per step (estimated from curve slope) - Final conclusion has 0% similarity to initial steps 2. **Model Performance** - Pruned method maintains accuracy while reducing tokens by 70% - Incorrect steps consistently use more tokens than correct ones 3. **Mathematical Consistency** - Equations show inverse relationship between speed and time - Final answer combines walking time (9km/1km/h=9h) + café stop (60min) ### Interpretation This diagram demonstrates a multi-stage reasoning evaluation system: 1. **Problem Decomposition**: The reward model breaks down the problem into candidate solutions 2. **Validation Process**: Each candidate is tested against mathematical constraints 3. **Optimization**: The pruned method achieves same accuracy with 70% fewer tokens 4. **Temporal Reasoning**: The solution requires combining distance/speed calculations with fixed time elements The system appears designed to: - Identify optimal reasoning paths - Quantify solution efficiency - Maintain mathematical rigor through equation-based validation - Balance accuracy with computational efficiency Notable anomaly: The final answer (195min) doesn't match the initial 9km/1km/h calculation (which would be 9h=540min), suggesting the problem involves additional constraints (like the café stop) that modify the base calculation.