# Technical Document Extraction: Bar Chart Analysis ## Title **Comparing Beam Search and Best-of-N with Unsupervised Difficulty Bins** --- ## Axes and Labels - **X-Axis**: "Test Questions Binned with Unsupervised Difficulty Bins" - Categories: 1, 2, 3, 4, 5 (representing difficulty bins) - **Y-Axis**: "MATH Test Accuracy (%)" - Range: 0% to 80% (increments of 10%) --- ## Legend - **Location**: Top-right corner of the chart - **Entries**: - **Beam Search**: Blue - **Best-of-N Weighted**: Orange - **Majority**: Green --- ## Chart Components ### Main Chart - **Structure**: Grouped bar chart with three methods (Beam Search, Best-of-N Weighted, Majority) compared across five difficulty bins. - **Bars**: - Each bin (1–5) contains three stacked segments corresponding to the methods. - Colors match the legend (blue, orange, green). --- ## Data Extraction and Trends ### Bin 1 (Easiest) - **Beam Search**: ~70% accuracy (blue segment) - **Best-of-N Weighted**: ~65% accuracy (orange segment) - **Majority**: ~45% accuracy (green segment) - **Trend**: Beam Search dominates, followed by Best-of-N Weighted, then Majority. ### Bin 2 (Moderate) - **Beam Search**: ~50% accuracy - **Best-of-N Weighted**: ~45% accuracy - **Majority**: ~30% accuracy - **Trend**: Beam Search maintains lead, but gaps narrow compared to Bin 1. ### Bin 3 (Medium-Hard) - **Beam Search**: ~20% accuracy - **Best-of-N Weighted**: ~15% accuracy - **Majority**: ~10% accuracy - **Trend**: All methods decline sharply, with Beam Search still outperforming others. ### Bin 4 (Hard) - **Beam Search**: ~10% accuracy - **Best-of-N Weighted**: ~5% accuracy - **Majority**: ~3% accuracy - **Trend**: Accuracy drops further; Beam Search remains the best performer. ### Bin 5 (Hardest) - **Beam Search**: ~5% accuracy - **Best-of-N Weighted**: ~3% accuracy - **Majority**: ~1% accuracy - **Trend**: Minimal accuracy across all methods; Beam Search still leads marginally. --- ## Key Observations 1. **Performance Degradation**: All methods show decreasing accuracy as difficulty increases (Bins 1 → 5). 2. **Method Comparison**: - **Beam Search** consistently outperforms other methods across all bins. - **Best-of-N Weighted** performs better than Majority but lags behind Beam Search. - **Majority** (baseline) shows the lowest accuracy in all bins. 3. **Unsupervised Difficulty Bins**: The binning strategy groups questions by difficulty, with Bin 1 being easiest and Bin 5 hardest. --- ## Spatial Grounding - **Legend**: Top-right corner (confirmed via visual inspection). - **Color Consistency**: - Blue segments = Beam Search (matches legend). - Orange segments = Best-of-N Weighted (matches legend). - Green segments = Majority (matches legend). --- ## Conclusion The chart demonstrates that **Beam Search** achieves the highest MATH Test Accuracy across all difficulty bins, while **Majority** serves as a weak baseline. Accuracy declines uniformly with increasing difficulty, but Beam Search maintains a clear advantage.