## Box Plot Comparison: Human Performance and Speedup Rates ### Overview The image contains two side-by-side box plots comparing performance metrics across different AI models. Panel (a) shows "Percentage of Human Beaten (%)" for four methods, while panel (b) displays "Speedup rate" for two methods. Dashed reference lines indicate human performance benchmarks in panel (a). ### Components/Axes **Panel (a):** - **X-axis (Method):** - GPT-4o (orig) [light blue] - GPT-4o (after) [dark blue] - o1-preview (orig) [peach] - o1-preview (after) [orange] - **Y-axis (Percentage of Human Beaten (%)):** 20–100% range - **Dashed Reference Lines:** - Red (~75%): Top 25% human performance - Blue (~50%): Median human performance - Purple (~25%): Bottom 25% human performance - **Legend:** Right-aligned, matching box colors to methods **Panel (b):** - **X-axis (Method):** - GPT-4o [blue] - o1-preview [orange] - **Y-axis (Speedup rate):** 0.0–1.0 range - **Legend:** Implied via color consistency with panel (a) ### Detailed Analysis **Panel (a):** - **GPT-4o (orig):** Median ~50%, range 20–95%, outliers near 15% and 100% - **GPT-4o (after):** Median ~65%, range 30–90%, tighter distribution - **o1-preview (orig):** Median ~60%, range 40–85%, wider spread - **o1-preview (after):** Median ~75%, range 50–95%, highest performance - **Key Thresholds:** - Top 25% human performance (75%) reached by o1-preview (after) - Median human performance (50%) exceeded by all methods except GPT-4o (orig) **Panel (b):** - **GPT-4o:** Median ~0.85, range 0.75–0.95, single outlier at 0.0 - **o1-preview:** Median ~0.9, range 0.8–1.0, no outliers - **Speedup Rate:** o1-preview consistently outperforms GPT-4o ### Key Observations 1. **Performance Improvement:** - o1-preview (after) achieves ~75% human performance, surpassing GPT-4o (after) at ~65% - GPT-4o (orig) has the lowest median performance (~50%) and widest variance 2. **Speedup Correlation:** - Higher human performance (panel a) correlates with higher speedup rates (panel b) - o1-preview achieves ~6% higher median speedup than GPT-4o 3. **Anomaly:** - GPT-4o has a single outlier at 0.0 speedup rate, suggesting potential data inconsistency ### Interpretation The data demonstrates that o1-preview (after) significantly outperforms GPT-4o in both human performance metrics and computational efficiency. The dashed reference lines in panel (a) contextualize AI performance against human benchmarks, showing o1-preview (after) approaching the top 25% human performance tier. The speedup rate in panel (b) reinforces this hierarchy, with o1-preview achieving near-optimal efficiency. The GPT-4o outlier at 0.0 speedup rate warrants investigation, as it contradicts the general trend of positive speedup values. These findings suggest o1-preview represents a substantial advancement over GPT-4o in both capability and efficiency.