# Technical Data Extraction: Revision Model Performance Analysis This document contains a detailed extraction of data from two side-by-side technical charts evaluating the performance of a "Revision Model" on the MATH test suite. --- ## Chart 1: Revision Model Pass@1 At Each Step ### Metadata and Axis Labels * **Title:** Revision Model Pass@1 At Each Step * **Y-Axis Label:** MATH Test Accuracy (%) * **Range:** 17 to 26 * **Markers:** 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 * **X-Axis Label:** Number of Generations * **Range:** 0 to 65+ * **Markers:** 0, 10, 20, 30, 40, 50, 60 ### Component Analysis: Scatter Plot * **Trend Verification:** The data points show a logarithmic-style growth pattern. There is a sharp increase in accuracy from generation 1 to approximately generation 15, followed by a plateau with high variance (noise) between 23% and 25% accuracy for the remainder of the steps. * **Key Data Points (Approximate):** * **Start:** ~18.2% at Generation 1. * **Initial Growth:** Reaches ~21.5% by Generation 5; ~24% by Generation 15. * **Outlier:** A notable dip occurs around Generation 15, dropping to ~20.4% before recovering. * **Peak:** The highest recorded accuracy appears to be ~25.2% at approximately Generation 51. * **End:** ~23.4% at Generation 64. --- ## Chart 2: Revision Model Parallel Verses Sequential ### Metadata and Axis Labels * **Title:** Revision Model Parallel Verses Sequential * **Y-Axis Label:** MATH Test Accuracy (%) * **Range:** 20 to 40 (Actual markers: 20, 25, 30, 35, 40) * **X-Axis Label:** Number of Generations (Logarithmic Scale) * **Markers:** $2^0$ (1), $2^1$ (2), $2^2$ (4), $2^3$ (8), $2^4$ (16), $2^5$ (32), $2^6$ (64) ### Legend and Spatial Grounding The legend is located in the upper-left quadrant of the chart area. * **Blue Line with Circle Marker:** Sequential Best-of-N Weighted * **Orange Line with Circle Marker:** Parallel Best-of-N Weighted * **Blue Line with Diamond/Small Circle:** Sequential Majority * **Orange Line with Diamond/Small Circle:** Parallel Majority ### Trend Verification and Data Extraction All four series show a strong upward trend as the number of generations increases. Sequential methods consistently outperform their parallel counterparts across both voting/weighting schemes. #### 1. Best-of-N Weighted Series (Top Two Lines) * **Trend:** These are the highest-performing methods. The gap between Sequential and Parallel is narrowest at $2^1$ and widest at $2^6$. * **Sequential Best-of-N Weighted (Dark Blue):** * $2^0$: ~18.5% * $2^6$: ~41.5% (Highest overall performance) * **Parallel Best-of-N Weighted (Dark Orange):** * $2^0$: ~18.5% * $2^6$: ~39.5% #### 2. Majority Series (Bottom Two Lines) * **Trend:** These follow a similar trajectory but at a lower accuracy offset (approx. 4-5% lower than Best-of-N). * **Sequential Majority (Light Blue):** * $2^0$: ~18.2% * $2^1$: ~18.5% (Stagnant initial growth) * $2^6$: ~37.5% * **Parallel Majority (Light Orange):** * $2^0$: ~18.2% * $2^1$: ~19.5% * $2^6$: ~35.0% ### Summary of Findings 1. **Sequential Advantage:** In the comparison of Parallel vs. Sequential, the Sequential approach provides a consistent performance boost of roughly 2-3 percentage points at higher generation counts. 2. **Methodology Impact:** "Best-of-N Weighted" significantly outperforms "Majority" voting regardless of whether the process is parallel or sequential. 3. **Scaling:** Accuracy scales effectively with the number of generations, showing no signs of a hard plateau within the $2^6$ (64) generation limit on the logarithmic chart.