# Technical Data Extraction: Comparing Test-time and Pretraining Compute
## 1. Document Overview
This image contains two line graphs comparing the effectiveness of increasing test-time compute versus pretraining compute across different problem difficulty levels on the MATH dataset.
* **Main Title:** Comparing Test-time and Pretraining Compute
* **Sub-charts:**
1. **Revisions** (Left)
2. **PRM Search** (Right)
* **Language:** English
---
## 2. Component Isolation
### A. Global Legend & Scales
* **Y-Axis (Both):** MATH Difficulty Level Accuracy (%)
* **Range:** 0 to 100 (Left chart), 0 to 80+ (Right chart).
* **Markers:** 0, 20, 40, 60, 80, 100.
* **X-Axis (Both):** Proportional to Inference FLOPs
* **Scale:** Logarithmic (Base 2).
* **Markers:** $2^0, 2^1, 2^2, 2^3, 2^4, 2^5, 2^6, 2^7, 2^8$.
* **Color Gradient Legend (Right Side):** Difficulty Level
* **Scale:** 1 (Purple/Top) to 5 (Blue/Bottom).
* **Mapping:**
* Level 1: Purple
* Level 2: Red/Pink
* Level 3: Green
* Level 4: Orange/Brown
* Level 5: Blue
* **Bottom Legend (Symbols & Lines):**
* **Star ($\star$):** Pretraining Compute
* **Circle-Line ($\bullet$):** Test-time Compute
* **Blue Dashed Vertical Line:** $R \gg 1$
* **Orange Dashed Vertical Line:** $R \approx 1$
* **Green Dashed Vertical Line:** $R \ll 1$
---
## 3. Data Analysis: Revisions (Left Chart)
### Trend Verification
All data series show a positive correlation between inference FLOPs and accuracy. The slope is steepest for Level 1 (Purple) and Level 2 (Red), while Level 5 (Blue) remains nearly flat near 0% accuracy.
### Data Points (Approximate Values)
| Difficulty Level (Color) | $2^0$ Accuracy | $2^8$ Accuracy | Trend Description |
| :--- | :--- | :--- | :--- |
| **1 (Purple)** | ~60% | ~90% | Rapid ascent, plateaus after $2^4$. |
| **2 (Red)** | ~22% | ~62% | Steady upward slope. |
| **3 (Green)** | ~8% | ~45% | Steady upward slope. |
| **4 (Orange)** | ~2% | ~18% | Shallow upward slope. |
| **5 (Blue)** | ~1% | ~3% | Nearly flat; very low accuracy. |
### Pretraining Compute ($\star$) Comparison
Stars are placed at $2^4, 2^6,$ and $2^8$ for various levels.
* At $2^8$, the test-time compute (circles) generally matches or slightly exceeds the pretraining compute (stars) for levels 2, 3, and 4.
---
## 4. Data Analysis: PRM Search (Right Chart)
### Trend Verification
The "PRM Search" method shows a much sharper "elbow" in the curve. Accuracy gains are very high between $2^0$ and $2^2$ for Level 1, then plateau. Lower difficulty levels show more linear growth.
### Data Points (Approximate Values)
| Difficulty Level (Color) | $2^0$ Accuracy | $2^8$ Accuracy | Trend Description |
| :--- | :--- | :--- | :--- |
| **1 (Purple)** | ~38% | ~85% | Sharp jump to $2^2$, then plateaus. |
| **2 (Red)** | ~11% | ~60% | Consistent upward slope. |
| **3 (Green)** | ~4% | ~35% | Consistent upward slope. |
| **4 (Orange)** | ~1% | ~16% | Shallow upward slope. |
| **5 (Blue)** | ~0% | ~2% | Minimal gains. |
---
## 5. Key Observations & Vertical Markers
The vertical dashed lines represent the ratio ($R$) of compute efficiency.
1. **$R \gg 1$ (Blue, $2^4$):** Indicates a regime where one form of compute is significantly more efficient.
2. **$R \approx 1$ (Orange, $2^6$):** Indicates a "break-even" point where test-time and pretraining compute yields similar returns.
3. **$R \ll 1$ (Green, $2^8$):** Indicates the opposite regime of the blue line.
**Summary of Findings:**
* Increasing test-time compute (moving right on the x-axis) consistently improves accuracy across all difficulty levels, though the returns diminish as difficulty increases (Level 5 is hardest).
* **PRM Search** appears to reach higher performance ceilings faster for Level 1 problems compared to the **Revisions** method, but Revisions starts with a higher baseline at $2^0$ FLOPs.
* The stars ($\star$) indicate that for many difficulty levels, increasing test-time compute can achieve the same accuracy as models with significantly more pretraining compute.
