# Technical Document Extraction: Pass Rate Analysis by Patch Size
## Chart 1: Pass Rate Comparison (Self-Play vs. Majority Voting)
### Axes and Labels
- **X-axis**: "Number of patches: BF x TW"
- Categories: `1x1`, `3x3`, `5x5`, `10x10`, `20x20`, `40x40`
- **Y-axis**: "Pass Rate (%)"
- Range: 45% to 62.5%
### Legend
- **Blue line**: Self-play
- **Green line**: Majority Voting
### Data Points and Trends
1. **Self-Play (Blue)**
- **Trend**: Steadily increasing with patch size.
- **Values**:
- `1x1`: 48.0%
- `3x3`: 52.6%
- `5x5`: 55.4%
- `10x10`: 58.8%
- `20x20`: 60.4%
- `40x40`: 60.4%
2. **Majority Voting (Green)**
- **Trend**: Gradual increase, plateauing at larger patch sizes.
- **Values**:
- `1x1`: 48.0%
- `3x3`: 48.8%
- `5x5`: 50.0%
- `10x10`: 51.0%
- `20x20`: 51.4%
- `40x40`: 51.2%
### Spatial Grounding
- Legend positioned in the **top-right corner** of the chart.
- All data points match legend colors:
- Blue circles (self-play) align with blue line.
- Green triangles (majority voting) align with green line.
---
## Chart 2: Pass Rate Comparison (Self-Play vs. Pass@N)
### Axes and Labels
- **X-axis**: "Number of patches: BF x TW"
- Categories: `1x1`, `3x3`, `5x5`, `10x10`, `20x20`, `40x40`
- **Y-axis**: "Pass Rate (%)"
- Range: 45% to 75%
### Legend
- **Blue line**: Self-play
- **Orange line**: Pass@N
### Data Points and Trends
1. **Self-Play (Blue)**
- **Trend**: Steady increase, plateauing at larger patch sizes.
- **Values**:
- `1x1`: 48.0%
- `3x3`: 52.6%
- `5x5`: 55.4%
- `10x10`: 58.8%
- `20x20`: 60.4%
- `40x40`: 60.4%
2. **Pass@N (Orange)**
- **Trend**: Sharp upward trajectory, outperforming self-play at all patch sizes.
- **Values**:
- `1x1`: 48.0%
- `3x3`: 60.4%
- `5x5`: 64.0%
- `10x10`: 67.4%
- `20x20`: 71.6%
- `40x40`: 74.8%
### Spatial Grounding
- Legend positioned in the **top-right corner** of the chart.
- All data points match legend colors:
- Blue circles (self-play) align with blue line.
- Orange diamonds (Pass@N) align with orange line.
---
## Key Observations
1. **Self-Play Performance**
- Both charts show self-play pass rates plateauing at `20x20` and `40x40` patches (~60.4%).
- Consistent across both metrics (majority voting and Pass@N).
2. **Pass@N Advantage**
- Pass@N significantly outperforms self-play, especially at larger patch sizes (e.g., 74.8% vs. 60.4% at `40x40`).
- Pass@N demonstrates a steeper growth curve compared to self-play.
3. **Majority Voting Limitation**
- Majority voting shows minimal improvement beyond `5x5` patches, suggesting diminishing returns.
## Conclusion
- Larger patch sizes improve pass rates for all methods, but **Pass@N** achieves the highest performance, particularly at scale.
- Self-play and majority voting exhibit similar trends but lag behind Pass@N in effectiveness.
