# Technical Document Extraction: Solve Rate Comparison Chart
## Chart Type
Bar chart comparing solve rates with and without natural language (NL) steps across multiple code generation tools.
## Axis Labels
- **X-axis**: Tool names (categories)
- AutoCodeRover-v2
- RepoGraph
- Moatless-Claude-3.5
- CodeStoryAide
- MarsCode
- Honeycomb
- Agentless
- **Y-axis**: Solve rate (0.0 to 0.7 in increments of 0.1)
## Legend
- **Blue bars**: "contains NL steps"
- **Purple bars**: "without steps"
## Data Points
| Tool | Contains NL Steps (Blue) | Without Steps (Purple) |
|-----------------------|--------------------------|------------------------|
| AutoCodeRover-v2 | ~0.55 | ~0.25 |
| RepoGraph | ~0.48 | ~0.27 |
| Moatless-Claude-3.5 | ~0.69 | ~0.40 |
| CodeStoryAide | ~0.65 | ~0.37 |
| MarsCode | ~0.74 | ~0.31 |
| Honeycomb | ~0.52 | ~0.30 |
| Agentless | ~0.53 | ~0.29 |
## Key Trends
1. **NL Steps Impact**: All tools show significantly higher solve rates when NL steps are included (blue bars) compared to when they are excluded (purple bars).
2. **Performance Thresholds**: Red dashed reference lines at 0.3, 0.4, 0.5, 0.6, and 0.7 indicate performance benchmarks. Most tools with NL steps exceed the 0.5 threshold.
3. **Top Performers**:
- MarsCode achieves the highest solve rate with NL steps (~0.74).
- Moatless-Claude-3.5 shows the largest improvement with NL steps (~0.69 vs. ~0.40 without).
4. **Lowest Performers**:
- RepoGraph has the lowest solve rate without NL steps (~0.27).
- Moatless-Claude-3.5 has the lowest solve rate with NL steps (~0.69) among tools with NL steps.
## Additional Observations
- **Red Dashed Lines**: Positioned at 0.3, 0.4, 0.5, 0.6, and 0.7 on the y-axis, likely representing target solve rate thresholds.
- **Consistency**: All tools demonstrate a positive correlation between NL step inclusion and solve rate improvement.
