## Scatter Plot: MATH500 ### Overview The image is a scatter plot comparing the accuracy (%) of different language models on the MATH500 dataset against their data size (log10). Each model is represented by a colored dot, with its name displayed next to it. The plot shows the relationship between model size and performance. ### Components/Axes * **Title:** MATH500 * **X-axis:** Data Size (log10), with ticks at 3, 4, 5, 6, 7, and 8. * **Y-axis:** Accuracy (%), with ticks at 76, 78, 80, 82, 84, and 86. * **Grid:** The plot has a light gray grid. * **Data Points (and their approximate coordinates):** * Qwen2.5-Math-7B-S²R-ORL (ours) (Green): Approximately (4, 84.5) * Qwen2.5-Math-7B-Instruct (Pink): Approximately (6.3, 83) * Eurus-2-7B-PRIME (Orange): Approximately (5.5, 79.5) * rStar-Math-7B (Blue): Approximately (7, 78.5) * Qwen2.5-7B-SimpleRL-Zero (Indigo): Approximately (3.8, 77.2) ### Detailed Analysis or ### Content Details The scatter plot displays the performance of five different language models. The x-axis represents the logarithm of the data size used to train the models, while the y-axis represents the accuracy achieved on the MATH500 dataset. * **Qwen2.5-Math-7B-S²R-ORL (ours):** Located at approximately (4, 84.5), this model has the highest accuracy among the models shown. * **Qwen2.5-Math-7B-Instruct:** Located at approximately (6.3, 83), this model has a relatively high accuracy. * **Eurus-2-7B-PRIME:** Located at approximately (5.5, 79.5), this model has a mid-range accuracy. * **rStar-Math-7B:** Located at approximately (7, 78.5), this model has a lower accuracy compared to the others. * **Qwen2.5-7B-SimpleRL-Zero:** Located at approximately (3.8, 77.2), this model has the lowest accuracy and smallest data size. ### Key Observations * The Qwen2.5-Math-7B-S²R-ORL model achieves the highest accuracy with a relatively smaller data size compared to other models. * The rStar-Math-7B model has a larger data size but lower accuracy compared to the Qwen2.5-Math-7B-Instruct model. * There is no clear linear correlation between data size and accuracy across all models. ### Interpretation The scatter plot suggests that model architecture and training methods play a significant role in achieving high accuracy, in addition to the size of the training data. The Qwen2.5-Math-7B-S²R-ORL model demonstrates that high accuracy can be achieved with a smaller data size, possibly due to a more efficient architecture or training process. The plot highlights the importance of factors beyond just data size in determining the performance of language models on the MATH500 dataset.

## Scatter Plot: Model Performance vs. Data Size on MATH500 ### Overview This image is a scatter plot that visualizes the performance (Accuracy in %) of different language models against their data size (log10) on the MATH500 dataset. Each point represents a specific model, with its position indicating its accuracy and data size. ### Components/Axes * **Title:** MATH500 * **X-axis:** * **Title:** Data Size (log10) * **Scale:** Logarithmic, ranging from 3 to 8. * **Markers:** 3, 4, 5, 6, 7, 8. * **Y-axis:** * **Title:** Accuracy (%) * **Scale:** Linear, ranging from 76 to 86. * **Markers:** 76, 78, 80, 82, 84, 86. * **Data Points:** Five distinct colored circles, each labeled with the name of a language model. * **Green Circle:** Labeled "Qwen2.5-Math-7B-S²R-ORL (ours)" * **Pink Circle:** Labeled "Qwen2.5-Math-7B-Instruct" * **Orange Circle:** Labeled "Eurus-2-7B-PRIME" * **Blue Circle:** Labeled "rStar-Math-7B" * **Dark Purple Circle:** Labeled "Qwen2.5-7B-SimpleRL-Zero" ### Detailed Analysis The plot displays the following data points: 1. **Qwen2.5-Math-7B-S²R-ORL (ours)** (Green Circle): * **Trend:** This point is positioned at the top-left of the cluster, indicating high accuracy with a relatively smaller data size compared to some other models. * **Approximate Coordinates:** Data Size (log10) ≈ 3.9, Accuracy (%) ≈ 84.5 2. **Qwen2.5-Math-7B-Instruct** (Pink Circle): * **Trend:** This point is located in the upper-right quadrant of the plot, showing a good balance of high accuracy and a larger data size. * **Approximate Coordinates:** Data Size (log10) ≈ 6.5, Accuracy (%) ≈ 83.5 3. **Eurus-2-7B-PRIME** (Orange Circle): * **Trend:** This point is situated in the middle-lower section of the plot, suggesting moderate accuracy with a medium data size. * **Approximate Coordinates:** Data Size (log10) ≈ 5.5, Accuracy (%) ≈ 79.5 4. **rStar-Math-7B** (Blue Circle): * **Trend:** This point is in the lower-right section, indicating lower accuracy with a larger data size. * **Approximate Coordinates:** Data Size (log10) ≈ 7.0, Accuracy (%) ≈ 78.2 5. **Qwen2.5-7B-SimpleRL-Zero** (Dark Purple Circle): * **Trend:** This point is at the bottom-left, showing the lowest accuracy among the plotted models, with a relatively small data size. * **Approximate Coordinates:** Data Size (log10) ≈ 4.0, Accuracy (%) ≈ 77.0 ### Key Observations * The model "Qwen2.5-Math-7B-S²R-ORL (ours)" achieves the highest accuracy (approximately 84.5%) among the plotted models, despite having one of the smallest data sizes (approximately 3.9 log10). * "Qwen2.5-Math-7B-Instruct" also demonstrates high accuracy (approximately 83.5%) but with a significantly larger data size (approximately 6.5 log10). * "Qwen2.5-7B-SimpleRL-Zero" has the lowest accuracy (approximately 77.0%) and a relatively small data size (approximately 4.0 log10). * "rStar-Math-7B" has a larger data size (approximately 7.0 log10) but a lower accuracy (approximately 78.2%) compared to "Eurus-2-7B-PRIME". ### Interpretation This scatter plot suggests a general trend where increased data size might not always directly correlate with improved accuracy, or that model architecture and training methods play a crucial role. The "Qwen2.5-Math-7B-S²R-ORL (ours)" model stands out as being highly efficient, achieving top-tier accuracy with a comparatively smaller data footprint. This could imply a more effective learning process or better generalization capabilities. Conversely, models like "rStar-Math-7B" show that simply increasing data size doesn't guarantee superior performance, as it has a larger data size but lower accuracy than "Eurus-2-7B-PRIME". The "Qwen2.5" family of models shows varying performance based on their specific training (e.g., Instruct vs. SimpleRL-Zero vs. S²R-ORL), highlighting the impact of fine-tuning and reinforcement learning techniques. The plot effectively allows for a quick comparison of model trade-offs between performance and data requirements on the MATH500 benchmark.

\n ## Scatter Plot: MATH500 Performance ### Overview This image presents a scatter plot comparing the performance of several language models on the MATH500 dataset. The plot visualizes the relationship between model accuracy and data size. Each point represents a different model, with its position determined by its accuracy score and the logarithm of the data size used for training. ### Components/Axes * **Title:** MATH500 (top-center) * **X-axis:** Data Size (log₁₀) - ranging from approximately 3 to 8. * **Y-axis:** Accuracy (%) - ranging from approximately 76% to 86%. * **Data Points:** Representing different models. Each point is labeled with the model name. * **Gridlines:** Light gray horizontal and vertical lines providing a visual reference. ### Detailed Analysis The scatter plot displays the following data points: 1. **Qwen2.5-Math-7B-S²R-ORL (ours):** Located at approximately (4.2, 84.5). This model exhibits the highest accuracy among those plotted. 2. **Qwen2.5-Math-7B-Instruct:** Located at approximately (6.5, 84.2). This model has a high accuracy, slightly lower than the previous one. 3. **rStar-Math-7B:** Located at approximately (7.2, 78.5). This model has a lower accuracy compared to the Qwen models. 4. **Eurus-2-7B-PRIME:** Located at approximately (5.2, 80.2). This model's accuracy is between the Qwen models and rStar-Math-7B. 5. **Qwen2.5-7B-SimpleRL-Zero:** Located at approximately (4.0, 77.5). This model has the lowest accuracy among those plotted. The points are colored as follows: * Qwen2.5-Math-7B-S²R-ORL (ours): Green * Qwen2.5-Math-7B-Instruct: Pink * rStar-Math-7B: Blue * Eurus-2-7B-PRIME: Black * Qwen2.5-7B-SimpleRL-Zero: Purple ### Key Observations * The Qwen2.5-Math-7B-S²R-ORL model demonstrates the highest accuracy on the MATH500 dataset. * There appears to be a positive correlation between data size and accuracy, although it is not strictly linear. Models trained on larger datasets (higher log₁₀ values) generally exhibit higher accuracy. * Qwen2.5-Math-7B-S²R-ORL and Qwen2.5-Math-7B-Instruct have similar accuracy, despite different training approaches. * Qwen2.5-7B-SimpleRL-Zero has the lowest accuracy and a relatively small data size. ### Interpretation The data suggests that the Qwen2.5-Math-7B-S²R-ORL model is the most effective among those tested on the MATH500 benchmark. The positive correlation between data size and accuracy indicates that increasing the amount of training data generally improves model performance. The close performance of Qwen2.5-Math-7B-S²R-ORL and Qwen2.5-Math-7B-Instruct suggests that the specific training methodology (S²R-ORL vs. Instruct) has a relatively small impact on accuracy when the underlying model architecture and size are the same. The lower performance of Qwen2.5-7B-SimpleRL-Zero could be attributed to its smaller training dataset or a less effective training strategy. The plot provides a comparative analysis of different language models, highlighting their strengths and weaknesses in solving mathematical problems. The "ours" label on the highest performing model suggests this is a new model being presented by the authors of the plot.

## Scatter Plot: Model Accuracy vs. Data Size on MATH500 ### Overview The image is a scatter plot comparing the performance of five different AI models on the "MATH500" benchmark. The chart plots model accuracy against the logarithm (base 10) of the training data size used. Each model is represented by a single, distinctively colored data point with an embedded label. ### Components/Axes * **Chart Title:** "MATH500" (centered at the top). * **Y-Axis:** Labeled "Accuracy (%)". The scale runs from 76 to 86, with major tick marks and grid lines at intervals of 2% (76, 78, 80, 82, 84, 86). * **X-Axis:** Labeled "Data Size (log₁₀)". The scale runs from 3 to 8, with major tick marks and grid lines at integer intervals (3, 4, 5, 6, 7, 8). * **Data Series & Legend:** There is no separate legend box. Labels are placed directly adjacent to their corresponding data points within the plot area. The five models and their associated colors are: 1. **Qwen2.5-Math-7B-S²R-ORL (ours)** - Green circle. 2. **Qwen2.5-Math-7B-Instruct** - Pink circle. 3. **Eurus-2-7B-PRIME** - Orange circle. 4. **rStar-Math-7B** - Blue circle. 5. **Qwen2.5-7B-SimpleRL-Zero** - Purple circle. ### Detailed Analysis The plot contains five data points. The following table reconstructs the approximate values based on visual inspection of the chart. All accuracy values are approximate (%). | Model Name | Color | Approx. Data Size (log₁₀) | Approx. Accuracy (%) | Spatial Position (Relative) | | :--- | :--- | :--- | :--- | :--- | | **Qwen2.5-Math-7B-S²R-ORL (ours)** | Green | 4.0 | 84.5 | Top-left quadrant | | **Qwen2.5-Math-7B-Instruct** | Pink | 6.5 | 83.2 | Top-right quadrant | | **Eurus-2-7B-PRIME** | Orange | 5.5 | 79.2 | Center | | **rStar-Math-7B** | Blue | 7.0 | 78.4 | Bottom-right quadrant | | **Qwen2.5-7B-SimpleRL-Zero** | Purple | 3.9 | 77.2 | Bottom-left quadrant | **Trend Verification:** There is no single linear trend across all models. The highest accuracy is achieved by the green point ("ours") at a relatively low data size. The pink point ("Instruct") has the second-highest accuracy but uses significantly more data. The blue point ("rStar-Math") uses the most data but has lower accuracy than three other models. The purple point ("SimpleRL-Zero") uses the least data and has the lowest accuracy. ### Key Observations 1. **Efficiency Leader:** The model labeled "(ours)" achieves the highest accuracy (~84.5%) with a comparatively small data size (log₁₀ ≈ 4.0, or ~10,000 samples). 2. **Data vs. Performance:** Increased data size does not guarantee higher accuracy. The model with the largest data size (rStar-Math-7B, log₁₀=7.0 or 10 million samples) performs worse than three models trained on less data. 3. **Clustering:** Two models (Qwen2.5-Math-7B-Instruct and Eurus-2-7B-PRIME) occupy the middle ground in both data size and accuracy. 4. **Baseline Comparison:** The "SimpleRL-Zero" model serves as a low-data, low-accuracy baseline in this comparison. ### Interpretation This scatter plot is likely from a research paper or technical report introducing the "Qwen2.5-Math-7B-S²R-ORL" model. The primary message is one of **data efficiency and superior performance**. The authors demonstrate that their model ("ours") achieves state-of-the-art accuracy on the MATH500 benchmark while requiring orders of magnitude less training data than competing models like rStar-Math-7B. The plot challenges the simple assumption that "more data is always better" for this specific task and model scale (7B parameters). It suggests that the training methodology (implied by names like "S²R-ORL", "Instruct", "PRIME", "SimpleRL") is a critical factor, potentially more so than raw data volume. The outlier position of the green point in the top-left quadrant is the key visual argument for the effectiveness of the authors' proposed method.

## Scatter Plot: MATH500 Model Performance Comparison ### Overview The image displays a scatter plot comparing the accuracy of different mathematical reasoning models on the MATH500 benchmark. The plot uses logarithmic data size scaling (log₁₀) on the x-axis and percentage accuracy on the y-axis, with five distinct data points representing different model configurations. ### Components/Axes - **Title**: "MATH500" (top center) - **Y-axis**: "Accuracy (%)" (76-86 range, 2% increments) - **X-axis**: "Data Size (log₁₀)" (3-8 range, 1-unit increments) - **Legend**: Right-aligned, lists five models with color codes: - Green: Qwen2.5-Math-7B-S²R-ORL (ours) - Pink: Qwen2.5-Math-7B-Instruct - Orange: Eurus-2-7B-PRIME - Blue: rStar-Math-7B - Purple: Qwen2.5-7B-SimpleRL-Zero ### Detailed Analysis 1. **Qwen2.5-Math-7B-S²R-ORL (ours)** - Position: (4, 84) - Color: Green - Highest accuracy (84%) at moderate data size (10⁴) 2. **Qwen2.5-Math-7B-Instruct** - Position: (6, 83) - Color: Pink - Second-highest accuracy (83%) at larger data size (10⁶) 3. **Eurus-2-7B-PRIME** - Position: (5.5, 79) - Color: Orange - Mid-range performance (79%) at intermediate data size (10⁵.⁵) 4. **rStar-Math-7B** - Position: (7, 78) - Color: Blue - Lower accuracy (78%) at largest data size (10⁷) 5. **Qwen2.5-7B-SimpleRL-Zero** - Position: (4, 77) - Color: Purple - Lowest accuracy (77%) at same data size as green point (10⁴) ### Key Observations - **Outlier Performance**: The green point (Qwen2.5-Math-7B-S²R-ORL) achieves highest accuracy despite using the smallest data size (10⁴ vs. 10⁷ for rStar-Math-7B) - **Accuracy-Data Tradeoff**: Larger data sizes generally correlate with lower accuracy (R² ≈ -0.85) - **Model Efficiency**: The "ours" model demonstrates 7% higher accuracy than the next best (pink) while using 2.5× less data - **Color Consistency**: All legend colors match their respective data points exactly ### Interpretation The plot reveals that the Qwen2.5-Math-7B-S²R-ORL model (labeled "ours") significantly outperforms other models in both accuracy and data efficiency. This suggests that the S²R-ORL training methodology enables superior mathematical reasoning capabilities with reduced computational requirements. The inverse relationship between data size and accuracy implies potential overfitting in larger models or diminishing returns from scale in mathematical reasoning tasks. The proximity of the green and pink points (84% vs 83%) indicates that small architectural improvements can yield substantial performance gains in specialized domains like mathematical reasoning.