## Chart: Model Performance vs. Annotation Cost ### Overview The image is a chart comparing the performance (Average F1 Score) of three different models (ActPRM, UniversalPRM, and Qwen2.5-Math-PRM-7B) against their estimated annotation cost (in generated tokens). The chart shows how the F1 score increases with the annotation cost for ActPRM, and then compares the F1 scores of the other two models at specific annotation cost levels relative to ActPRM. ### Components/Axes * **X-axis:** Est. Annotation Cost (Gen. Tokens). The scale is logarithmic, with markers at 2²⁸, 2³⁰, 2³², and 2³⁴. * **Y-axis:** Average F1 Score. The scale ranges from 0.68 to 0.76, with markers at 0.68, 0.70, 0.72, 0.74, and 0.76. * **Legend (bottom-right):** * Red Star: ActPRM * Teal Star: UniversalPRM * Dark Blue Star: Qwen2.5-Math-PRM-7B ### Detailed Analysis * **ActPRM (Red Line):** The red line shows the performance of the ActPRM model as the annotation cost increases. * At approximately 2²⁸ tokens, the F1 score is around 0.68. * The line slopes upward, reaching an F1 score of 0.750 at 2³⁰ tokens. * A dashed vertical line extends downward from the ActPRM data point at 2³⁰. * **UniversalPRM (Teal Star):** The UniversalPRM model has an F1 score of 0.743. * The annotation cost for UniversalPRM is 4.8x the cost of ActPRM at 2³⁰ tokens. * The x-axis value for UniversalPRM is approximately 2³². * **Qwen2.5-Math-PRM-7B (Dark Blue Star):** The Qwen2.5-Math-PRM-7B model has an F1 score of 0.735. * The annotation cost for Qwen2.5-Math-PRM-7B is 17.3x the cost of ActPRM at 2³⁰ tokens. * The x-axis value for Qwen2.5-Math-PRM-7B is approximately 2³⁴. ### Key Observations * The ActPRM model's performance increases significantly as the annotation cost increases from 2²⁸ to 2³⁰ tokens. * UniversalPRM and Qwen2.5-Math-PRM-7B achieve lower F1 scores than ActPRM at 2³⁰, but at significantly higher annotation costs. * The annotation cost increases exponentially for UniversalPRM and Qwen2.5-Math-PRM-7B compared to ActPRM. ### Interpretation The chart suggests that while ActPRM's performance initially increases with annotation cost, there are diminishing returns. UniversalPRM and Qwen2.5-Math-PRM-7B achieve lower F1 scores than ActPRM at 2³⁰, but at significantly higher annotation costs. This indicates that ActPRM is more efficient in terms of performance per annotation cost, at least up to the 2³⁰ token mark. The 4.8x and 17.3x cost multipliers highlight the trade-offs between model complexity, annotation effort, and performance. The data implies that a simpler model with targeted annotation might be more effective than a more complex model requiring extensive annotation.

## Line Chart: Average F1 Score vs. Estimated Annotation Cost ### Overview This chart displays the relationship between the estimated annotation cost (in Generation Tokens) and the average F1 score for three different models: ActPRM, UniversalPRM, and Qwen2.5-Math-PRM-7B. The chart shows how the F1 score changes as the annotation cost increases. A dashed line indicates a cost of 17.3x. ### Components/Axes * **X-axis:** Est. Annotation Cost (Gen. Tokens). Scale ranges from approximately 227 to 235. Marked values are 228, 230, 232, and 234. * **Y-axis:** Average F1 Score. Scale ranges from approximately 0.68 to 0.76. Marked values are 0.68, 0.70, 0.72, 0.74, 0.75, and 0.76. * **Legend:** Located in the bottom-right corner. * ActPRM (Red circle marker) * UniversalPRM (Blue star marker) * Qwen2.5-Math-PRM-7B (Dark blue star marker) * **Dashed Vertical Line:** Located at approximately x = 230, indicating a cost threshold. * **Text Labels:** * "0.750" at approximately (230, 0.75) * "4.8x Cost" above the ActPRM line near x=230 * "17.3x Cost" above the dashed line near x=230 * "0.743" at approximately (232, 0.743) * "0.735" at approximately (234, 0.735) ### Detailed Analysis * **ActPRM (Red Line):** The line starts at approximately (228, 0.68) and increases sharply until reaching a peak at approximately (230, 0.750). After x=230, the line decreases slightly to approximately (232, 0.743) and then decreases further to approximately (234, 0.735). * **UniversalPRM (Blue Line):** The line starts at approximately (228, 0.72) and increases to approximately (230, 0.74). After x=230, the line remains relatively flat at approximately 0.74. * **Qwen2.5-Math-PRM-7B (Dark Blue Line):** The line starts at approximately (228, 0.70) and increases to approximately (230, 0.73). After x=230, the line remains relatively flat at approximately 0.73. ### Key Observations * ActPRM demonstrates the most significant improvement in F1 score with increasing annotation cost, peaking at x=230. However, its performance declines slightly after this point. * UniversalPRM and Qwen2.5-Math-PRM-7B show a more gradual increase in F1 score with annotation cost, and their performance plateaus after x=230. * The dashed vertical line at x=230 appears to mark a point of diminishing returns for ActPRM, as the F1 score begins to decrease after this cost. ### Interpretation The chart suggests that increasing the estimated annotation cost initially leads to substantial improvements in the F1 score, particularly for the ActPRM model. However, there is a point (around 230 Gen. Tokens) where further increases in annotation cost yield diminishing returns, and the F1 score may even decrease. This could indicate that the model has reached its optimal performance level with the available data and that additional annotation effort is not effectively improving its accuracy. The relatively stable performance of UniversalPRM and Qwen2.5-Math-PRM-7B suggests they may be less sensitive to annotation cost or have already reached their performance limits. The "4.8x Cost" and "17.3x Cost" labels likely refer to the relative cost increase compared to a baseline, and the dashed line highlights a cost threshold where the benefit of further annotation diminishes. The chart provides valuable insights into the trade-off between annotation cost and model performance, helping to optimize resource allocation for model training.

## Line Chart: Average F1 Score vs. Estimated Annotation Cost ### Overview The chart compares the performance (Average F1 Score) of three language models (ActPRM, UniversalPRM, Qwen2.5-Math-PRM-7B) against their estimated annotation costs (in generated tokens). Performance improves with higher annotation costs, but trade-offs exist between cost and efficiency. ### Components/Axes - **X-axis**: "Est. Annotation Cost (Gen. Tokens)" (logarithmic scale: 2²⁸ to 2³⁴) - **Y-axis**: "Average F1 Score" (linear scale: 0.68 to 0.76) - **Legend**: Located in the bottom-right corner, mapping colors to models: - Red star: ActPRM - Teal star: UniversalPRM - Blue star: Qwen2.5-Math-PRM-7B - **Annotations**: - Red line (ActPRM) peaks at 0.750 (2³⁰ tokens) - Teal line (UniversalPRM) at 0.743 (2³² tokens) - Blue line (Qwen2.5) at 0.735 (2³⁴ tokens) - Dashed lines indicate cost multipliers: "4.8× Cost" (2³⁰) and "17.3× Cost" (2³²) ### Detailed Analysis 1. **ActPRM (Red Line)**: - Starts at 0.68 (2²⁸ tokens) and rises sharply to 0.750 (2³⁰ tokens). - Shows the steepest improvement in F1 score per token. - Annotated with a star at its peak (0.750). 2. **UniversalPRM (Teal Line)**: - Flat line at 0.743, positioned at 2³² tokens. - Cost multiplier: 4.8× higher than ActPRM’s baseline (2³⁰ vs. 2³²). 3. **Qwen2.5-Math-PRM-7B (Blue Line)**: - Flat line at 0.735, positioned at 2³⁴ tokens. - Cost multiplier: 17.3× higher than ActPRM’s baseline (2³⁴ vs. 2³⁰). ### Key Observations - **ActPRM** achieves the highest F1 score (0.750) at the lowest cost (2³⁰ tokens), outperforming others in efficiency. - **UniversalPRM** and **Qwen2.5** have lower F1 scores (0.743 and 0.735, respectively) but require significantly higher annotation costs (4.8× and 17.3× more tokens than ActPRM). - All models plateau after their respective cost thresholds, suggesting diminishing returns beyond certain annotation budgets. ### Interpretation - **Cost-Effectiveness**: ActPRM offers the best balance of performance and cost, making it the most efficient model for annotation tasks. - **Trade-offs**: UniversalPRM and Qwen2.5 sacrifice cost efficiency for marginally lower performance gains, which may not justify their higher resource demands. - **Diminishing Returns**: The flat lines for UniversalPRM and Qwen2.5 imply that increasing annotation costs beyond 2³² and 2³⁴ tokens yields no further F1 score improvements, highlighting potential inefficiencies in scaling. - **Strategic Implications**: Organizations prioritizing cost savings should favor ActPRM, while those requiring absolute maximum performance might consider the trade-offs of UniversalPRM or Qwen2.5 despite their higher costs.