## Line Chart: ARC-C ### Overview The image is a line chart comparing the accuracy of different models (Step-Level Online, Instance-Level Online, Step-Level Offline, and SFT Baseline) against varying percentages of training data. The x-axis represents the percentage of training data, and the y-axis represents accuracy. ### Components/Axes * **Title:** ARC-C * **X-axis:** Training Data % (with ticks at 10, 20, 30, 40, and 50) * **Y-axis:** Accuracy (with ticks at 55, 60, 65, 70, 75) * **Legend:** Located in the bottom-left corner. * Step-Level (Online) - Green line with star markers * Instance-Level (Online) - Blue line with triangle markers * Step-Level (Offline) - Yellow line with star markers * SFT Baseline - Dashed magenta line ### Detailed Analysis * **Step-Level (Online) - Green:** The line starts at approximately 72.2% accuracy with 10% training data. It increases to 74.7% at 20% training data, peaks at 76.4% at 30% training data, then decreases slightly to 75.6% at 40% training data, and ends at 75.8% at 50% training data. * **Instance-Level (Online) - Blue:** The line starts at 66.5% accuracy with 10% training data. It increases to 72.2% at 20% training data, peaks at 73.3% at 30% training data, increases to 75.2% at 40% training data, and then decreases to 73.4% at 50% training data. * **Step-Level (Offline) - Yellow:** The line starts at 69.2% accuracy with 10% training data. It increases to 70.8% at 20% training data, then decreases to 67.3% at 30% training data, and further decreases to 66.5% at 40% training data. * **SFT Baseline - Magenta:** The line is horizontal and constant at 60.6% accuracy across all training data percentages. ### Key Observations * Step-Level (Online) consistently outperforms the other models, maintaining the highest accuracy across all training data percentages. * Instance-Level (Online) shows a significant increase in accuracy with increasing training data up to 40%, after which it slightly decreases. * Step-Level (Offline) initially increases in accuracy but then decreases as the training data percentage increases. * SFT Baseline remains constant, indicating no improvement with increased training data. ### Interpretation The data suggests that Step-Level (Online) is the most effective model for this task, as it consistently achieves the highest accuracy. Instance-Level (Online) also performs well, showing improvement with more training data. Step-Level (Offline) appears to be less effective, as its accuracy decreases with higher training data percentages. The SFT Baseline serves as a control, demonstrating a fixed level of accuracy regardless of the amount of training data. The relationship between the models highlights the importance of the chosen approach (Step-Level vs. Instance-Level) and whether the training is done online or offline. The trends indicate that online step-level learning is the most beneficial for this particular task and dataset.

## Line Chart: ARC-C Accuracy vs. Training Data Percentage ### Overview This line chart displays the accuracy of different training methods (Step-Level Online, Instance-Level Online, Step-Level Offline, and SFT Baseline) on the ARC-C dataset as a function of the percentage of training data used. The chart aims to demonstrate how performance scales with increasing data availability for each method. ### Components/Axes * **Title:** ARC-C (positioned at the top-center) * **X-axis:** Training Data % (ranging from approximately 5% to 55%, with markers at 10, 20, 30, 40, and 50) * **Y-axis:** Accuracy (ranging from approximately 55 to 80, with markers at 60, 65, 70, 75) * **Legend:** Located at the bottom-left corner, containing the following labels and corresponding colors: * Step-Level (Online) - Green * Instance-Level (Online) - Blue * Step-Level (Offline) - Yellow/Orange * SFT Baseline - Magenta/Purple (dashed line) ### Detailed Analysis The chart contains four distinct lines representing the accuracy of each training method. * **Step-Level (Online) - Green:** This line exhibits an upward trend, starting at approximately 69.2 accuracy at 10% training data and reaching approximately 75.8 accuracy at 50% training data. Specific data points are: * 10%: 69.2 * 20%: 74.7 * 30%: 76.4 * 40%: 75.6 * 50%: 75.8 * **Instance-Level (Online) - Blue:** This line also shows an upward trend, beginning at approximately 72.2 accuracy at 10% training data and reaching approximately 73.4 accuracy at 50% training data. Specific data points are: * 10%: 72.2 * 20%: 72.2 * 30%: 73.3 * 40%: 75.2 * 50%: 73.4 * **Step-Level (Offline) - Yellow/Orange:** This line initially increases, then decreases. It starts at approximately 66.5 accuracy at 10% training data, peaks at approximately 70.8 accuracy at 20% training data, and then declines to approximately 66.5 accuracy at 50% training data. Specific data points are: * 10%: 66.5 * 20%: 70.8 * 30%: 67.3 * 40%: 66.5 * 50%: 66.5 * **SFT Baseline - Magenta/Purple (dashed line):** This line is horizontal, indicating a constant accuracy of approximately 60.6 across all training data percentages. ### Key Observations * The "Step-Level (Online)" and "Instance-Level (Online)" methods consistently outperform the "Step-Level (Offline)" method and the "SFT Baseline." * The "Step-Level (Offline)" method shows an initial improvement with increasing data, but then plateaus and even slightly decreases. * The "SFT Baseline" remains constant, suggesting it is not affected by the amount of training data. * The "Instance-Level (Online)" method shows a relatively flat performance curve, indicating diminishing returns with increased training data. ### Interpretation The data suggests that online training methods ("Step-Level (Online)" and "Instance-Level (Online)") are more effective than offline training ("Step-Level (Offline)") and the "SFT Baseline" for the ARC-C dataset. The "Step-Level (Online)" method achieves the highest accuracy, particularly with more training data. The "Instance-Level (Online)" method shows a more moderate improvement, while the "Step-Level (Offline)" method's performance is unstable. The constant performance of the "SFT Baseline" indicates that it may be limited by its initial parameters or training process. The diminishing returns observed with the "Instance-Level (Online)" method suggest that beyond a certain point, adding more training data does not significantly improve accuracy. This could be due to the model reaching its capacity or the data becoming redundant. The "Step-Level (Offline)" method's initial increase followed by a decrease could indicate overfitting to the initial training data.

## Line Chart: ARC-C Accuracy vs. Training Data Percentage ### Overview The image is a line chart titled "ARC-C" that plots the performance (Accuracy) of three different training methods and one baseline against the percentage of training data used. The chart compares "Step-Level (Online)", "Instance-Level (Online)", and "Step-Level (Offline)" methods against a fixed "SFT Baseline". ### Components/Axes * **Title:** "ARC-C" (centered at the top). * **Y-Axis:** Labeled "Accuracy". The scale runs from 55 to 75, with major gridlines at intervals of 5 (55, 60, 65, 70, 75). * **X-Axis:** Labeled "Training Data %". The scale runs from approximately 5 to 55, with labeled tick marks at 10, 20, 30, 40, and 50. * **Legend:** Located in the bottom-left quadrant of the plot area. It contains four entries: 1. **Step-Level (Online):** Represented by a solid green line with star (★) markers. 2. **Instance-Level (Online):** Represented by a solid blue line with upward-pointing triangle (▲) markers. 3. **Step-Level (Offline):** Represented by a solid yellow/gold line with star (★) markers. 4. **SFT Baseline:** Represented by a dashed pink/magenta horizontal line. ### Detailed Analysis **Data Series and Exact Values:** 1. **Step-Level (Online) - Green Line with Stars:** * Trend: Shows a steady upward trend from 10% to 30% training data, peaks at 30%, then slightly declines and plateaus. * Data Points: * At 10%: Accuracy = 72.2 * At 20%: Accuracy = 74.7 * At 30%: Accuracy = 76.4 (Peak) * At 40%: Accuracy = 75.6 * At 50%: Accuracy = 75.8 2. **Instance-Level (Online) - Blue Line with Triangles:** * Trend: Shows a strong upward trend from 10% to 40%, then a noticeable drop at 50%. * Data Points: * At 10%: Accuracy = 66.5 * At 20%: Accuracy = 72.2 * At 30%: Accuracy = 73.3 * At 40%: Accuracy = 75.2 (Peak) * At 50%: Accuracy = 73.4 3. **Step-Level (Offline) - Yellow Line with Stars:** * Trend: Increases slightly from 10% to 20%, then shows a consistent downward trend as training data increases beyond 20%. * Data Points: * At 10%: Accuracy = 69.2 * At 20%: Accuracy = 70.8 (Peak) * At 30%: Accuracy = 67.3 * At 40%: Accuracy = 66.5 * (No data point is plotted for 50%). 4. **SFT Baseline - Dashed Pink Line:** * Trend: Constant horizontal line, indicating a fixed performance level independent of the training data percentage shown. * Value: Accuracy = 60.6 (labeled on the right side of the chart). ### Key Observations * **Performance Hierarchy:** The "Step-Level (Online)" method consistently achieves the highest accuracy across all training data percentages shown. * **Online vs. Offline:** Both online methods (Step-Level and Instance-Level) significantly outperform the offline method ("Step-Level (Offline)") and the SFT Baseline at all data points. * **Diminishing Returns/Overfitting:** The "Step-Level (Online)" method's performance plateaus after 30% data. The "Instance-Level (Online)" method's performance drops after 40% data, suggesting potential overfitting or diminishing returns with more data for this method. * **Offline Method Decline:** The "Step-Level (Offline)" method shows a clear negative correlation between training data percentage and accuracy beyond the 20% mark. * **Baseline Comparison:** All three experimental methods provide a substantial improvement over the "SFT Baseline" of 60.6 accuracy. ### Interpretation This chart demonstrates the comparative effectiveness of different training paradigms on the ARC-C benchmark. The data suggests that **online training methods (both Step-Level and Instance-Level) are superior to the offline method and the standard SFT baseline** for this task, yielding accuracy gains of approximately 10-16 percentage points. The **"Step-Level (Online)" approach appears to be the most robust and effective**, maintaining high performance even as training data increases. The peak performance for this method occurs with 30% of the training data, after which additional data provides minimal benefit. The **decline in performance for the "Instance-Level (Online)" method at 50% data and the consistent decline for the "Step-Level (Offline)" method** are critical anomalies. They indicate that simply adding more data is not universally beneficial and can be detrimental depending on the training strategy. This could point to issues like overfitting, noise in the additional data, or a mismatch between the training objective and the evaluation metric when data scales. In summary, the chart argues for the efficacy of online, step-level training for maximizing accuracy on ARC-C, while cautioning that the benefits of increased data are not automatic and are highly dependent on the specific training methodology employed.

## Line Chart: ARC-C Accuracy vs Training Data Percentage ### Overview The chart compares the accuracy of four different training approaches (Step-Level Online, Instance-Level Online, Step-Level Offline, and SFT Baseline) across varying percentages of training data (10-50%). Accuracy is measured on the y-axis (55-75), while training data percentage is on the x-axis (10-50%). ### Components/Axes - **X-axis**: Training Data % (10, 20, 30, 40, 50) - **Y-axis**: Accuracy (55-75, increments of 5) - **Legend**: Located at bottom-left, with four entries: - Green line with stars: Step-Level (Online) - Blue line with triangles: Instance-Level (Online) - Yellow line with stars: Step-Level (Offline) - Purple dashed line: SFT Baseline - **Title**: "ARC-C" at top-center ### Detailed Analysis 1. **Step-Level (Online)** (Green): - Starts at 72.2% accuracy at 10% training data - Peaks at 76.4% at 30% training data - Slight decline to 75.8% at 50% training data - Trend: Initial sharp increase followed by stabilization 2. **Instance-Level (Online)** (Blue): - Begins at 66.5% at 10% training data - Rises to 73.3% at 30% training data - Peaks at 75.2% at 40% training data - Drops to 73.4% at 50% training data - Trend: Gradual improvement then decline 3. **Step-Level (Offline)** (Yellow): - Starts at 69.2% at 10% training data - Peaks at 70.8% at 20% training data - Declines to 66.5% at 50% training data - Trend: Early peak followed by steady decline 4. **SFT Baseline** (Purple dashed): - Constant at 60.6% across all training data percentages - Trend: Flat line with no variation ### Key Observations - Step-Level (Online) consistently outperforms all other methods - Instance-Level (Online) shows the most significant improvement between 10% and 40% training data - Step-Level (Offline) underperforms compared to its online counterpart - SFT Baseline remains the lowest-performing method throughout - All methods show diminishing returns beyond 30-40% training data ### Interpretation The data demonstrates that online training approaches (both step-level and instance-level) significantly outperform offline methods and the SFT baseline. The Step-Level (Online) method achieves the highest accuracy (76.4%) at 30% training data, suggesting optimal performance at this threshold. The Instance-Level (Online) method shows the most dramatic improvement (from 66.5% to 75.2%) as training data increases, indicating strong scalability. The consistent decline in Step-Level (Offline) performance highlights the importance of online training dynamics. The flat SFT Baseline suggests it represents a fundamental lower bound for this particular task. The diminishing returns observed across all methods beyond 40% training data implies potential inefficiencies in utilizing larger datasets without proportional accuracy gains.