## Line Graphs: Lichess Puzzle Accuracy vs. Training Step for Qwen2.5-7B and Llama3.1-8B ### Overview The image presents two line graphs comparing the performance of two language models, Qwen2.5-7B and Llama3.1-8B, on a Lichess puzzle accuracy task. Each graph plots the accuracy against the training step, with two lines representing training with and without reasoning (SFT - Supervised Fine-Tuning). ### Components/Axes * **Titles:** * Left Graph: Qwen2.5-7B * Right Graph: Llama3.1-8B * **Y-axis (Lichess Puzzle Acc):** * Label: Lichess Puzzle Acc * Scale: 0.00 to 0.30, with increments of 0.05 (0.00, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30) * **X-axis (Training Step):** * Label: Training Step * Scale: 0 to 150, with increments of 30 (0, 30, 60, 90, 120, 150) * **Legend (located in the center of the two graphs):** * Blue line: w/ Reasoning SFT * Gray line: w/o Reasoning SFT ### Detailed Analysis **Left Graph: Qwen2.5-7B** * **Blue Line (w/ Reasoning SFT):** * Trend: Generally increasing, with a slight plateau after 90 training steps. * Data Points: * 0 Training Step: ~0.21 * 30 Training Step: ~0.22 * 60 Training Step: ~0.25 * 90 Training Step: ~0.28 * 120 Training Step: ~0.28 * 150 Training Step: ~0.29 * **Gray Line (w/o Reasoning SFT):** * Trend: Rapidly increasing initially, then plateaus after 90 training steps. * Data Points: * 0 Training Step: ~0.01 * 30 Training Step: ~0.07 * 60 Training Step: ~0.19 * 90 Training Step: ~0.28 * 120 Training Step: ~0.29 * 150 Training Step: ~0.29 **Right Graph: Llama3.1-8B** * **Blue Line (w/ Reasoning SFT):** * Trend: Increasing, then plateaus after 60 training steps. * Data Points: * 0 Training Step: ~0.21 * 30 Training Step: ~0.24 * 60 Training Step: ~0.26 * 90 Training Step: ~0.27 * 120 Training Step: ~0.27 * 150 Training Step: ~0.29 * **Gray Line (w/o Reasoning SFT):** * Trend: Rapid initial increase, followed by fluctuations and a plateau. * Data Points: * 0 Training Step: ~0.01 * 30 Training Step: ~0.31 * 60 Training Step: ~0.28 * 90 Training Step: ~0.31 * 120 Training Step: ~0.33 * 150 Training Step: ~0.33 ### Key Observations * For both models, training with reasoning (blue line) generally results in higher initial accuracy. * The "w/o Reasoning SFT" (gray line) for both models shows a steeper initial increase in accuracy compared to "w/ Reasoning SFT" (blue line). * Llama3.1-8B (right graph) shows more fluctuation in the "w/o Reasoning SFT" line compared to Qwen2.5-7B (left graph). * Both models reach similar accuracy levels at the end of the training steps, regardless of whether reasoning is included in the SFT. ### Interpretation The graphs suggest that while training with reasoning (SFT) provides a better starting point for both models, training without reasoning catches up as the training progresses. The fluctuations in Llama3.1-8B's "w/o Reasoning SFT" line might indicate instability or sensitivity to the training data. The similar final accuracy levels suggest that both models can achieve comparable performance on the Lichess puzzle task, regardless of the initial training approach. The data indicates that reasoning SFT is more important in the early stages of training.

\n ## Line Chart: Lichess Puzzle Accuracy vs. Training Step for Two Models ### Overview This image presents two line charts comparing the performance of two language models, Qwen2.5-7B and Llama3.1-8B, on the Lichess puzzle accuracy task during training. Each chart displays two lines representing models trained *with* reasoning Supervised Fine-Tuning (SFT) and *without* reasoning SFT. The x-axis represents the training step, and the y-axis represents the Lichess puzzle accuracy. ### Components/Axes * **X-axis (Both Charts):** "Training Step", ranging from 0 to 150. Gridlines are present at intervals of 30. * **Y-axis (Both Charts):** "Lichess Puzzle Acc", ranging from 0.00 to 0.30. Gridlines are present at intervals of 0.05. * **Chart 1 Title:** "Qwen2.5-7B" * **Chart 2 Title:** "Llama3.1-8B" * **Legend (Both Charts):** Located in the bottom-left corner. * Blue Line: "w/ Reasoning SFT" * Gray Line: "w/o Reasoning SFT" ### Detailed Analysis or Content Details **Qwen2.5-7B Chart:** * **w/ Reasoning SFT (Blue Line):** The line starts at approximately 0.02 at Training Step 0. It increases rapidly to around 0.22 at Training Step 30. The line continues to increase, leveling off around 0.28-0.29 between Training Steps 90 and 150. * **w/o Reasoning SFT (Gray Line):** The line starts at approximately 0.00 at Training Step 0. It increases more slowly than the blue line, reaching around 0.15 at Training Step 30. The line continues to increase, reaching approximately 0.27 at Training Step 90, and then fluctuates between 0.27 and 0.29 until Training Step 150. **Llama3.1-8B Chart:** * **w/ Reasoning SFT (Blue Line):** The line starts at approximately 0.02 at Training Step 0. It increases rapidly to around 0.24 at Training Step 30. The line plateaus around 0.26-0.27 between Training Steps 60 and 150, with some minor fluctuations. * **w/o Reasoning SFT (Gray Line):** The line starts at approximately 0.01 at Training Step 0. It increases rapidly to around 0.20 at Training Step 30. The line continues to increase, reaching approximately 0.31 at Training Step 60, then decreases to around 0.28 at Training Step 90, and fluctuates between 0.28 and 0.31 until Training Step 150. ### Key Observations * For both models, training *with* reasoning SFT consistently results in higher Lichess puzzle accuracy than training *without* reasoning SFT, especially in the early stages of training. * The Qwen2.5-7B model appears to reach a plateau in accuracy earlier than the Llama3.1-8B model. * The Llama3.1-8B model *without* reasoning SFT shows a significant peak in accuracy around Training Step 60, followed by a slight decrease and then stabilization. This is a notable outlier. * The difference in performance between the two SFT strategies diminishes as training progresses, suggesting that the benefits of reasoning SFT may decrease over time. ### Interpretation The data suggests that incorporating reasoning SFT significantly improves the ability of both Qwen2.5-7B and Llama3.1-8B to solve Lichess puzzles. This indicates that explicitly training the models to reason enhances their performance on this task. The plateau observed in Qwen2.5-7B suggests that the model may have reached its capacity for improvement on this specific task with the given training data and methodology. The peak and subsequent stabilization in Llama3.1-8B (without reasoning SFT) could be due to overfitting to a specific subset of puzzles or a temporary convergence during training. The diminishing difference between the two SFT strategies as training progresses could indicate that the benefits of reasoning SFT are more pronounced during the initial stages of learning, and that other factors become more important as the models become more proficient. Further investigation would be needed to understand the cause of the Llama3.1-8B outlier.

## Line Graphs: Lichess Puzzle Accuracy vs. Training Steps for Qwen2.5-7B and Llama3.1-8B ### Overview The image contains two side-by-side line graphs comparing the performance of two AI models (Qwen2.5-7B and Llama3.1-8B) during training. Each graph tracks "Lichess Puzzle Accuracy" (y-axis) against "Training Step" (x-axis, 0–150). Two data series are shown per model: - **Blue line**: Performance with Reasoning SFT (Supervised Fine-Tuning) - **Gray line**: Performance without Reasoning SFT ### Components/Axes - **X-axis (Training Step)**: - Range: 0 to 150 (increments of 30) - Labels: "Training Step" - **Y-axis (Lichess Puzzle Acc)**: - Range: 0.00 to 0.30 (increments of 0.05) - Labels: "Lichess Puzzle Acc" - **Legends**: - Positioned in the bottom-left corner of each graph. - Blue: "w/ Reasoning SFT" - Gray: "w/o Reasoning SFT" ### Detailed Analysis #### Qwen2.5-7B (Left Graph) - **Blue line (w/ Reasoning SFT)**: - Starts at ~0.20 (step 0) and increases steadily to ~0.29 (step 150). - Slope: Gradual upward trend with minimal fluctuations. - **Gray line (w/o Reasoning SFT)**: - Starts at 0.00 (step 0) and rises sharply to ~0.25 (step 60), then plateaus. - Slope: Steep initial increase, followed by a plateau. #### Llama3.1-8B (Right Graph) - **Blue line (w/ Reasoning SFT)**: - Starts at ~0.20 (step 0) and increases to ~0.28 (step 150). - Slope: Steady upward trend with minor fluctuations. - **Gray line (w/o Reasoning SFT)**: - Starts at 0.00 (step 0) and spikes to ~0.30 (step 30), then fluctuates between ~0.28–0.30. - Slope: Rapid initial rise, followed by volatility. ### Key Observations 1. **Performance Trends**: - Both models show improved performance with Reasoning SFT (blue lines outperform gray lines initially). - Qwen2.5-7B’s gray line converges with the blue line by step 150 (~0.29 vs. ~0.29). - Llama3.1-8B’s gray line surpasses the blue line (~0.30 vs. ~0.28) but exhibits instability. 2. **Model Differences**: - Llama3.1-8B achieves higher peak accuracy (0.30) but with greater variability. - Qwen2.5-7B demonstrates more stable convergence between SFT and non-SFT approaches. 3. **Anomalies**: - Llama3.1-8B’s gray line shows a sharp dip to ~0.28 at step 60, suggesting potential overfitting or instability. ### Interpretation The data suggests that **Reasoning SFT improves performance** for both models, but the impact varies: - **Qwen2.5-7B**: SFT provides a consistent boost, with non-SFT performance catching up over time. - **Llama3.1-8B**: SFT yields higher initial gains, but non-SFT performance eventually exceeds SFT, possibly due to overfitting or architectural differences. The graphs highlight the trade-off between stability (Qwen) and peak performance (Llama), with Llama’s volatility raising questions about the reliability of non-SFT training. Further investigation into training dynamics and model architecture could clarify these trends.