## Line Chart: Accuracy vs. Number of Reasoning Hops ### Overview The image is a line chart comparing the accuracy of three different models (Base Model, SFT Only, and SFT+RL) as the number of reasoning hops increases from 2 to 5. The chart highlights the generalization performance of the models on unseen complexity, with a specific focus on the improvement of the SFT+RL model. ### Components/Axes * **X-axis:** Number of Reasoning Hops (values: 2, 3, 4, 5) * **Y-axis:** Accuracy (%) (scale: 60 to 95, with increments of 5) * **Legend (bottom-right):** * Blue line with circle markers: Base Model * Magenta line with square markers: SFT Only * Orange line with diamond markers: SFT+RL * **Annotation:** "Generalization (unseen complexity)" is written above the data points at x=5, with an arrow indicating a +11.1% increase in accuracy for the SFT+RL model. * A shaded region spans the area where the number of reasoning hops is greater than 3. ### Detailed Analysis * **Base Model (Blue):** The accuracy starts at approximately 68% at 2 hops, decreases to approximately 64% at 3 hops, increases to approximately 68% at 4 hops, and reaches approximately 70% at 5 hops. * **SFT Only (Magenta):** The accuracy starts at approximately 77% at 2 hops, decreases to approximately 74% at 3 hops, increases to approximately 79% at 4 hops, and decreases slightly to approximately 78% at 5 hops. * **SFT+RL (Orange):** The accuracy starts at approximately 85% at 2 hops, decreases to approximately 81% at 3 hops, increases to approximately 87% at 4 hops, and increases significantly to approximately 89% at 5 hops. ### Key Observations * The SFT+RL model consistently outperforms the other two models across all numbers of reasoning hops. * The SFT+RL model shows the most significant improvement in accuracy when the number of reasoning hops increases from 4 to 5, as indicated by the "+11.1%" annotation. * The Base Model has the lowest accuracy across all reasoning hops. * All models experience a slight dip in accuracy when the number of reasoning hops increases from 2 to 3. ### Interpretation The data suggests that the SFT+RL model is better at generalizing to unseen complexity compared to the Base Model and SFT Only model. The significant increase in accuracy for the SFT+RL model when the number of reasoning hops is 5 indicates that it is particularly effective at handling more complex reasoning tasks. The shaded region and the "Generalization (unseen complexity)" annotation emphasize that the models' performance at 5 reasoning hops is indicative of their ability to handle unseen, more complex scenarios. The +11.1% improvement highlights the value of reinforcement learning (RL) in improving the model's generalization capabilities. The dip in accuracy from 2 to 3 hops for all models could indicate a threshold of complexity where initial reasoning steps are less effective, but this is overcome as the number of hops increases further, especially for the SFT+RL model.

\n ## Line Chart: Accuracy vs. Reasoning Hops ### Overview This line chart depicts the relationship between the number of reasoning hops and the accuracy of three different models: a Base Model, an SFT (Supervised Fine-Tuning) Only model, and an SFT+RL (Reinforcement Learning) model. The chart illustrates how accuracy changes as the complexity of reasoning increases, indicated by the number of reasoning hops. A shaded region in the upper-right corner highlights the "Generalization (unseen complexity)" and a corresponding accuracy increase of +11.1%. ### Components/Axes * **X-axis:** Number of Reasoning Hops (ranging from 2 to 5). * **Y-axis:** Accuracy (%) (ranging from 60% to 95%). * **Data Series:** * Base Model (Blue, dashed circle line) * SFT Only (Magenta, dashed square line) * SFT+RL (Orange, dashed diamond line) * **Legend:** Located in the bottom-right corner, clearly labeling each data series with its corresponding color and marker. * **Annotation:** "Generalization (unseen complexity)" with "+11.1%" positioned in the top-right corner, indicating an accuracy improvement. ### Detailed Analysis * **Base Model (Blue):** The line starts at approximately 69% accuracy at 2 reasoning hops, decreases to around 64% at 3 hops, and then gradually increases to approximately 70% at 5 hops. The trend is generally flat with a slight dip in the middle. * **SFT Only (Magenta):** The line begins at approximately 76% accuracy at 2 reasoning hops, decreases to around 74% at 3 hops, increases to approximately 80% at 4 hops, and then slightly decreases to around 79% at 5 hops. This line shows a more pronounced increase between 3 and 4 hops. * **SFT+RL (Orange):** The line starts at approximately 85% accuracy at 2 reasoning hops, decreases to around 81% at 3 hops, and then increases sharply to approximately 87% at 4 hops, and then slightly decreases to around 86% at 5 hops. This line consistently demonstrates the highest accuracy across all reasoning hops. **Specific Data Points (approximate):** | Reasoning Hops | Base Model (%) | SFT Only (%) | SFT+RL (%) | |---|---|---|---| | 2 | 69 | 76 | 85 | | 3 | 64 | 74 | 81 | | 4 | 68 | 80 | 87 | | 5 | 70 | 79 | 86 | ### Key Observations * The SFT+RL model consistently outperforms both the Base Model and the SFT Only model across all reasoning hops. * The Base Model exhibits the lowest accuracy and the most fluctuating performance. * All models show a dip in accuracy at 3 reasoning hops, potentially indicating a point of increased complexity. * The largest performance gain for the SFT+RL model occurs between 3 and 4 reasoning hops. * The annotation highlights a significant generalization improvement of +11.1% at 5 reasoning hops, specifically related to unseen complexity. ### Interpretation The data suggests that incorporating Reinforcement Learning (RL) into Supervised Fine-Tuning (SFT) significantly improves the model's ability to handle complex reasoning tasks. The SFT+RL model demonstrates a clear advantage in accuracy, particularly as the number of reasoning hops increases, indicating a better capacity for generalization to unseen complexities. The dip in accuracy at 3 reasoning hops for all models could represent a threshold where the reasoning process becomes more challenging, requiring more sophisticated learning techniques. The +11.1% generalization improvement at 5 hops further emphasizes the benefits of the SFT+RL approach for tackling complex, real-world problems. The Base Model's lower performance suggests that simply scaling up the model size or training data may not be sufficient to achieve high accuracy in complex reasoning scenarios; targeted fine-tuning and reinforcement learning are crucial.

## Line Chart: Model Accuracy vs. Reasoning Complexity ### Overview The image is a line chart comparing the performance of three different models—Base Model, SFT Only, and SFT+RL—on a task requiring increasing numbers of reasoning steps. The chart demonstrates how accuracy changes as the complexity of the reasoning task (measured in "hops") increases, with a specific focus on generalization to higher, unseen complexity levels. ### Components/Axes * **Chart Type:** Multi-line chart with markers. * **X-Axis:** Labeled **"Number of Reasoning Hops"**. It has discrete integer markers at 2, 3, 4, and 5. * **Y-Axis:** Labeled **"Accuracy (%)"**. The scale runs from 60 to 95, with major gridlines at intervals of 5% (60, 65, 70, 75, 80, 85, 90, 95). * **Legend:** Located in the bottom-right corner of the plot area. It defines three data series: * **Base Model:** Represented by a purple line with circular markers (●). * **SFT Only:** Represented by a pink/magenta line with square markers (■). * **SFT+RL:** Represented by an orange line with diamond markers (◆). * **Annotations:** * A shaded beige region spans the x-axis from 4 to 5 hops. Text within this region reads: **"Generalization (unseen complexity)"**. * A double-headed vertical arrow connects the final data points of the "SFT+RL" and "Base Model" series at x=5. A label next to it reads: **"+11.1%"**. ### Detailed Analysis **Data Series Trends and Approximate Values:** 1. **Base Model (Purple, ●):** * **Trend:** Shows a slight dip from 2 to 3 hops, followed by a gradual, steady increase from 3 to 5 hops. * **Data Points (Approximate):** * 2 Hops: ~68% * 3 Hops: ~64% * 4 Hops: ~68% * 5 Hops: ~70% 2. **SFT Only (Pink, ■):** * **Trend:** Follows a similar pattern to the Base Model—a dip from 2 to 3 hops, then a recovery and increase. It consistently performs better than the Base Model. * **Data Points (Approximate):** * 2 Hops: ~77% * 3 Hops: ~74% * 4 Hops: ~79% * 5 Hops: ~78% 3. **SFT+RL (Orange, ◆):** * **Trend:** Also exhibits a dip from 2 to 3 hops, but then shows the strongest upward trajectory from 3 to 5 hops. It is the top-performing model at every data point. * **Data Points (Approximate):** * 2 Hops: ~85% * 3 Hops: ~81% * 4 Hops: ~87% * 5 Hops: ~89% **Performance Gap:** The annotation "+11.1%" quantifies the performance advantage of the **SFT+RL** model over the **Base Model** at the highest complexity level (5 reasoning hops). The gap between the top (SFT+RL) and middle (SFT Only) lines also appears to widen slightly at 4 and 5 hops compared to 2 hops. ### Key Observations 1. **Universal Dip at 3 Hops:** All three models experience a noticeable drop in accuracy when moving from 2 to 3 reasoning hops, suggesting this specific increase in complexity poses a common challenge. 2. **Consistent Performance Hierarchy:** The ranking of the models (SFT+RL > SFT Only > Base Model) is maintained across all tested levels of reasoning complexity. 3. **Generalization Performance:** The shaded "Generalization" region highlights that the task at 4 and 5 hops was likely not seen during training. The chart shows that all models, especially SFT+RL, recover and improve accuracy in this region, indicating successful generalization. 4. **Widening Advantage:** The performance gap between the most advanced model (SFT+RL) and the baseline appears largest at the highest complexity (5 hops), suggesting the benefits of the combined SFT and RL approach scale with task difficulty. ### Interpretation This chart provides strong evidence for the effectiveness of a training pipeline that combines Supervised Fine-Tuning (SFT) with Reinforcement Learning (RL) for complex reasoning tasks. * **The Data Suggests:** While SFT alone provides a significant boost over the base model, adding RL on top of SFT yields further substantial gains, particularly as the reasoning chain grows longer and more complex. The dip at 3 hops might indicate a phase where the reasoning structure becomes non-trivial, challenging all models before they adapt to longer chains. * **Element Relationships:** The x-axis (complexity) directly challenges the models, whose performance is measured on the y-axis (accuracy). The legend defines the independent variable (training method), while the annotation explicitly calls out the most critical finding: superior generalization to higher, unseen complexity. * **Notable Anomaly/Trend:** The most significant trend is not just the higher accuracy of SFT+RL, but its steeper positive slope from 3 to 5 hops. This indicates it is not only better but also *more robust* to increasing complexity, which is the key to solving real-world problems requiring multi-step deduction. The +11.1% gap is a concrete measure of this robustness advantage at the limit of the tested data.

## Line Chart: Model Accuracy vs. Reasoning Hops ### Overview The chart compares the accuracy of three models (Base Model, SFT Only, SFT+RL) across 2 to 5 reasoning hops. A shaded region labeled "Generalization (unseen complexity)" spans hops 3–4, with an arrow highlighting an 11.1% accuracy increase for SFT+RL between hops 4 and 5. ### Components/Axes - **X-axis**: "Number of Reasoning Hops" (2, 3, 4, 5). - **Y-axis**: "Accuracy (%)" (60–95, 5% increments). - **Legend**: - Blue circle: Base Model - Pink square: SFT Only - Orange diamond: SFT+RL - **Shaded Region**: Horizontal band between hops 3–4 (light orange). - **Arrow**: Vertical arrow from hop 4 to 5 on the SFT+RL line, labeled "+11.1%". ### Detailed Analysis - **Base Model (Blue)**: - Hop 2: ~68% - Hop 3: ~64% - Hop 4: ~67% - Hop 5: ~70% - *Trend*: Slightly declining then recovering. - **SFT Only (Pink)**: - Hop 2: ~77% - Hop 3: ~74% - Hop 4: ~79% - Hop 5: ~78% - *Trend*: Initial drop, then slight recovery. - **SFT+RL (Orange)**: - Hop 2: ~85% - Hop 3: ~81% - Hop 4: ~87% - Hop 5: ~89% - *Trend*: Initial drop, then sharp recovery and peak. - **Shaded Region**: Highlights hops 3–4, possibly indicating a focus on generalization. - **Arrow**: Emphasizes a 11.1% accuracy gain for SFT+RL between hops 4 and 5. ### Key Observations 1. **SFT+RL dominates** in accuracy across all hops, especially after hop 4. 2. **Generalization focus**: The shaded region (hops 3–4) and arrow suggest SFT+RL excels in unseen complexity. 3. **Base Model underperforms** consistently, with minimal improvement. 4. **SFT Only** shows moderate performance but lags behind SFT+RL. ### Interpretation The data demonstrates that **SFT+RL** significantly outperforms other models, particularly in higher-complexity scenarios (hops 4–5). The 11.1% accuracy jump in SFT+RL between hops 4 and 5, highlighted by the arrow, suggests that reinforcement learning (RL) enhances generalization to unseen tasks. The shaded region (hops 3–4) may represent a critical zone for evaluating model robustness. While SFT Only improves slightly with more hops, its performance plateaus, indicating limitations in handling complexity. The Base Model’s decline at hop 3 and subsequent recovery hints at instability in reasoning processes. Overall, the chart underscores the value of combining SFT with RL for complex, real-world applications.