## Pie Charts: Synthetic Problems Difficulty for Different Models ### Overview The image contains four pie charts, each representing the distribution of difficulty levels for synthetic problems encountered by different models: SwS-3B, SwS-7B, SwS-7B-Math, and SwS-32B. The difficulty levels range from 0 to 8, and each slice of the pie chart indicates the percentage of problems falling into that difficulty category. A red line highlights the slices representing difficulty levels 2 through 7. ### Components/Axes Each pie chart has the following components: * **Title:** Specifies the model for which the difficulty distribution is shown (e.g., "Synthetic Problems Difficulty for SwS-3B"). * **Slices:** Each slice represents a difficulty level (0-8). * **Labels:** Each slice is labeled with the difficulty level (0-8) and the corresponding percentage of problems at that difficulty. * **Colors:** Each difficulty level is represented by a distinct color. The colors appear consistent across all four charts. ### Detailed Analysis **Chart 1: Synthetic Problems Difficulty for SwS-3B** * Difficulty 0 (Teal): 32.2% * Difficulty 1 (Coral): 11.0% * Difficulty 2 (Light Purple): 6.9% * Difficulty 3 (Pink): 5.6% * Difficulty 4 (Light Green): 5.0% * Difficulty 5 (Yellow): 4.9% * Difficulty 6 (Tan): 5.3% * Difficulty 7 (Gray): 7.1% * Difficulty 8 (Light Blue): 22.0% **Chart 2: Synthetic Problems Difficulty for SwS-7B** * Difficulty 0 (Teal): 23.3% * Difficulty 1 (Coral): 9.9% * Difficulty 2 (Light Purple): 7.1% * Difficulty 3 (Pink): 6.0% * Difficulty 4 (Light Green): 5.6% * Difficulty 5 (Yellow): 5.6% * Difficulty 6 (Tan): 6.1% * Difficulty 7 (Gray): 7.9% * Difficulty 8 (Light Blue): 28.6% **Chart 3: Synthetic Problems Difficulty for SwS-7B-Math** * Difficulty 0 (Teal): 30.2% * Difficulty 1 (Coral): 9.7% * Difficulty 2 (Light Purple): 6.4% * Difficulty 3 (Pink): 5.3% * Difficulty 4 (Light Green): 4.9% * Difficulty 5 (Yellow): 4.8% * Difficulty 6 (Tan): 5.3% * Difficulty 7 (Gray): 7.7% * Difficulty 8 (Light Blue): 25.7% **Chart 4: Synthetic Problems Difficulty for SwS-32B** * Difficulty 0 (Teal): 18.8% * Difficulty 1 (Coral): 9.2% * Difficulty 2 (Light Purple): 6.9% * Difficulty 3 (Pink): 5.8% * Difficulty 4 (Light Green): 5.5% * Difficulty 5 (Yellow): 5.5% * Difficulty 6 (Tan): 6.2% * Difficulty 7 (Gray): 8.2% * Difficulty 8 (Light Blue): 33.8% ### Key Observations * Difficulty levels 0 and 8 consistently represent the largest percentages across all models. * The distribution of difficulty levels 2 through 7 is relatively similar across all models, with each level accounting for a small percentage of the total. * SwS-32B has the highest percentage of problems at difficulty level 8 (33.8%) and the lowest percentage at difficulty level 0 (18.8%) compared to the other models. ### Interpretation The pie charts provide a visual representation of the difficulty distribution of synthetic problems for different models. The data suggests that the models encounter a significant portion of problems at the easiest (0) and hardest (8) difficulty levels. The red line highlights the distribution of the mid-range difficulty levels (2-7), which appear to be relatively consistent across all models. The SwS-32B model seems to encounter a higher proportion of the most difficult problems compared to the other models, potentially indicating a different learning or problem-solving approach. The data could be used to compare the performance and learning characteristics of the different models on synthetic problems.

## Pie Charts: Synthetic Problems Difficulty for SwS Variants ### Overview This image contains four pie charts, each representing the distribution of "Synthetic Problems Difficulty" for different "SwS" variants. The variants are SwS-3B, SwS-7B, SwS-7B-Math, and SwS-32B. Each slice of the pie chart represents a difficulty level, indicated by a number (0 through 8), and its corresponding percentage of the total problems for that variant. ### Components/Axes Each pie chart has the following characteristics: - **Title**: Each chart is titled "Synthetic Problems Difficulty for [SwS Variant Name]". - **Slices**: Each pie chart is divided into multiple slices, representing different difficulty levels. - **Labels**: Each slice is labeled with a number representing the difficulty level and a percentage in parentheses, indicating its proportion of the total. The difficulty levels range from 0 to 8. - **Colors**: Distinct colors are used for each difficulty level across all charts, though the specific mapping of color to difficulty level might vary slightly in hue between charts. A red outline is present around some of the smaller slices in each pie chart. ### Detailed Analysis **1. Synthetic Problems Difficulty for SwS-3B** | Difficulty Level | Percentage | Color | | :--------------- | :--------- | :---------- | | 0 | 32.2% | Teal | | 1 | 11.0% | Orange | | 2 | 6.9% | Blue-grey | | 3 | 5.6% | Purple | | 4 | 5.0% | Yellow | | 5 | 4.9% | Tan | | 6 | 5.3% | Light Brown | | 7 | 7.1% | Grey | | 8 | 22.0% | Light Blue | **2. Synthetic Problems Difficulty for SwS-7B** | Difficulty Level | Percentage | Color | | :--------------- | :--------- | :---------- | | 0 | 23.3% | Teal | | 1 | 9.9% | Orange | | 2 | 7.1% | Blue-grey | | 3 | 6.0% | Purple | | 4 | 5.6% | Yellow | | 5 | 5.6% | Tan | | 6 | 6.1% | Light Brown | | 7 | 7.9% | Grey | | 8 | 28.6% | Light Blue | **3. Synthetic Problems Difficulty for SwS-7B-Math** | Difficulty Level | Percentage | Color | | :--------------- | :--------- | :---------- | | 0 | 30.2% | Teal | | 1 | 9.7% | Orange | | 2 | 6.4% | Blue-grey | | 3 | 5.3% | Purple | | 4 | 4.9% | Yellow | | 5 | 4.8% | Tan | | 6 | 5.3% | Light Brown | | 7 | 7.7% | Grey | | 8 | 25.7% | Light Blue | **4. Synthetic Problems Difficulty for SwS-32B** | Difficulty Level | Percentage | Color | | :--------------- | :--------- | :---------- | | 0 | 18.8% | Teal | | 1 | 9.2% | Orange | | 2 | 6.9% | Blue-grey | | 3 | 5.8% | Purple | | 4 | 5.5% | Yellow | | 5 | 5.5% | Tan | | 6 | 6.2% | Light Brown | | 7 | 8.2% | Grey | | 8 | 33.8% | Light Blue | ### Key Observations - **Dominant Difficulties**: Across all variants, difficulty level 0 and difficulty level 8 consistently represent the largest proportions of problems. Difficulty 0 is generally the largest slice, followed closely by difficulty 8. - **SwS-3B**: Has the highest percentage for difficulty 0 (32.2%) and a significant portion for difficulty 8 (22.0%). - **SwS-7B**: Shows the highest percentage for difficulty 8 (28.6%) and a moderate percentage for difficulty 0 (23.3%). - **SwS-7B-Math**: Similar to SwS-3B, with a high percentage for difficulty 0 (30.2%) and a substantial portion for difficulty 8 (25.7%). - **SwS-32B**: Exhibits the highest percentage for difficulty 8 (33.8%) and the lowest percentage for difficulty 0 (18.8%) among the four charts. - **Mid-range Difficulties**: Difficulties 1 through 7 generally represent smaller, more evenly distributed percentages, typically ranging from approximately 4.8% to 11.0%. - **Consistency**: The relative ordering and approximate percentages of difficulties 1 through 7 appear relatively consistent across the different SwS variants, with some minor variations. ### Interpretation The data suggests that for all analyzed SwS variants, synthetic problems are predominantly concentrated at the two extremes of the difficulty scale: very easy (difficulty 0) and very hard (difficulty 8). This pattern implies a potential bimodal distribution of problem difficulty, with fewer problems existing in the intermediate difficulty ranges. The variations between the SwS variants highlight differences in their problem generation or selection strategies. For instance, SwS-32B appears to generate a higher proportion of very difficult problems (difficulty 8) compared to the other variants, while SwS-3B and SwS-7B-Math lean more towards generating very easy problems (difficulty 0). The "Math" variant of SwS-7B shows a distribution very similar to the base SwS-3B, suggesting that the mathematical focus might not drastically alter the overall difficulty distribution compared to a non-math specific variant. This bimodal distribution could indicate that the problem generation systems are optimized to create either trivial or challenging tasks, potentially for specific testing or training purposes. The red outlines around some of the smaller slices might be a visual cue to draw attention to these less frequent difficulty levels, or they could represent a specific category of problems within the dataset. Further investigation into the nature of "difficulty 0" and "difficulty 8" problems, as well as the context of these SwS variants, would be necessary to fully understand the implications of this observed distribution.

\n ## Chart: Synthetic Problems Difficulty for SwS Models ### Overview The image presents four pie charts, each representing the distribution of difficulty levels for synthetic problems solved by different SwS (likely Solver-based Systems) models: SwS-3B, SwS-7B, SwS-7B-Math, and SwS-32B. Each pie chart shows the percentage of problems falling into difficulty levels ranging from 0 to 8. ### Components/Axes Each chart has a title indicating the SwS model being analyzed. The pie slices are labeled with the difficulty level (0-8) and the corresponding percentage of problems at that level. There are no explicit axes, as it's a pie chart representation. ### Detailed Analysis or Content Details **1. SwS-3B:** * Difficulty 0: 32.2% (Dark Blue) * Difficulty 1: 11.0% (Light Blue) * Difficulty 2: 6.9% (Yellow) * Difficulty 3: 5.6% (Orange) * Difficulty 4: 5.0% (Red) * Difficulty 5: 4.9% (Pink) * Difficulty 6: 5.3% (Purple) * Difficulty 7: 7.1% (Brown) * Difficulty 8: 22.0% (Green) **2. SwS-7B:** * Difficulty 0: 23.3% (Dark Blue) * Difficulty 1: 9.9% (Light Blue) * Difficulty 2: 7.1% (Yellow) * Difficulty 3: 6.0% (Orange) * Difficulty 4: 5.6% (Red) * Difficulty 5: 6.1% (Pink) * Difficulty 6: 6.1% (Purple) * Difficulty 7: 7.9% (Brown) * Difficulty 8: 28.6% (Green) **3. SwS-7B-Math:** * Difficulty 0: 30.2% (Dark Blue) * Difficulty 1: 9.7% (Light Blue) * Difficulty 2: 6.4% (Yellow) * Difficulty 3: 5.3% (Orange) * Difficulty 4: 4.9% (Red) * Difficulty 5: 4.8% (Pink) * Difficulty 6: 5.3% (Purple) * Difficulty 7: 7.7% (Brown) * Difficulty 8: 25.7% (Green) **4. SwS-32B:** * Difficulty 0: 18.8% (Dark Blue) * Difficulty 1: 9.2% (Light Blue) * Difficulty 2: 6.9% (Yellow) * Difficulty 3: 5.8% (Orange) * Difficulty 4: 5.5% (Red) * Difficulty 5: 5.4% (Pink) * Difficulty 6: 6.2% (Purple) * Difficulty 7: 8.2% (Brown) * Difficulty 8: 33.8% (Green) ### Key Observations * Difficulty level 0 consistently represents the largest proportion of problems across all models, ranging from 18.8% (SwS-32B) to 32.2% (SwS-3B). * Difficulty level 8 is the second most frequent for SwS-3B, SwS-7B, and SwS-32B, while for SwS-7B-Math it is also the second most frequent. * The distribution of difficulty levels appears relatively similar across all four models, with a concentration in the lower difficulty levels (0-4). * SwS-32B has the highest percentage of problems at difficulty level 8 (33.8%), suggesting it may be able to handle more complex problems compared to the other models. ### Interpretation The data suggests that the synthetic problems used to evaluate these SwS models are generally easier, as the majority fall into the lower difficulty levels. The slight variations in distribution across the models might indicate differences in their capabilities, with SwS-32B showing a greater ability to tackle more challenging problems (as evidenced by the higher percentage at difficulty level 8). The relatively consistent distribution across the models suggests that the problem generation process is similar for all, and the models are responding in a comparable manner. The fact that no model excels significantly in solving high-difficulty problems could indicate a limitation in the problem generation process or the models' inherent capabilities. Further investigation would be needed to determine the specific types of problems that fall into the higher difficulty levels and whether the models struggle with specific problem characteristics.

## Pie Charts: Synthetic Problems Difficulty Distribution for Four Models ### Overview The image displays four pie charts arranged in a 2x2 grid. Each chart visualizes the distribution of synthetic problem difficulty levels (labeled 0 through 8) for a different AI model variant. The charts share a consistent color scheme for the difficulty levels. The overall title for each chart is "Synthetic Problems Difficulty for [Model Name]". ### Components/Axes * **Chart Titles (Top of each pie):** * Top-Left: `Synthetic Problems Difficulty for SwS-3B` * Top-Right: `Synthetic Problems Difficulty for SwS-7B` * Bottom-Left: `Synthetic Problems Difficulty for SwS-7B-Math` * Bottom-Right: `Synthetic Problems Difficulty for SwS-32B` * **Data Series (Difficulty Levels):** Each pie is segmented into 9 categories, labeled 0 to 8. The color mapping is consistent across all charts: * Level 0: Teal/Green * Level 1: Orange * Level 2: Light Purple/Blue * Level 3: Pink * Level 4: Light Green * Level 5: Yellow * Level 6: Tan/Beige * Level 7: Grey * Level 8: Light Blue * **Labels:** Each segment is directly labeled with its difficulty level number and its percentage of the total, formatted as `[Level] ([Percentage]%)`. ### Detailed Analysis **1. SwS-3B (Top-Left Chart)** * **Trend:** The distribution is heavily skewed towards the lowest (0) and highest (8) difficulty levels, forming a bimodal shape. * **Data Points (Clockwise from top):** | Level | Percentage | | :---- | :--------- | | 0 | 32.2% | | 1 | 11.0% | | 2 | 6.9% | | 3 | 5.6% | | 4 | 5.0% | | 5 | 4.9% | | 6 | 5.3% | | 7 | 7.1% | | 8 | 22.0% | **2. SwS-7B (Top-Right Chart)** * **Trend:** The distribution remains bimodal but becomes more balanced between the extremes compared to SwS-3B. The proportion of the highest difficulty (8) increases notably. * **Data Points (Clockwise from top):** | Level | Percentage | | :---- | :--------- | | 0 | 23.3% | | 1 | 9.9% | | 2 | 7.1% | | 3 | 6.0% | | 4 | 5.6% | | 5 | 5.6% | | 6 | 6.1% | | 7 | 7.9% | | 8 | 28.6% | **3. SwS-7B-Math (Bottom-Left Chart)** * **Trend:** This distribution closely mirrors that of SwS-3B, with a strong peak at Level 0 and a secondary peak at Level 8. The middle-difficulty segments (1-7) are relatively small and uniform. * **Data Points (Clockwise from top):** | Level | Percentage | | :---- | :--------- | | 0 | 30.2% | | 1 | 9.7% | | 2 | 6.4% | | 3 | 5.3% | | 4 | 4.9% | | 5 | 4.8% | | 6 | 5.3% | | 7 | 7.7% | | 8 | 25.7% | **4. SwS-32B (Bottom-Right Chart)** * **Trend:** This chart shows the most significant shift. The proportion of the highest difficulty level (8) becomes dominant, while the lowest difficulty (0) shrinks considerably. The distribution is less bimodal and more skewed toward higher difficulties. * **Data Points (Clockwise from top):** | Level | Percentage | | :---- | :--------- | | 0 | 18.8% | | 1 | 9.2% | | 2 | 6.9% | | 3 | 5.8% | | 4 | 5.5% | | 5 | 5.5% | | 6 | 6.2% | | 7 | 8.2% | | 8 | 33.8% | ### Key Observations 1. **Bimodal Dominance:** All four models exhibit a bimodal distribution, with the vast majority of problems falling into either the easiest (Level 0) or hardest (Level 8) categories. The middle difficulties (1-7) consistently represent a much smaller portion of the total. 2. **Model Size Correlation:** As the model size increases from SwS-3B to SwS-32B, there is a clear trend: the percentage of the hardest problems (Level 8) increases (22.0% -> 28.6% -> 25.7% -> 33.8%), while the percentage of the easiest problems (Level 0) generally decreases (32.2% -> 23.3% -> 30.2% -> 18.8%). 3. **SwS-7B-Math Anomaly:** The SwS-7B-Math model's distribution is more similar to the smaller SwS-3B than to its similarly-sized SwS-7B counterpart. This suggests the math-focused fine-tuning may have altered the problem difficulty profile. 4. **SwS-32B Shift:** The largest model (SwS-32B) shows the most pronounced shift toward higher difficulty, with Level 8 comprising over a third of all problems. ### Interpretation The data suggests a strong relationship between model scale/capability and the difficulty of synthetic problems it is evaluated on or generates. The consistent bimodal pattern indicates that the synthetic problem generation process tends to create tasks that are either very straightforward or highly challenging, with fewer intermediate problems. The key insight is that **larger or more specialized models (like SwS-32B and SwS-7B) are associated with a higher proportion of difficult (Level 8) problems.** This could imply one of two things: 1. **Evaluation Bias:** The benchmark or generation method is designed to scale problem difficulty with model size, ensuring larger models are tested on harder tasks. 2. **Model Capability:** Larger models are either capable of solving harder problems (thus being evaluated on them) or are generating more complex synthetic problems themselves. The outlier, SwS-7B-Math, demonstrates that domain-specific tuning (mathematics) can significantly alter the difficulty profile, making it more similar to a smaller base model in this specific metric. This highlights that model size is not the sole determinant of the problem difficulty distribution; training objective plays a crucial role. In summary, the charts reveal a deliberate or emergent scaling of problem difficulty with model size, characterized by a persistent focus on the extremes of the difficulty spectrum.

## Pie Charts: Synthetic Problems Difficulty for SwS-3B, SwS-7B, SwS-7B-Math, SwS-32B ### Overview The image contains four pie charts comparing the distribution of synthetic problem difficulty levels (0–8) across four SwS versions: SwS-3B, SwS-7B, SwS-7B-Math, and SwS-32B. Each chart uses distinct colors for difficulty levels, with percentages labeled for each segment. ### Components/Axes - **Legend**: Integrated into each chart, with colors mapped to difficulty levels (0–8). Colors are consistent across charts for the same difficulty level (e.g., teal for 0, blue for 8). - **Labels**: Difficulty levels (0–8) and percentages are placed clockwise around each pie chart. - **Colors**: - 0: Teal (#008080) - 1: Red (#FF0000) - 2: Light Blue (#ADD8E6) - 3: Pink (#FFC0CB) - 4: Green (#00FF00) - 5: Yellow (#FFFF00) - 6: Orange (#FFA500) - 7: Gray (#808080) - 8: Light Blue (#ADD8E6) ### Detailed Analysis #### SwS-3B - **0 (32.2%)**: Largest segment (teal). - **8 (22.0%)**: Second-largest (light blue). - **1 (11.0%)**: Third-largest (red). - Remaining segments (2–7) range from 2.9% to 7.1%. #### SwS-7B - **0 (23.3%)**: Largest segment (teal). - **8 (28.6%)**: Second-largest (light blue). - **1 (9.9%)**: Third-largest (red). - Remaining segments (2–7) range from 5.0% to 7.9%. #### SwS-7B-Math - **0 (30.2%)**: Largest segment (teal). - **8 (25.7%)**: Second-largest (light blue). - **1 (9.7%)**: Third-largest (red). - Remaining segments (2–7) range from 4.8% to 7.7%. #### SwS-32B - **8 (33.8%)**: Largest segment (light blue). - **0 (18.8%)**: Second-largest (teal). - **1 (9.2%)**: Third-largest (red). - Remaining segments (2–7) range from 5.5% to 8.2%. ### Key Observations 1. **Difficulty 0 Dominance**: SwS-3B and SwS-7B-Math have the highest percentages for difficulty 0 (32.2% and 30.2%, respectively), while SwS-32B has the lowest (18.8%). 2. **Difficulty 8 Trends**: SwS-32B has the highest difficulty 8 percentage (33.8%), significantly higher than SwS-7B (28.6%) and SwS-7B-Math (25.7%). 3. **Middle Difficulties**: SwS-7B and SwS-32B show more balanced distributions for difficulties 4–6 compared to SwS-3B and SwS-7B-Math. 4. **Consistency in Color Mapping**: All charts use identical color coding for the same difficulty levels, ensuring cross-chart comparability. ### Interpretation The data suggests that SwS-32B excels at handling harder problems (difficulty 8), while SwS-3B and SwS-7B-Math are more effective at easier problems (difficulty 0). The middle difficulty levels (4–6) show less variation across models, indicating similar performance in average-difficulty problem-solving. The stark contrast in difficulty 8 performance between SwS-32B and other models highlights potential architectural or training differences optimized for complex tasks. The consistent color mapping across charts enables straightforward visual comparison of difficulty distributions.