## Scatter Plot: SWE-Bench Verified vs. Activated Model Size ### Overview The image is a scatter plot comparing the performance of various language models on the SWE-Bench verification task against their activated model size. The x-axis represents the activated model size in billions of parameters (B), and the y-axis represents the SWE-Bench verified percentage. Each blue dot represents a different model, with its name and performance percentage labeled next to it. A red star highlights the "Scale-SWE" model. ### Components/Axes * **X-axis:** Activated Model Size (B). The scale is non-linear, with markers at 3, 7, 32, and 70. * **Y-axis:** SWE-Bench Verified (%). The scale ranges from 10 to 70, with markers at intervals of 10. * **Data Points:** Each data point is a blue circle, labeled with the model name and its corresponding SWE-Bench verified percentage. * **Highlighted Point:** The "Scale-SWE" model is marked with a red star. ### Detailed Analysis Here's a breakdown of the data points, their approximate positions, and the trend they exhibit: * **Scale-SWE:** (Red Star) Located at approximately (3, 64%). SWE-Bench Verified is 64%. * **Qwen3-30A3B-Instruct:** Located at approximately (3, 22%). SWE-Bench Verified is 22%. * **Qwen3-Coder-30A3B:** Located at approximately (3, 52%). SWE-Bench Verified is 52.2%. * **GLM-4.7-Flash:** Located at approximately (3, 59%). SWE-Bench Verified is 59.2%. * **SWE-Mirror-7B:** Located at approximately (7, 23%). SWE-Bench Verified is 22.8%. * **SWE-Lego-8B:** Located at approximately (7, 42%). SWE-Bench Verified is 42.2%. * **SWE-Gym:** Located at approximately (32, 21%). SWE-Bench Verified is 20.6%. * **R2E-Gym:** Located at approximately (32, 34%). SWE-Bench Verified is 34.4%. * **SWE-smith:** Located at approximately (32, 40%). SWE-Bench Verified is 40.2%. * **Llama3-SWE-RL:** Located at approximately (70, 41%). SWE-Bench Verified is 41%. * **SWE-Fixer:** Located at approximately (70, 33%). SWE-Bench Verified is 32.8%. * **KAT-Dev-32B:** Located at approximately (32, 62%). SWE-Bench Verified is 62.4%. * **SWE-Lego-32B:** Located at approximately (32, 53%). SWE-Bench Verified is 52.6%. * **SWE-Mirror-32B:** Located at approximately (32, 52%). SWE-Bench Verified is 52.2%. ### Key Observations * There is a general trend of increasing SWE-Bench verified percentage as the activated model size increases, but this trend is not strictly linear. * There is a cluster of models around the 32B activated model size, with varying SWE-Bench verified percentages. * The "Scale-SWE" model has the highest SWE-Bench verified percentage (64%) and is highlighted with a red star. * The models with the smallest activated model size (3B) have the lowest SWE-Bench verified percentages. ### Interpretation The scatter plot suggests that, in general, larger language models tend to perform better on the SWE-Bench verification task. However, the performance is not solely determined by model size, as models with similar sizes can have significantly different performance. This indicates that other factors, such as model architecture, training data, and training methods, also play a crucial role in determining the performance on this benchmark. The "Scale-SWE" model stands out as having the best performance, suggesting it may have a particularly effective architecture or training regime for this task. The clustering of models around 32B suggests this may be a common or practical model size.

## Scatter Plot: SWE-Bench Verified vs. Activated Model Size ### Overview This scatter plot visualizes the relationship between the Activated Model Size (in Billions - B) and the percentage of SWE-Bench Verified for various models. The plot displays data points representing different models, with their corresponding sizes and verification percentages. A red star highlights a model named "Scale-SWE" with a notably high verification percentage. ### Components/Axes * **X-axis:** Activated Model Size (B) - Ranges from approximately 3 to 70. * **Y-axis:** SWE-Bench Verified (%) - Ranges from approximately 10% to 70%. * **Data Points:** Blue circles representing individual models. * **Highlight:** A red star representing the "Scale-SWE" model. * **Labels:** Each data point is labeled with the model name and its SWE-Bench Verified percentage. ### Detailed Analysis The data points are scattered across the plot, showing a general trend of increasing SWE-Bench Verified percentage with increasing Activated Model Size, but with significant variance. Here's a breakdown of the data points, reading from left to right (approximately): * **Qwen3-30A3B-Instruct:** Activated Model Size ≈ 3B, SWE-Bench Verified ≈ 22%. * **SWE-Mirror-7B:** Activated Model Size ≈ 7B, SWE-Bench Verified ≈ 22.8%. * **GLM-4.7-Flash:** Activated Model Size ≈ 7B, SWE-Bench Verified ≈ 59.2%. * **Qwen3-Coder-30A3B:** Activated Model Size ≈ 7B, SWE-Bench Verified ≈ 52.2%. * **SWE-Lego-8B:** Activated Model Size ≈ 7B, SWE-Bench Verified ≈ 42.2%. * **SWE-Gym:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 20.6%. * **R2E-Gym:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 34.4%. * **SWE-smith:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 40.2%. * **Llama3-SWE-RL:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 41%. * **SWE-Fixer:** Activated Model Size ≈ 70B, SWE-Bench Verified ≈ 32.8%. * **KAT-Dev-32B:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 62.4%. * **SWE-Lego-32B:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 52.6%. * **SWE-Mirror-32B:** Activated Model Size ≈ 32B, SWE-Bench Verified ≈ 52.2%. * **Scale-SWE:** Activated Model Size ≈ 64B, SWE-Bench Verified ≈ 64%. (Highlighted with a red star) ### Key Observations * **Scale-SWE** significantly outperforms all other models, achieving the highest SWE-Bench Verified percentage (64%) with a model size of 64B. * There's a wide range of SWE-Bench Verified percentages for models with similar Activated Model Sizes (e.g., around 32B). * The relationship between model size and verification percentage isn't strictly linear; larger models don't always guarantee higher verification. * Models with smaller sizes (around 3-7B) generally have lower verification percentages. ### Interpretation The data suggests that while increasing the Activated Model Size generally correlates with improved SWE-Bench verification, it's not the sole determinant of performance. The "Scale-SWE" model demonstrates that specific architectural choices or training methodologies can lead to substantial gains in verification accuracy, even at comparable model sizes. The variance in verification percentages for models of similar sizes indicates that factors beyond size, such as training data, model architecture, and optimization techniques, play a crucial role. The plot highlights the importance of considering multiple factors when evaluating and comparing language models. The red star draws attention to a model that is an outlier in terms of performance, suggesting it may be a particularly effective or well-optimized model. The data could be used to inform decisions about model selection and resource allocation, guiding developers towards models that offer the best balance between size and performance.

## Scatter Plot: AI Model Performance on SWE-Bench Verified vs. Activated Model Size ### Overview This image is a scatter plot comparing the performance of various AI models on the "SWE-Bench Verified" benchmark against their "Activated Model Size" in billions of parameters (B). The chart highlights the performance of a model named "Scale-SWE" with a prominent red star, indicating it as a key result or the subject of the analysis. ### Components/Axes * **X-Axis:** Labeled "Activated Model Size (B)". It has major tick marks at values 3, 7, 32, and 70. The axis represents the scale of the model in billions of parameters. * **Y-Axis:** Labeled "SWE-Bench Verified (%)". It ranges from 10 to 70 with increments of 10. This axis represents the accuracy or success rate percentage on the benchmark. * **Data Points:** Each model is represented by a blue circular dot, except for "Scale-SWE" which is marked with a large red star. Each data point is accompanied by a text label stating the model name and its exact performance percentage in parentheses. * **Legend:** There is no separate legend box. The identification of each data point is provided by its adjacent text label. ### Detailed Analysis The plot contains 14 distinct data points. Below is a list of all models, their approximate activated size (based on x-axis position), and their exact SWE-Bench Verified score: 1. **Scale-SWE (64%)**: Marked with a red star. Positioned at approximately 3B on the x-axis and 64% on the y-axis. 2. **GLM-4.7-Flash (59.2%)**: Blue dot. Positioned at ~3B, 59.2%. 3. **Qwen3-Coder-30A3B (52.2%)**: Blue dot. Positioned at ~3B, 52.2%. 4. **Qwen3-30A3B-Instruct (22%)**: Blue dot. Positioned at ~3B, 22%. 5. **SWE-Mirror-7B (22.8%)**: Blue dot. Positioned at ~7B, 22.8%. 6. **SWE-Lego-8B (42.2%)**: Blue dot. Positioned at ~8B (slightly right of 7B), 42.2%. 7. **SWE-smith (40.2%)**: Blue dot. Positioned at ~32B, 40.2%. 8. **R2E-Gym (34.4%)**: Blue dot. Positioned at ~32B, 34.4%. 9. **SWE-Gym (20.6%)**: Blue dot. Positioned at ~32B, 20.6%. 10. **KAT-Dev-32B (62.4%)**: Blue dot. Positioned at ~32B, 62.4%. 11. **SWE-Lego-32B (52.6%)**: Blue dot. Positioned at ~32B, 52.6%. 12. **SWE-Mirror-32B (52.2%)**: Blue dot. Positioned at ~32B, 52.2%. 13. **Llama3-SWE-RL (41%)**: Blue dot. Positioned at ~70B, 41%. 14. **SWE-Fixer (32.8%)**: Blue dot. Positioned at ~70B, 32.8%. ### Key Observations * **Performance vs. Size Trend:** There is a general, but not strict, positive correlation between activated model size and benchmark performance. The highest-performing models (Scale-SWE, KAT-Dev-32B) are at the top of the chart. * **Significant Outlier:** **Scale-SWE** is a major outlier. It achieves the highest score (64%) while having one of the smallest activated model sizes (~3B). This suggests exceptional efficiency or a different architectural approach. * **Clustering at 32B:** A dense cluster of models exists around the 32B size mark, with performance varying widely from 20.6% (SWE-Gym) to 62.4% (KAT-Dev-32B). * **Diminishing Returns at 70B:** The two models shown at 70B (Llama3-SWE-RL, SWE-Fixer) do not outperform the top models at 32B or 3B, indicating that simply increasing size does not guarantee better performance on this specific benchmark. * **Performance Tiers:** Models can be loosely grouped into tiers: * **Top Tier (>60%):** Scale-SWE, KAT-Dev-32B. * **High Tier (50-60%):** GLM-4.7-Flash, SWE-Lego-32B, SWE-Mirror-32B, Qwen3-Coder-30A3B. * **Mid Tier (40-50%):** SWE-Lego-8B, SWE-smith, Llama3-SWE-RL. * **Lower Tier (<40%):** R2E-Gym, SWE-Fixer, SWE-Mirror-7B, Qwen3-30A3B-Instruct, SWE-Gym. ### Interpretation This chart is likely from a technical report or paper introducing the "Scale-SWE" model. The primary message is that Scale-SWE achieves state-of-the-art performance (64%) on the SWE-Bench Verified benchmark while using a significantly smaller activated model size (~3B parameters) compared to other top performers like KAT-Dev-32B (32B). This challenges the common assumption that larger models always perform better and highlights the importance of model architecture, training data, or fine-tuning techniques (implied by names like "SWE-RL", "SWE-Lego", "SWE-Mirror"). The wide performance spread among models of similar size (especially at 32B) suggests that the SWE-Bench benchmark is highly sensitive to model specialization and training methodology for software engineering tasks, not just raw scale. The plot serves as a compelling visual argument for the efficiency and effectiveness of the Scale-SWE approach.

## Scatter Plot: SWE-Bench Verified (%) vs. Activated Model Size (B) ### Overview The image is a scatter plot comparing the percentage of SWE-Bench verified results against the activated model size (in billions of parameters). Data points are labeled with model names and their corresponding verification percentages. A red star marks "Scale-SWE(64%)" as the highest-performing model, while blue dots represent other models with varying sizes and verification rates. ### Components/Axes - **X-axis**: "Activated Model Size (B)" ranging from 3 to 70 (in billions of parameters). - **Y-axis**: "SWE-Bench Verified (%)" ranging from 10% to 70%. - **Legend**: - Red star: **Scale-SWE(64%)** (top-left corner). - Blue dots: Other models (e.g., GLM-4.7-Flash, Qwen3-Coder-30A3B, etc.). ### Detailed Analysis - **Model Sizes and Verification Rates**: - **Scale-SWE(64%)**: Red star at (3B, 64%). - **KAT-Dev-32B(62.4%)**: Blue dot at (32B, 62.4%). - **SWE-Lego-32B(52.6%)**: Blue dot at (32B, 52.6%). - **SWE-Mirror-32B(52.2%)**: Blue dot at (32B, 52.2%). - **GLM-4.7-Flash(59.2%)**: Blue dot at (7B, 59.2%). - **Qwen3-Coder-30A3B(52.2%)**: Blue dot at (30A3B, 52.2%). - **SWE-Lego-8B(42.2%)**: Blue dot at (8B, 42.2%). - **SWE-Smith(40.2%)**: Blue dot at (32B, 40.2%). - **R2E-Gym(34.4%)**: Blue dot at (32B, 34.4%). - **Llama3-SWE-RL(41%)**: Blue dot at (70B, 41%). - **SWE-Fixer(32.8%)**: Blue dot at (32B, 32.8%). - **SWE-Gym(20.6%)**: Blue dot at (32B, 20.6%). - **Qwen3-30A3B-Instruct(22%)**: Blue dot at (30A3B, 22%). - **SWE-Mirror-7B(22.8%)**: Blue dot at (7B, 22.8%). ### Key Observations 1. **Scale-SWE(64%)** is the highest-performing model, despite being the smallest (3B). 2. **KAT-Dev-32B(62.4%)** is the second-highest, with a larger model size (32B). 3. **Larger models (70B)** like **Llama3-SWE-RL(41%)** and **SWE-Fixer(32.8%)** show lower verification rates compared to mid-sized models. 4. **Mid-sized models (32B)** exhibit a mix of high (e.g., KAT-Dev-32B) and low (e.g., SWE-Gym) performance. 5. **Smaller models (3B–8B)** like **Scale-SWE** and **GLM-4.7-Flash** achieve higher verification rates than some larger models. ### Interpretation The data suggests that **model size does not directly correlate with SWE-Bench verification performance**. While larger models (e.g., 70B) generally underperform compared to mid-sized models (e.g., 32B), exceptions like **Scale-SWE(64%)** demonstrate that architectural efficiency or optimization can outweigh size. The presence of high-performing smaller models (e.g., **GLM-4.7-Flash(59.2%)**) highlights the importance of model design over sheer scale. This implies that **verification efficiency** may depend on factors like training data quality, task-specific tuning, or algorithmic innovation rather than just parameter count.