## Bar Chart: Mean Accuracy and Macro Average After Injection of Internal Error ### Overview The image is a bar chart comparing the mean accuracy and macro average of different models after injecting internal errors. The chart displays the performance of four categories (SCL15, GSM8K-SC, PRM800K-SC, and Macro Average) across various models. Error bars indicate 95% confidence intervals. ### Components/Axes * **Title:** Mean accuracy and macro average (95% confidence intervals) after injection of internal error * **X-axis:** Models (DeepSeek-R1-0528, QwQ-32B, Qwen3-235B-A22B (thinking), Qwen3-30B-A3B (thinking), Qwen3-14B (thinking), gemma-3-27b-it, Qwen3-32B (thinking), gemma-3-12b-it, Phi-4-reasoning-plus) * **Y-axis:** Accuracy (scale from 0.0 to 1.0, incrementing by 0.2) * **Legend:** Located in the top-right corner. * SCL15 (light blue) * GSM8K-SC (light orange) * PRM800K-SC (light green) * Macro Average (red) ### Detailed Analysis **Model Performance Breakdown:** 1. **DeepSeek-R1-0528:** * SCL15: ~0.98 with a small confidence interval. * GSM8K-SC: ~0.94 with a small confidence interval. * PRM800K-SC: ~0.78 with a moderate confidence interval. * Macro Average: ~0.91. 2. **QwQ-32B:** * SCL15: ~0.94 with a small confidence interval. * GSM8K-SC: ~0.93 with a small confidence interval. * PRM800K-SC: ~0.77 with a moderate confidence interval. * Macro Average: ~0.91. 3. **Qwen3-235B-A22B (thinking):** * SCL15: ~0.91 with a small confidence interval. * GSM8K-SC: ~0.92 with a small confidence interval. * PRM800K-SC: ~0.77 with a moderate confidence interval. * Macro Average: ~0.89. 4. **Qwen3-30B-A3B (thinking):** * SCL15: ~0.85 with a small confidence interval. * GSM8K-SC: ~0.90 with a small confidence interval. * PRM800K-SC: ~0.76 with a moderate confidence interval. * Macro Average: ~0.88. 5. **Qwen3-14B (thinking):** * SCL15: ~0.85 with a small confidence interval. * GSM8K-SC: ~0.94 with a small confidence interval. * PRM800K-SC: ~0.74 with a moderate confidence interval. * Macro Average: ~0.84. 6. **gemma-3-27b-it:** * SCL15: ~0.83 with a small confidence interval. * GSM8K-SC: ~0.82 with a small confidence interval. * PRM800K-SC: ~0.78 with a moderate confidence interval. * Macro Average: ~0.82. 7. **Qwen3-32B (thinking):** * SCL15: ~0.80 with a moderate confidence interval. * GSM8K-SC: ~0.91 with a small confidence interval. * PRM800K-SC: ~0.72 with a moderate confidence interval. * Macro Average: ~0.81. 8. **gemma-3-12b-it:** * SCL15: ~0.78 with a moderate confidence interval. * GSM8K-SC: ~0.78 with a small confidence interval. * PRM800K-SC: ~0.76 with a moderate confidence interval. * Macro Average: ~0.77. 9. **Phi-4-reasoning-plus:** * SCL15: ~0.75 with a moderate confidence interval. * GSM8K-SC: ~0.73 with a small confidence interval. * PRM800K-SC: ~0.71 with a moderate confidence interval. * Macro Average: ~0.71. ### Key Observations * **SCL15 consistently shows high accuracy** across all models, generally above 0.8, except for the last two models (gemma-3-12b-it and Phi-4-reasoning-plus). * **GSM8K-SC also exhibits high accuracy**, often comparable to or slightly higher than SCL15. * **PRM800K-SC generally has lower accuracy** compared to the other two, with more variability as indicated by the larger confidence intervals. * **Macro Average generally falls between PRM800K-SC and the higher-performing SCL15 and GSM8K-SC.** * The models **DeepSeek-R1-0528 and QwQ-32B** show the highest overall accuracy across all categories. * The models **gemma-3-12b-it and Phi-4-reasoning-plus** show the lowest overall accuracy across all categories. ### Interpretation The bar chart illustrates the performance of different models under the stress of injected internal errors. The SCL15 and GSM8K-SC categories consistently outperform PRM800K-SC, suggesting they are more robust to the introduced errors. The Macro Average provides a general performance metric, reflecting the combined performance of all categories. The confidence intervals indicate the reliability of the accuracy measurements; larger intervals suggest greater variability in performance. The models DeepSeek-R1-0528 and QwQ-32B appear to be the most resilient to internal errors, while gemma-3-12b-it and Phi-4-reasoning-plus are the least. This information is valuable for selecting models that maintain high accuracy even when faced with internal inconsistencies or noise.

## Bar Chart: Mean Accuracy and Macro Average (95% Confidence Intervals) after Injection of Internal Error ### Overview This bar chart displays the mean accuracy and macro average, with 95% confidence intervals, for several models after the injection of internal error. The x-axis represents different models, and the y-axis represents accuracy. Four data series are presented: SCLIS (light blue), GSM8K-SC (light green), PRM800K-SC (light orange), and Macro Average (red). Error bars indicate the 95% confidence intervals. ### Components/Axes * **Title:** "Mean accuracy and macro average (95% confidence intervals) after injection of internal error" (positioned at the top-center) * **X-axis Label:** "Models" (positioned at the bottom-center) * **Models (Categories):** Deepseek-rl-7.5B, QWQ-32B, Owen3-33B-v2.28 (thinking), Owen3-30B-v3.3B (thinking), Owen3-34B (thinking), gemma-3.2B-it, gemma-3.2B (thinking), gemma-3.12B-it, Phi-3-reasoning-plus * **Y-axis Label:** "Accuracy" (positioned on the left-center) * **Y-axis Scale:** 0.0 to 1.0, with increments of 0.2. * **Legend:** Located at the top-right corner. * **SCLIS:** Light Blue * **GSM8K-SC:** Light Green * **PRM800K-SC:** Light Orange * **Macro Average:** Red ### Detailed Analysis The chart consists of nine models, each with four bars representing the accuracy of SCLIS, GSM8K-SC, PRM800K-SC, and the Macro Average. The error bars represent the 95% confidence interval for each measurement. * **Deepseek-rl-7.5B:** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.908, with an error bar ranging from approximately 0.88 to 0.93. * PRM800K-SC: Approximately 0.894, with an error bar ranging from approximately 0.86 to 0.92. * Macro Average: Approximately 0.933, with an error bar ranging from approximately 0.90 to 0.96. * **QWQ-32B:** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.884, with an error bar ranging from approximately 0.85 to 0.91. * PRM800K-SC: Approximately 0.864, with an error bar ranging from approximately 0.83 to 0.89. * Macro Average: Approximately 0.914, with an error bar ranging from approximately 0.88 to 0.94. * **Owen3-33B-v2.28 (thinking):** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.876, with an error bar ranging from approximately 0.84 to 0.90. * PRM800K-SC: Approximately 0.845, with an error bar ranging from approximately 0.81 to 0.87. * Macro Average: Approximately 0.906, with an error bar ranging from approximately 0.87 to 0.93. * **Owen3-30B-v3.3B (thinking):** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.815, with an error bar ranging from approximately 0.78 to 0.84. * PRM800K-SC: Approximately 0.784, with an error bar ranging from approximately 0.75 to 0.81. * Macro Average: Approximately 0.851, with an error bar ranging from approximately 0.81 to 0.88. * **Owen3-34B (thinking):** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.843, with an error bar ranging from approximately 0.81 to 0.87. * PRM800K-SC: Approximately 0.804, with an error bar ranging from approximately 0.77 to 0.83. * Macro Average: Approximately 0.882, with an error bar ranging from approximately 0.84 to 0.91. * **gemma-3.2B-it:** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.815, with an error bar ranging from approximately 0.78 to 0.84. * PRM800K-SC: Approximately 0.763, with an error bar ranging from approximately 0.73 to 0.79. * Macro Average: Approximately 0.858, with an error bar ranging from approximately 0.82 to 0.89. * **gemma-3.2B (thinking):** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.804, with an error bar ranging from approximately 0.77 to 0.83. * PRM800K-SC: Approximately 0.763, with an error bar ranging from approximately 0.73 to 0.79. * Macro Average: Approximately 0.858, with an error bar ranging from approximately 0.82 to 0.89. * **gemma-3.12B-it:** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.804, with an error bar ranging from approximately 0.77 to 0.83. * PRM800K-SC: Approximately 0.707, with an error bar ranging from approximately 0.67 to 0.74. * Macro Average: Approximately 0.839, with an error bar ranging from approximately 0.80 to 0.87. * **Phi-3-reasoning-plus:** * SCLIS: Approximately 0.998, with a very small error bar. * GSM8K-SC: Approximately 0.67, with an error bar ranging from approximately 0.64 to 0.70. * PRM800K-SC: Approximately 0.643, with an error bar ranging from approximately 0.61 to 0.67. * Macro Average: Approximately 0.757, with an error bar ranging from approximately 0.72 to 0.79. ### Key Observations * SCLIS consistently exhibits the highest accuracy across all models, nearly reaching 1.0. * The Macro Average generally falls between the accuracy of GSM8K-SC and PRM800K-SC. * The accuracy of GSM8K-SC and PRM800K-SC tends to decrease as the models progress from Deepseek-rl-7.5B to Phi-3-reasoning-plus. * Phi-3-reasoning-plus has the lowest accuracy among all models for GSM8K-SC and PRM800K-SC. ### Interpretation The data suggests that SCLIS is the most robust model in maintaining accuracy after the injection of internal error. The decreasing accuracy of GSM8K-SC and PRM800K-SC as the models change indicates that the later models may be more susceptible to internal errors. The Macro Average provides a balanced view of performance, but it is heavily influenced by the high accuracy of SCLIS. The significant drop in accuracy for Phi-3-reasoning-plus, particularly for GSM8K-SC and PRM800K-SC, suggests a potential vulnerability or limitation in this model's architecture or training data when dealing with internal errors. The consistent high performance of SCLIS could be due to its specific design or training methodology, making it more resilient to such errors. The error bars provide a measure of uncertainty, and it's important to consider these intervals when comparing the performance of different models.

## Bar Chart: Mean Accuracy and Macro Average After Internal Error Injection ### Overview This is a grouped bar chart displaying the mean accuracy (with 95% confidence intervals) of nine different language models across three specific benchmarks and a macro average. The chart evaluates model performance after the injection of an internal error. The overall trend shows a general decrease in accuracy from left to right across the listed models. ### Components/Axes * **Title:** "Mean accuracy and macro average (95% confidence intervals) after injection of internal error" * **Y-Axis:** Labeled "Accuracy". Scale ranges from 0.0 to 1.0, with major gridlines at 0.2 intervals. * **X-Axis:** Labeled "Models". Lists nine distinct model names, rotated for readability. * **Legend:** Positioned in the top-right corner. Contains four categories with corresponding colors: * **SCLI5:** Light blue bars. * **GSM8K-SC:** Light orange/peach bars. * **PRM800K-SC:** Light green bars. * **Macro Average:** Red bars. * **Data Representation:** Each model has a cluster of four bars (one for each legend category). Black error bars (I-beams) are present on each bar, representing the 95% confidence interval. ### Detailed Analysis **Model-by-Model Data Extraction (Approximate Values):** 1. **DeepSeek-R1-0528** * SCLI5: ~0.99 (Very high, small error bar) * GSM8K-SC: ~0.96 * PRM800K-SC: ~0.77 * Macro Average: **0.908** (Explicitly labeled) 2. **QwQ-32B** * SCLI5: ~0.97 * GSM8K-SC: ~0.95 * PRM800K-SC: ~0.77 * Macro Average: **0.894** (Explicitly labeled) 3. **Qwen3-235B-A22B (thinking)** * SCLI5: ~0.94 * GSM8K-SC: ~0.94 (Nearly equal to SCLI5) * PRM800K-SC: ~0.76 * Macro Average: **0.876** (Explicitly labeled) 4. **Qwen3-30B-A3B (thinking)** * SCLI5: ~0.84 * GSM8K-SC: ~0.92 (Noticeably higher than SCLI5) * PRM800K-SC: ~0.78 * Macro Average: **0.845** (Explicitly labeled) 5. **Qwen3-14B (thinking)** * SCLI5: ~0.85 * GSM8K-SC: ~0.94 (Highest in this cluster) * PRM800K-SC: ~0.74 * Macro Average: **0.843** (Explicitly labeled) 6. **gemma-3-27b-it** * SCLI5: ~0.88 (Highest in this cluster) * GSM8K-SC: ~0.78 * PRM800K-SC: ~0.79 * Macro Average: **0.815** (Explicitly labeled) 7. **Qwen3-32B (thinking)** * SCLI5: ~0.80 * GSM8K-SC: ~0.91 (Significantly higher than SCLI5) * PRM800K-SC: ~0.70 * Macro Average: **0.804** (Explicitly labeled) 8. **gemma-3-12b-it** * SCLI5: ~0.75 * GSM8K-SC: ~0.79 * PRM800K-SC: ~0.75 * Macro Average: **0.763** (Explicitly labeled) 9. **Phi-4-reasoning-plus** * SCLI5: ~0.73 * GSM8K-SC: ~0.71 * PRM800K-SC: ~0.68 (Lowest bar in the entire chart) * Macro Average: **0.707** (Explicitly labeled) ### Key Observations * **Benchmark Performance Hierarchy:** Across nearly all models, the **GSM8K-SC** (orange) and **SCLI5** (blue) benchmarks consistently yield higher accuracy scores than the **PRM800K-SC** (green) benchmark. PRM800K-SC is the lowest-performing benchmark for every model except `gemma-3-27b-it`, where it is marginally higher than GSM8K-SC. * **Model Performance Trend:** There is a clear downward trend in the **Macro Average** (red bars) from left to right, starting at 0.908 for `DeepSeek-R1-0528` and ending at 0.707 for `Phi-4-reasoning-plus`. This suggests the models are ordered roughly from highest to lowest overall performance under the tested condition. * **Variability in Benchmark Lead:** While SCLI5 is often the top or near-top performer (e.g., `DeepSeek-R1-0528`, `gemma-3-27b-it`), several "thinking" models from the Qwen3 series (`Qwen3-30B-A3B`, `Qwen3-14B`, `Qwen3-32B`) show a distinct pattern where **GSM8K-SC** accuracy significantly outperforms their SCLI5 accuracy. * **Confidence Intervals:** The error bars (95% CI) are generally wider for the PRM800K-SC benchmark compared to the others, indicating greater variability or uncertainty in the results for that specific task. The Macro Average bars have moderate error bars. ### Interpretation This chart provides a comparative analysis of model robustness or capability degradation following an internal error injection. The data suggests: 1. **Task-Dependent Resilience:** Models do not fail uniformly across different types of tasks. Their performance is highly dependent on the specific benchmark (SCLI5, GSM8K-SC, PRM800K-SC), indicating that the "internal error" impacts reasoning or knowledge recall in a task-specific manner. The consistently lower scores on PRM800K-SC might point to this benchmark being more sensitive to the injected error or testing a capability that is more fragile. 2. **Model Architecture/Training Implications:** The superior performance of models like `DeepSeek-R1-0528` and `QwQ-32B` on the macro average suggests they possess greater overall stability or error-correction mechanisms under the tested conditions. The reversal where GSM8K-SC outperforms SCLI5 in several Qwen3 models hints at a potential specialization or a different failure mode in their "thinking" variants. 3. **Benchmarking Insight:** The chart underscores the importance of using multiple, diverse benchmarks (like the three shown) to evaluate AI systems. Relying on a single metric (e.g., only GSM8K-SC) would paint an incomplete and potentially misleading picture of a model's post-error behavior. The macro average serves as a useful, but not sufficient, single summary statistic. In essence, the visualization moves beyond simple accuracy reporting to reveal nuanced patterns of failure and resilience across different AI models and cognitive tasks when subjected to a controlled perturbation.

## Bar Chart: Mean accuracy and macro average (95% confidence intervals) after injection of internal error ### Overview The chart compares the mean accuracy and macro average performance of nine AI models across four categories (SCLI5, GSM8K-SC, PRM800K-SC, Macro Average) after internal error injection. Accuracy values range from 0.0 to 1.0, with 95% confidence intervals represented by error bars. ### Components/Axes - **X-axis**: Models (DeepSeek-R1-0528, QwQ-32B, Qwen3-235B-A228 (thinking), Qwen3-30B-A38 (thinking), Qwen3-14B (thinking), gemma-3-27b-it, Qwen3-32B (thinking), gemma-3-120-it, Phi-4-reasoning-plus) - **Y-axis**: Accuracy (0.0–1.0 in 0.2 increments) - **Legend**: - Blue: SCLI5 - Orange: GSM8K-SC - Green: PRM800K-SC - Red: Macro Average - **Error Bars**: Vertical lines representing 95% confidence intervals (values in parentheses) ### Detailed Analysis 1. **DeepSeek-R1-0528** - SCLI5: 0.998 (±0.012) - GSM8K-SC: 0.965 (±0.015) - PRM800K-SC: 0.772 (±0.021) - Macro Average: 0.908 (±0.018) 2. **QwQ-32B** - SCLI5: 0.978 (±0.014) - GSM8K-SC: 0.952 (±0.018) - PRM800K-SC: 0.770 (±0.020) - Macro Average: 0.894 (±0.017) 3. **Qwen3-235B-A228 (thinking)** - SCLI5: 0.954 (±0.016) - GSM8K-SC: 0.953 (±0.019) - PRM800K-SC: 0.758 (±0.023) - Macro Average: 0.876 (±0.019) 4. **Qwen3-30B-A38 (thinking)** - SCLI5: 0.843 (±0.020) - GSM8K-SC: 0.921 (±0.017) - PRM800K-SC: 0.775 (±0.022) - Macro Average: 0.845 (±0.018) 5. **Qwen3-14B (thinking)** - SCLI5: 0.856 (±0.019) - GSM8K-SC: 0.942 (±0.016) - PRM800K-SC: 0.741 (±0.024) - Macro Average: 0.843 (±0.019) 6. **gemma-3-27b-it** - SCLI5: 0.879 (±0.018) - GSM8K-SC: 0.778 (±0.021) - PRM800K-SC: 0.789 (±0.020) - Macro Average: 0.815 (±0.017) 7. **Qwen3-32B (thinking)** - SCLI5: 0.798 (±0.022) - GSM8K-SC: 0.913 (±0.019) - PRM800K-SC: 0.703 (±0.025) - Macro Average: 0.804 (±0.018) 8. **gemma-3-120-it** - SCLI5: 0.763 (±0.023) - GSM8K-SC: 0.789 (±0.020) - PRM800K-SC: 0.762 (±0.024) - Macro Average: 0.763 (±0.019) 9. **Phi-4-reasoning-plus** - SCLI5: 0.731 (±0.025) - GSM8K-SC: 0.718 (±0.026) - PRM800K-SC: 0.667 (±0.027) - Macro Average: 0.707 (±0.020) ### Key Observations - **Highest Performance**: DeepSeek-R1-0528 achieves the highest accuracy across all categories (SCLI5: 0.998, Macro Average: 0.908). - **Lowest Performance**: Phi-4-reasoning-plus has the lowest accuracy (SCLI5: 0.731, Macro Average: 0.707). - **Macro Average Consistency**: The Macro Average (red bars) is consistently lower than individual model accuracies, indicating aggregation reduces performance. - **Error Bar Variability**: Larger error bars (e.g., Phi-4-reasoning-plus: ±0.025–0.027) suggest greater uncertainty in lower-performing models. - **Model-Specific Trends**: - SCLI5 (blue) and GSM8K-SC (orange) generally outperform PRM800K-SC (green). - Qwen3-30B-A38 and Qwen3-14B show significant drops in SCLI5 accuracy compared to other models. ### Interpretation The data suggests that internal error injection reduces model robustness, with performance degradation varying by architecture. SCLI5 and GSM8K-SC demonstrate higher resilience, while PRM800K-SC struggles across all models. The Macro Average's lower values highlight the challenges of combining diverse models under error conditions. Notably, larger error bars in weaker models (e.g., Phi-4) indicate less reliable measurements, emphasizing the need for targeted improvements in error-handling capabilities. The consistent underperformance of PRM800K-SC suggests architectural limitations in handling injected errors compared to other frameworks.