## Chart: Proportion vs. Score for Different Models and Categories ### Overview The image presents a series of line graphs arranged in a grid. Each graph displays the relationship between the proportion of scores greater than or equal to a given score, for three different categories: Facilitation, Irrelevance, and Interference. The graphs are grouped by model type (L3.2-1B, L3.2-3B, L3.2-3B-I, and L3.1-8B) and category (Syntax, Common Sense, and Math). ### Components/Axes * **Title:** (a) (top-left) * **X-axis:** Score, ranging from 0 to 1.0 in increments of 0.5. * **Y-axis:** Proportion ≥ Score, ranging from 0% to 100% in increments of 50%. * **Models (Columns):** L3.2-1B, L3.2-3B, L3.2-3B-I, L3.1-8B. * **Categories (Rows):** Syntax, Common Sense, Math. * **Legend (Bottom):** * Facilitation (Green line) * Irrelevance (Blue line) * Interference (Red line) ### Detailed Analysis **Syntax Category:** * **L3.2-1B:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. * **L3.2-3B:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. * **L3.2-3B-I:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. * **L3.1-8B:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. **Common Sense Category:** * **L3.2-1B:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. * **L3.2-3B:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. * **L3.2-3B-I:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. * **L3.1-8B:** * Irrelevance (Blue): Starts at 100%, remains high until Score ~0.8, then drops sharply to ~10%. * Facilitation (Green): Starts at ~30%, gradually decreases to ~10%. * Interference (Red): Starts at ~20%, gradually decreases to ~5%. **Math Category:** * **L3.2-1B:** * Irrelevance (Blue): Starts at ~100%, decreases to ~10% at Score = 1. * Facilitation (Green): Starts at ~100%, decreases to ~10% at Score = 1. * Interference (Red): Starts at ~10%, remains low. * **L3.2-3B:** * Irrelevance (Blue): Starts at ~100%, decreases to ~10% at Score = 1. * Facilitation (Green): Starts at ~100%, decreases to ~10% at Score = 1. * Interference (Red): Starts at ~10%, remains low. * **L3.2-3B-I:** * Irrelevance (Blue): Starts at ~70%, decreases to ~10% at Score = 1. * Facilitation (Green): Starts at ~100%, decreases to ~10% at Score = 1. * Interference (Red): Starts at ~10%, remains low. * **L3.1-8B:** * Irrelevance (Blue): Starts at ~60%, decreases to ~10% at Score = 1. * Facilitation (Green): Starts at ~60%, decreases to ~10% at Score = 1. * Interference (Red): Starts at ~10%, remains low. ### Key Observations * For Syntax and Common Sense, the Irrelevance scores are consistently high across all models until a score of approximately 0.8, after which they drop sharply. * For Syntax and Common Sense, Facilitation and Interference scores are relatively low and decrease gradually with increasing score. * For Math, Facilitation and Irrelevance scores start high and decrease gradually, while Interference scores remain low. * The models L3.2-1B, L3.2-3B, and L3.2-3B-I show very similar performance within each category. * Model L3.1-8B shows a slightly different trend in the Math category compared to the other models. ### Interpretation The data suggests that: * Irrelevance is a significant factor in Syntax and Common Sense tasks, as a large proportion of scores are high until a certain threshold. * Facilitation and Interference play a less prominent role in Syntax and Common Sense, with lower proportions of high scores. * In Math tasks, both Facilitation and Irrelevance are initially high but decrease as the required score increases, indicating that these factors become less influential at higher performance levels. * The models L3.2-1B, L3.2-3B, and L3.2-3B-I perform similarly across all categories, while L3.1-8B exhibits some differences, particularly in the Math category. This could indicate that L3.1-8B has a different approach or strengths/weaknesses compared to the other models.

\n ## Chart: Proportion of Scores by Model and Category ### Overview The image presents a 3x4 grid of line charts, visualizing the proportion of scores (y-axis, ranging from 0% to 100%) greater than or equal to a given score (x-axis, ranging from 0 to 1). The charts are categorized by model (L3.2-1B, L3.2-3B, L3.2-3B-I, L3.1-8B) along the top row and evaluation category (Syntax, Common Sense, Match) down the left column. Each chart displays three lines representing different phenomena: Facilitation, Irrelevance, and Interference. A legend in the bottom-center identifies the colors associated with each phenomenon. ### Components/Axes * **X-axis:** "Score" (ranging from 0 to 1) * **Y-axis:** "Proportion ≥ Score" (ranging from 0% to 100%) * **Models (Top Row):** L3.2-1B, L3.2-3B, L3.2-3B-I, L3.1-8B * **Categories (Left Column):** Syntax, Common Sense, Match * **Legend:** * Facilitation (Light Green) * Irrelevance (Light Blue) * Interference (Light Red) ### Detailed Analysis The chart is structured as a grid of 12 individual line charts. Each chart shows the proportion of scores above a given threshold for each of the three phenomena (Facilitation, Irrelevance, Interference) for a specific model and category combination. **Model L3.2-1B:** * **Syntax:** Facilitation starts at approximately 95% and decreases to around 60% as the score increases to 1. Irrelevance starts at approximately 5% and increases to around 40% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Common Sense:** Facilitation starts at approximately 90% and decreases to around 50% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 50% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Match:** Facilitation starts at approximately 90% and decreases to around 40% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 60% as the score increases to 1. Interference remains consistently low, around 0-10%. **Model L3.2-3B:** * **Syntax:** Facilitation starts at approximately 95% and decreases to around 60% as the score increases to 1. Irrelevance starts at approximately 5% and increases to around 40% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Common Sense:** Facilitation starts at approximately 90% and decreases to around 50% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 50% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Match:** Facilitation starts at approximately 90% and decreases to around 40% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 60% as the score increases to 1. Interference remains consistently low, around 0-10%. **Model L3.2-3B-I:** * **Syntax:** Facilitation starts at approximately 95% and decreases to around 60% as the score increases to 1. Irrelevance starts at approximately 5% and increases to around 40% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Common Sense:** Facilitation starts at approximately 90% and decreases to around 50% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 50% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Match:** Facilitation starts at approximately 90% and decreases to around 40% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 60% as the score increases to 1. Interference remains consistently low, around 0-10%. **Model L3.1-8B:** * **Syntax:** Facilitation starts at approximately 95% and decreases to around 60% as the score increases to 1. Irrelevance starts at approximately 5% and increases to around 40% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Common Sense:** Facilitation starts at approximately 90% and decreases to around 50% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 50% as the score increases to 1. Interference remains consistently low, around 0-10%. * **Match:** Facilitation starts at approximately 90% and decreases to around 40% as the score increases to 1. Irrelevance starts at approximately 10% and increases to around 60% as the score increases to 1. Interference remains consistently low, around 0-10%. ### Key Observations * Facilitation consistently has the highest proportion of scores, especially at lower score values. * Irrelevance generally increases as the score increases. * Interference remains consistently low across all models and categories. * The trends are remarkably similar across all four models and all three categories. * There are no significant outliers or anomalies. ### Interpretation The data suggests that, across all models and evaluation categories, the phenomenon of "Facilitation" is dominant, meaning that a large proportion of scores are high. As the score threshold increases, the proportion of "Facilitation" decreases while the proportion of "Irrelevance" increases. This indicates that as the required performance level rises, the model is more likely to produce irrelevant outputs. "Interference" is consistently minimal, suggesting that the model rarely produces outputs that actively hinder performance. The consistency of these trends across different models (L3.2-1B, L3.2-3B, L3.2-3B-I, L3.1-8B) and evaluation categories (Syntax, Common Sense, Match) suggests a fundamental characteristic of these models' behavior. The models generally perform well, but their performance degrades as the required score increases, leading to more irrelevant outputs. The lack of significant differences between the models suggests that increasing model size (from 1B to 8B parameters) or using different training methods (as indicated by the "-I" suffix) does not substantially alter this overall trend. The consistent low level of interference suggests that the models are generally not actively harmful, but rather become less useful as the performance requirements increase.

## [Multi-Panel Line Chart]: Performance Proportion by Score Across Models and Domains ### Overview The image displays a 3x4 grid of line charts, labeled as panel (a). It visualizes the performance of four different language models across three cognitive domains. Each chart plots the "Proportion ≥ Score" (y-axis) against a normalized "Score" (x-axis, ranging from 0 to 1). The data is broken down by three experimental conditions, represented by colored lines. ### Components/Axes * **Overall Grid Structure:** * **Columns (Models):** Four columns, each headed by a model identifier in a gray box at the top: `L3.2-1B`, `L3.2-3B`, `L3.2-3B-I`, `L3.1-8B`. * **Rows (Domains):** Three rows, each labeled on the right side in a vertical gray box: `Syntax` (top row), `Common Sense` (middle row), `Math` (bottom row). * **Axes (for each subplot):** * **Y-axis:** Labeled "Proportion ≥ Score" on the far left of the grid. Scale is marked at 0%, 50%, and 100%. * **X-axis:** Labeled "Score" at the bottom center of the grid. Scale is marked at 0, 0.5, and 1. * **Legend:** Positioned at the bottom center of the entire figure. It defines three colored lines: * **Green line:** `Facilitation` * **Blue line:** `Irrelevance` * **Red line:** `Interference` ### Detailed Analysis The following describes the trend for each line in each subplot. Values are approximate visual estimates. **Row 1: Syntax** * **L3.2-1B:** Blue (Irrelevance) starts near 100% at Score=0, declines steadily to ~50% at Score=1. Green (Facilitation) starts near 40%, declines to near 0%. Red (Interference) starts near 20%, declines to near 0%. * **L3.2-3B:** Blue starts near 100%, declines to ~50%. Green starts near 35%, declines to near 0%. Red starts near 20%, declines to near 0%. * **L3.2-3B-I:** Blue starts near 100%, declines to ~50%. Green starts near 35%, declines to near 0%. Red starts near 20%, declines to near 0%. * **L3.1-8B:** Blue starts near 100%, declines to ~50%. Green starts near 35%, declines to near 0%. Red starts near 20%, declines to near 0%. **Row 2: Common Sense** * **L3.2-1B:** Blue starts near 100%, declines to ~70%. Green starts near 20%, declines to near 0%. Red starts near 15%, declines to near 0%. * **L3.2-3B:** Blue starts near 100%, declines to ~50%. Green starts near 40%, declines to near 0%. Red starts near 20%, declines to near 0%. * **L3.2-3B-I:** Blue starts near 100%, declines to ~60%. Green starts near 30%, declines to near 0%. Red starts near 15%, declines to near 0%. * **L3.1-8B:** Blue starts near 100%, declines to ~50%. Green starts near 45%, declines to near 0%. Red starts near 15%, declines to near 0%. **Row 3: Math** * **L3.2-1B:** Blue starts near 100%, declines sharply to near 0% by Score=0.7. Green starts near 90%, declines to near 0% by Score=1. Red starts near 20%, declines to near 0%. * **L3.2-3B:** Blue starts near 100%, declines to near 0% by Score=1. Green starts near 70%, declines to near 0% by Score=1. Red starts near 10%, declines to near 0%. * **L3.2-3B-I:** Blue starts near 100%, declines to near 0% by Score=1. Green starts near 65%, declines to near 0% by Score=1. Red starts near 10%, declines to near 0%. * **L3.1-8B:** Blue starts near 100%, declines to near 0% by Score=1. Green starts near 60%, declines to near 0% by Score=1. Red starts near 10%, declines to near 0%. ### Key Observations 1. **Consistent Hierarchy:** In nearly all charts, the blue line (Irrelevance) is the highest, followed by the green line (Facilitation), with the red line (Interference) consistently the lowest. 2. **Domain-Specific Behavior in Math:** The Math domain (bottom row) shows a distinctly different pattern. The blue (Irrelevance) and green (Facilitation) lines start much closer together and both decline more steeply toward zero as the score increases, compared to the Syntax and Common Sense domains. 3. **Model Similarity:** The four models (columns) show remarkably similar patterns within each domain row. The most notable difference is in the Math domain, where the starting point for the green line (Facilitation) appears slightly lower for the larger models (L3.2-3B, L3.2-3B-I, L3.1-8B) compared to L3.2-1B. 4. **Universal Low Interference:** The red line (Interference) is consistently low (starting below 20%) and flat across all models and domains, indicating this condition yields poor performance regardless of score threshold. ### Interpretation This chart likely comes from a study on how different types of contextual information affect language model performance on various tasks. The "Proportion ≥ Score" metric suggests it's showing the cumulative distribution of scores—what fraction of test samples achieved at least a given score. * **What the data suggests:** The results demonstrate a clear and consistent effect of context type. **Irrelevance** (blue) provides the best baseline performance, suggesting models perform best when given context that is not directly related but also not contradictory. **Facilitation** (green) provides a moderate boost over **Interference** (red), which severely hampers performance. This hierarchy holds across syntax and common sense tasks. * **The Math Anomaly:** The dramatic convergence and steep decline of the Irrelevance and Facilitation lines in the Math domain indicate that for mathematical reasoning, the type of context matters less as the required score threshold increases. High-scoring math problems appear to be difficult for all models regardless of context, with performance dropping to near zero for scores above ~0.7-0.8. * **Model Scaling:** The similarity across model sizes (from 1B to 8B parameters) suggests that this pattern of context sensitivity is a fundamental characteristic of the model architectures or training paradigms being tested, rather than an artifact of model scale. The slight differences in the Math domain may hint at subtle scaling effects for complex reasoning. * **Underlying Mechanism:** The consistently poor performance under **Interference** implies that contradictory information is highly disruptive. The superiority of **Irrelevance** over **Facilitation** is intriguing; it may suggest that models are better at ignoring irrelevant information than they are at correctly utilizing subtly helpful information, or that the "Facilitation" context in this experiment introduced complexity that offset its benefits.

## Line Chart Grid: Model Performance Across Categories ### Overview The image displays a 4x3 grid of line charts comparing three performance metrics (Facilitation, Irrelevance, Interference) across four model configurations (L3.2-1B, L3.2-3B, L3.2-3B-I, L3.1-8B) and three categories (Syntax, Common Sense, Math). Each chart shows how the proportion of instances meeting or exceeding a performance score threshold changes as the score threshold increases from 0 to 1. ### Components/Axes - **X-axis**: Score (0 to 1 in 0.1 increments) - **Y-axis**: Proportion ≥ Score (0% to 100% in 50% increments) - **Legend**: - Green: Facilitation - Blue: Irrelevance - Red: Interference - **Chart Titles**: Model configurations (e.g., L3.2-1B) - **Category Labels**: Right-side text indicating evaluation domain (Syntax, Common Sense, Math) ### Detailed Analysis #### Model Configurations 1. **L3.2-1B** - **Syntax**: Blue (Irrelevance) starts near 100% at Score 0, drops sharply to ~50% at Score 0.5, then plateaus. Green (Facilitation) starts ~40%, rises to ~60% at Score 0.2, then declines. Red (Interference) starts ~20%, peaks at ~30% at Score 0.3, then declines. - **Common Sense**: Similar pattern to Syntax, with Irrelevance dropping faster. - **Math**: Irrelevance drops more gradually, Facilitation shows a U-shaped curve. 2. **L3.2-3B** - **Syntax**: Irrelevance drops from ~90% to ~40% by Score 0.5. Facilitation peaks at ~50% at Score 0.3. - **Common Sense**: Irrelevance declines more gradually than Syntax. - **Math**: Facilitation shows a steeper decline after Score 0.5. 3. **L3.2-3B-I** - **Syntax**: Irrelevance drops sharply to ~30% at Score 0.5. Facilitation peaks earlier (~0.2) than L3.2-3B. - **Common Sense**: Similar to Syntax but with less pronounced Facilitation peak. - **Math**: Facilitation declines more steeply after Score 0.5. 4. **L3.1-8B** - **Syntax**: Irrelevance drops from ~85% to ~45% at Score 0.5. Facilitation peaks at ~55% at Score 0.3. - **Common Sense**: Irrelevance decline is more gradual than Syntax. - **Math**: Facilitation shows a bimodal pattern with peaks at Scores 0.2 and 0.7. ### Key Observations - **Irrelevance** consistently decreases with higher scores across all models and categories, suggesting improved performance at higher thresholds. - **Facilitation** exhibits varied patterns: U-shaped curves in Math (L3.2-1B), bimodal in Math (L3.1-8B), and single peaks in Syntax/Common Sense. - **Interference** shows minimal impact in most charts, with only slight fluctuations near Score 0.3-0.5. - **Model Differences**: L3.2-3B-I shows more pronounced Facilitation peaks than L3.2-3B, while L3.1-8B demonstrates the most complex Math performance patterns. ### Interpretation The data suggests that: 1. **Threshold Sensitivity**: All models show diminishing returns in performance as score thresholds increase, with Irrelevance being the most sensitive metric. 2. **Facilitation Variability**: The U-shaped and bimodal patterns in Math indicate potential trade-offs between different skill levels or task types. 3. **Model Architecture Impact**: The L3.2-3B-I variant's earlier Facilitation peak suggests architectural modifications (e.g., interference mitigation) may improve mid-range performance. 4. **Category-Specific Behavior**: Math performance shows more complex dynamics than Syntax/Common Sense, possibly reflecting different cognitive demands. The charts highlight trade-offs between different performance dimensions and suggest that model configuration significantly impacts how these metrics interact across evaluation domains.