## Heatmap: Baseline - Core Generalization - Qwen-2.5 1.5B ### Overview The image is a heatmap visualizing the accuracy of a baseline model (Qwen-2.5 1.5B) on a core generalization task. The heatmap displays accuracy percentages for different "Types" (1 to 7) across varying sequence "Lengths" (0 to 19). The color intensity corresponds to the accuracy, with darker blue indicating higher accuracy and lighter blue indicating lower accuracy. ### Components/Axes * **Title:** Baseline - Core Generalization - Qwen-2.5 1.5B * **Y-axis:** "Type" labeled 1 to 7. * **X-axis:** "Length" labeled 0 to 19. * **Colorbar (Right):** "Accuracy (%)" ranging from 0 to 100, with a gradient from light blue (0%) to dark blue (100%). ### Detailed Analysis The heatmap presents accuracy values for each combination of "Type" and "Length." Here's a breakdown of the data: * **Type 1:** Accuracy starts at 100% for length 0, then decreases to 88.7% (length 1), 92.3% (length 2), 80.7% (length 3), 76.7% (length 4), 72.7% (length 5), 71.7% (length 6), 75.7% (length 7), 73.0% (length 8), and 77.3% (length 9). * **Type 2:** Accuracy is high across all lengths, starting at 99.3% (length 0), 98.0% (length 1), 100.0% (length 2), 97.0% (length 3), 96.3% (length 4), 95.7% (length 5), 96.7% (length 6), 96.7% (length 7), 97.3% (length 8), and 97.3% (length 9). * **Type 3:** Accuracy starts at 100% (length 0), then decreases to 97.7% (length 1), 94.0% (length 2), 90.3% (length 3), 86.7% (length 4), 80.0% (length 5), 75.3% (length 6), 76.3% (length 7), 77.0% (length 8), 77.3% (length 9), 73.0% (length 10), 77.3% (length 11), 69.7% (length 12), 75.3% (length 13), 79.0% (length 14), 75.3% (length 15), 72.0% (length 16), 78.3% (length 17), 76.7% (length 18), and 71.3% (length 19). * **Type 4:** Accuracy starts at 96.0% (length 0), then decreases to 95.3% (length 1), 89.7% (length 2), 90.0% (length 3), 80.3% (length 4), 74.7% (length 5), 78.3% (length 6), 75.7% (length 7), 76.7% (length 8), 73.3% (length 9), and 53.3% (length 10). * **Type 5:** Accuracy values are only available for lengths 7 to 19, starting at 69.3% (length 7), 72.3% (length 8), 71.0% (length 9), 83.3% (length 10), 77.3% (length 11), 79.7% (length 12), 76.7% (length 13), 79.7% (length 14), 71.3% (length 15), 79.7% (length 16), 74.7% (length 17), 70.7% (length 18), and 77.3% (length 19). * **Type 6:** Accuracy is consistently high across all lengths, starting at 100.0% (length 0), 100.0% (length 1), 99.0% (length 2), 98.0% (length 3), 98.3% (length 4), 97.7% (length 5), 98.7% (length 6), 98.0% (length 7), 96.0% (length 8), 96.3% (length 9), 96.3% (length 10), 94.3% (length 11), 93.7% (length 12), 95.3% (length 13), 94.7% (length 14), 91.7% (length 15), 95.3% (length 16), 94.7% (length 17), and 93.3% (length 18). * **Type 7:** Accuracy starts at 100.0% (length 0), then decreases to 98.3% (length 1), 97.0% (length 2), 94.0% (length 3), 92.7% (length 4), 89.7% (length 5), 85.3% (length 6), 87.0% (length 7), 81.3% (length 8), 82.3% (length 9), 83.7% (length 10), 77.7% (length 11), 74.0% (length 12), and 73.7% (length 13). ### Key Observations * Types 2 and 6 consistently show high accuracy across all lengths. * Types 1, 3, 4, and 7 show a general decreasing trend in accuracy as the length increases. * Type 5 has missing data for shorter lengths (0-6). * Type 4 shows a significant drop in accuracy at length 10 (53.3%). ### Interpretation The heatmap illustrates the performance of the Qwen-2.5 1.5B model on different types of tasks or data categories ("Types") as the sequence length increases. The high accuracy for Types 2 and 6 suggests that the model generalizes well for these specific tasks, regardless of the input length. The decreasing accuracy for Types 1, 3, 4, and 7 indicates that the model's performance degrades as the sequence length increases, possibly due to the model's difficulty in handling longer dependencies or increased complexity. The missing data for Type 5 at shorter lengths could indicate that this type of task is only relevant or defined for longer sequences. The significant drop in accuracy for Type 4 at length 10 could be due to a specific characteristic of the data or task at that length, which the model struggles to handle. Overall, the heatmap provides insights into the model's strengths and weaknesses in generalizing across different tasks and sequence lengths.

## Heatmap: Baseline - Core Generalization - Qwen-2.5 1.5B ### Overview This image presents a heatmap visualizing the accuracy of a model (Qwen-2.5 1.5B) across different sequence lengths and input types. The heatmap displays accuracy as a percentage, with color intensity representing the accuracy level. ### Components/Axes * **Title:** Baseline - Core Generalization - Qwen-2.5 1.5B (Top-center) * **X-axis:** Length (ranging from 0 to 19, in increments of 1). (Bottom-center) * **Y-axis:** Type (with categories: 'H', '2', 'M', '4', 'U', '7'). (Left-center) * **Color Scale:** Accuracy (%) ranging from 0 to 100. (Right-center) The color gradient transitions from light blue (low accuracy) to dark teal/green (high accuracy). ### Detailed Analysis The heatmap is a 6x20 grid, with each cell representing the accuracy for a specific combination of 'Type' and 'Length'. I will analyze each row (Type) and describe the trend, then list the approximate values. * **Type 'H'**: Accuracy is consistently high, starting at approximately 100% for Length 0 and decreasing slightly to around 73.3% for Length 19. The trend is a gentle downward slope. * Length 0: 100.0% * Length 1: 88.7% * Length 2: 92.3% * Length 3: 80.7% * Length 4: 72.7% * Length 5: 71.7% * Length 6: 73.0% * Length 7: 73.3% * **Type '2'**: Accuracy starts very high at approximately 99.3% for Length 0, and decreases to around 91.3% for Length 19. The trend is a gentle downward slope. * Length 0: 99.3% * Length 1: 98.0% * Length 2: 100.0% * Length 3: 96.3% * Length 4: 95.7% * Length 5: 96.7% * Length 6: 97.3% * Length 7: 91.3% * **Type 'M'**: Accuracy begins at approximately 97.7% for Length 0, and decreases to around 71.3% for Length 19. The trend is a downward slope, slightly steeper than 'H' and '2'. * Length 0: 97.7% * Length 1: 94.0% * Length 2: 86.7% * Length 3: 80.0% * Length 4: 75.3% * Length 5: 77.0% * Length 6: 71.0% * Length 7: 73.3% * **Type '4'**: Accuracy starts at approximately 96.0% for Length 0, and decreases to around 53.3% for Length 19. The trend is a more pronounced downward slope. * Length 0: 96.0% * Length 1: 95.3% * Length 2: 89.0% * Length 3: 80.3% * Length 4: 74.8% * Length 5: 75.7% * Length 6: 73.3% * Length 7: 53.3% * **Type 'U'**: Accuracy starts at approximately 69.3% for Length 0, and increases to around 79.7% for Length 7, then decreases to around 70.7% for Length 19. The trend is a slight increase followed by a decrease. * Length 0: 69.3% * Length 1: 72.3% * Length 2: 80.3% * Length 3: 77.3% * Length 4: 79.7% * Length 5: 79.9% * Length 6: 71.3% * Length 7: 74.7% * **Type '7'**: Accuracy starts at approximately 100.0% for Length 0, and decreases to around 73.7% for Length 19. The trend is a gentle downward slope. * Length 0: 100.0% * Length 1: 98.3% * Length 2: 97.0% * Length 3: 94.7% * Length 4: 89.7% * Length 5: 85.3% * Length 6: 81.3% * Length 7: 77.7% ### Key Observations * The highest accuracies are generally observed for shorter lengths (0-5) across most types. * Type 'H' and '7' consistently exhibit the highest accuracy across all lengths. * Type '4' shows the most significant decrease in accuracy as length increases. * Type 'U' has a unique pattern of initially increasing accuracy before decreasing. * Accuracy generally decreases as the sequence length increases for most types. ### Interpretation The heatmap demonstrates the model's performance on different input types and sequence lengths. The consistent high accuracy for types 'H' and '7' suggests the model is well-suited for those specific input characteristics. The decreasing accuracy with increasing length indicates a potential limitation in the model's ability to handle longer sequences effectively. The unique behavior of type 'U' might suggest a specific pattern or characteristic within that input type that initially benefits from increased length but then becomes detrimental. This data is valuable for understanding the model's strengths and weaknesses and for guiding further development or fine-tuning efforts. The "Baseline" in the title suggests this is a starting point for comparison with other models or configurations. The "Core Generalization" indicates the test focuses on fundamental capabilities rather than specialized tasks.

## Heatmap: Baseline - Core Generalization - Qwen-2.5 1.5B ### Overview This image is a heatmap visualizing the performance accuracy (in percentage) of a model named "Qwen-2.5 1.5B" on a "Core Generalization" task. The chart plots performance across two dimensions: "Type" (y-axis) and "Length" (x-axis). The color intensity of each cell represents the accuracy percentage, with a corresponding color bar legend on the right. ### Components/Axes * **Title:** "Baseline - Core Generalization - Qwen-2.5 1.5B" (centered at the top). * **Y-Axis (Vertical):** Labeled "Type". It contains 7 discrete categories, numbered 1 through 7 from top to bottom. * **X-Axis (Horizontal):** Labeled "Length". It contains 20 discrete categories, numbered 0 through 19 from left to right. * **Legend/Color Bar:** Located on the far right of the chart. It is a vertical gradient bar labeled "Accuracy (%)". The scale runs from 0 (lightest blue/white) at the bottom to 100 (darkest blue) at the top, with tick marks at 0, 20, 40, 60, 80, and 100. * **Data Cells:** The main chart area is a grid where each cell's color corresponds to an accuracy value. The numerical accuracy percentage is printed in white text within each colored cell. ### Detailed Analysis The following table reconstructs the data from the heatmap. Empty cells indicate no data point was recorded for that Type/Length combination. | Type \ Length | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | | :------------ | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | | **1** | 100.0 | 88.7 | 92.3 | 80.7 | 76.7 | 72.7 | 71.7 | 75.7 | 73.0 | 77.3 | | | | | | | | | | | | **2** | | 99.3 | 98.0 | 100.0 | 97.0 | 96.3 | 95.7 | 96.7 | 96.7 | 97.3 | 97.3 | | | | | | | | | | | **3** | 100.0 | 97.7 | 94.0 | 90.3 | 86.7 | 80.0 | 75.3 | 76.3 | 77.0 | 77.3 | 73.0 | 77.3 | 69.7 | 75.3 | 79.0 | 75.3 | 72.0 | 78.3 | 76.7 | 71.3 | | **4** | | 96.0 | 95.3 | 89.7 | 90.0 | 80.3 | 74.7 | 78.3 | 75.7 | 76.7 | 73.3 | 53.3 | | | | | | | | | | **5** | | | | | | | | 69.3 | 72.3 | 71.0 | 83.3 | 77.3 | 79.7 | 76.7 | 79.7 | 71.3 | 79.7 | 74.7 | 70.7 | 77.3 | | **6** | 100.0 | 100.0 | 99.0 | 98.0 | 98.3 | 97.7 | 98.7 | 98.0 | 96.0 | 96.3 | 96.3 | 94.3 | 93.7 | 95.3 | 94.7 | 91.7 | 95.3 | 94.7 | 93.3 | | | **7** | 100.0 | 98.3 | 97.0 | 94.0 | 92.7 | 89.7 | 85.3 | 87.0 | 81.3 | 82.3 | 83.7 | 77.7 | 74.0 | 73.7 | | | | | | | ### Key Observations 1. **Performance Range:** Accuracy values range from a low of **53.3%** (Type 4, Length 11) to multiple perfect scores of **100.0%**. 2. **Type 6 Dominance:** Type 6 exhibits the strongest and most consistent performance, maintaining accuracy above 91.7% across all measured lengths (0-18). It starts at 100% and shows only a very gradual decline. 3. **Type 4 Anomaly:** Type 4 shows a significant performance drop at **Length 11 (53.3%)**, which is the lowest value in the entire dataset. This is a sharp outlier compared to its neighboring values (73.3% at Length 10 and no data after). 4. **Length Coverage:** Different "Types" are evaluated over different ranges of "Length": * Types 1, 2, and 7 are evaluated for shorter lengths (0-9, 1-10, and 0-13 respectively). * Types 3, 5, and 6 are evaluated for longer lengths (0-19, 7-19, and 0-18 respectively). * Type 4 is evaluated for lengths 1-11. 5. **General Trend:** For most types, there is a general downward trend in accuracy as "Length" increases, though the rate of decline varies significantly by type. Type 6 is the most resilient to increasing length. 6. **Color Correlation:** The color gradient accurately reflects the numerical values. The darkest blue cells correspond to 100% or high-90s accuracy, while the lightest blue cell corresponds to the 53.3% value. ### Interpretation This heatmap provides a diagnostic view of the Qwen-2.5 1.5B model's ability to generalize core tasks as a function of problem "Type" and "Length". * **What the data suggests:** The model's generalization capability is highly dependent on the specific "Type" of task. It demonstrates robust, near-perfect performance on Type 6 across a wide range of lengths, suggesting this task type is well-learned or inherently easier for the model. Conversely, the dramatic failure of Type 4 at Length 11 indicates a specific weakness or a point where the task complexity exceeds the model's capacity for that particular type. * **Relationship between elements:** The "Type" axis likely represents different categories or formulations of a core reasoning or generalization task. The "Length" axis likely represents the complexity or sequential length of the problem instance. The chart reveals an interaction effect: the impact of increasing length on accuracy is not uniform but is mediated by the task type. * **Notable patterns and anomalies:** * **The Type 4 Cliff:** The drop to 53.3% is the most salient anomaly. It could indicate a specific failure mode, a data distribution gap, or a threshold effect where the model's reasoning breaks down for that type at that specific length. * **The Type 6 Plateau:** The sustained high performance of Type 6 is notable. It suggests the model has a strong, length-invariant representation for this task type. * **Missing Data:** The staggered start and end points for different types (e.g., Type 5 starts at Length 7) imply the evaluation was designed to test types over their relevant or challenging length ranges, rather than a uniform grid. In summary, the heatmap is a valuable tool for identifying model strengths (Type 6), weaknesses (Type 4 at Length 11), and the varying sensitivity of different task types to increasing problem length. It guides further investigation into why certain types generalize better than others.

# Technical Document Extraction: Heatmap Analysis ## Title **Baseline - Core Generalization - Qwen-2.5 1.5B** --- ## Axes and Labels - **X-Axis (Horizontal):** - Label: `Length` - Values: `0` to `19` (integer increments) - Spatial Position: Bottom edge of heatmap - **Y-Axis (Vertical):** - Label: `Type` - Values: `1` to `7` (integer increments) - Spatial Position: Left edge of heatmap - **Colorbar (Legend):** - Label: `Accuracy (%)` - Range: `0%` (lightest blue) to `100%` (darkest blue) - Spatial Position: Right edge of heatmap --- ## Heatmap Structure - **Rows:** 7 (Types 1–7) - **Columns:** 20 (Lengths 0–19) - **Cell Values:** Accuracy percentages (e.g., `100.0`, `88.7`, `92.3`, etc.) --- ## Data Table Reconstruction | Type \ Length | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | |---------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------| | **1** | 100.0 | 88.7 | 92.3 | 80.7 | 76.7 | 72.7 | 71.7 | 75.7 | 73.0 | 77.3 | | | | | | | | | | | | **2** | | 99.3 | 98.0 | 100.0 | 97.0 | 96.3 | 95.7 | 96.7 | 96.7 | 97.3 | 97.3 | | | | | | | | | | | **3** | 100.0 | 97.7 | 94.0 | 90.3 | 86.7 | 80.0 | 75.3 | 76.3 | 77.0 | 77.3 | 73.0 | 77.3 | 69.7 | 75.3 | 79.0 | 75.3 | 72.0 | 78.3 | 76.7 | 71.3 | | **4** | | 96.0 | 95.3 | 89.7 | 90.0 | 80.3 | 74.7 | 78.3 | 75.7 | 76.7 | 73.3 | 53.3 | | | | | | | | | | **5** | | | | | | | | 69.3 | 72.3 | 71.0 | 83.3 | 77.3 | 79.7 | 76.7 | 79.7 | 71.3 | 79.7 | 74.7 | 70.7 | 77.3 | | **6** | 100.0 | 100.0 | 99.0 | 98.0 | 98.3 | 97.7 | 98.7 | 98.0 | 96.0 | 96.3 | 96.3 | 94.3 | 93.7 | 95.3 | 94.7 | 91.7 | 95.3 | 94.7 | 93.3 | | | **7** | 100.0 | 98.3 | 97.0 | 94.0 | 92.7 | 89.7 | 85.3 | 87.0 | 81.3 | 82.3 | 83.7 | 77.7 | 74.0 | 73.7 | | | | | | | --- ## Key Trends 1. **General Pattern:** - Accuracy decreases as `Length` increases for most `Type` values. - Exceptions: - Type 6 maintains high accuracy (90%+) until `Length=18`, then drops sharply. - Type 5 shows a peak at `Length=10` (83.3%) before declining. 2. **Type-Specific Observations:** - **Type 1:** Steady decline from 100% (Length 0) to 71.3% (Length 19). - **Type 2:** High accuracy (97–99%) until `Length=10`, then gradual decline. - **Type 3:** Moderate decline (100% → 71.3%) with minor fluctuations. - **Type 4:** Sharp drop at `Length=11` (53.3%), lowest among all types. - **Type 5:** Bimodal pattern (low at `Length=7`, peak at `Length=10`). - **Type 6:** Near-perfect accuracy until `Length=18`, then drops to 77.3%. - **Type 7:** Gradual decline (100% → 73.7%) with no sharp drops. --- ## Spatial Grounding - **Legend Placement:** Right edge, spanning full height of heatmap. - **Title Placement:** Centered at the top of the heatmap. - **Axis Alignment:** - X-axis labels centered below columns. - Y-axis labels left-aligned along rows. --- ## Color Consistency Check - **Dark Blue Cells:** Correspond to values ≥90% (e.g., Type 6, Length 0–18). - **Light Blue Cells:** Correspond to values ≤70% (e.g., Type 4, Length 11). - **Mid-Range Blue:** Values between 70–90% (e.g., Type 3, Length 6–10). --- ## Additional Notes - **Missing Data:** - Cells with no values (e.g., Type 1, Length 14–19) are visually empty. - Assumed to represent non-applicable or undefined data points. - **Language:** All text is in English. No non-English content detected. --- ## Summary This heatmap visualizes the accuracy of core generalization across 7 types and 20 lengths for the Qwen-2.5 1.5B model. Accuracy trends show a general decline with increasing length, with notable exceptions for Types 4, 5, and 6. The colorbar provides a clear mapping of accuracy percentages to visual intensity.