## Heatmap: Baseline - Long-to-Short - Qwen-2.5 1.5B ### Overview The image is a heatmap displaying accuracy percentages for different types and lengths. The x-axis represents "Length" and the y-axis represents "Type". The color intensity corresponds to the accuracy percentage, with darker green indicating higher accuracy and lighter shades indicating lower accuracy. ### Components/Axes * **Title:** Baseline - Long-to-Short - Qwen-2.5 1.5B * **Y-axis:** Type (categorical), with values 1, 2, 3, 4, 5, 6, 7 * **X-axis:** Length (numerical), with values 0, 1, 2, 3, 4, 7, 8, 9, 10, 11 * **Colorbar (right side):** Accuracy (%), ranging from 0 to 100, with a gradient from light green to dark green. ### Detailed Analysis or ### Content Details The heatmap presents accuracy values for each combination of "Type" and "Length". Here's a breakdown of the values: * **Type 1:** * Length 0: 0.0% * Length 1: 0.0% * Length 2: 18.7% * Length 3: 28.3% * Length 4: 44.7% * **Type 2:** * Length 0: 69.0% * Length 1: 88.7% * Length 2: 95.7% * Length 3: 90.3% * Length 4: 86.0% * **Type 3:** * Length 0: 0.0% * Length 1: 53.7% * Length 2: 75.0% * Length 3: 81.7% * Length 4: 73.7% * **Type 4:** * Length 0: 47.7% * Length 1: 59.7% * Length 2: 68.7% * Length 3: 67.7% * Length 4: 65.7% * **Type 5:** * Length 7: 46.0% * Length 8: 50.7% * Length 9: 55.3% * Length 10: 63.0% * Length 11: 60.7% * **Type 6:** * Length 0: 0.3% * Length 1: 78.7% * Length 2: 97.0% * Length 3: 96.3% * Length 4: 96.3% * **Type 7:** * Length 0: 0.0% * Length 1: 18.7% * Length 2: 53.7% * Length 3: 73.3% * Length 4: 78.7% ### Key Observations * Types 1, 2, 3, 4, 6, and 7 have data for lengths 0-4. * Type 5 has data for lengths 7-11. * Type 6 shows very high accuracy for lengths 2, 3, and 4. * Types 1, 3, and 7 have 0% accuracy for length 0. * Type 2 shows high accuracy across lengths 0-4. ### Interpretation The heatmap visualizes the performance of the "Qwen-2.5 1.5B" model on a "Long-to-Short" task, broken down by "Type" and "Length". The data suggests that the model's accuracy varies significantly depending on the type of input and its length. Type 6 appears to perform exceptionally well for lengths 2-4, while Types 1, 3, and 7 struggle with inputs of length 0. Type 5 shows a moderate accuracy range for lengths 7-11. The heatmap allows for a quick comparison of the model's performance across different input characteristics, highlighting areas where the model excels and areas where it needs improvement.

\n ## Heatmap: Baseline - Long-to-Short - Qwen-2.5 1.5B ### Overview This image presents a heatmap visualizing the accuracy of a Qwen-2.5 1.5B model in a Long-to-Short task. The heatmap displays accuracy percentages across different 'Type' categories (1 through 7) and varying 'Length' values (0 to 11). The color gradient represents the accuracy, ranging from 0% (lightest shade) to 100% (darkest shade). ### Components/Axes * **Title:** Baseline - Long-to-Short - Qwen-2.5 1.5B * **X-axis:** Length (ranging from 0 to 11, with integer values) * **Y-axis:** Type (ranging from 1 to 7, with integer values) * **Color Scale/Legend:** Located on the right side of the heatmap. It represents Accuracy (%) ranging from 0 to 100, with a gradient from light green to dark green. * **Data Points:** Each cell in the heatmap represents the accuracy for a specific combination of Type and Length. The values are displayed within each cell. ### Detailed Analysis The heatmap contains 7 rows (Types 1-7) and 12 columns (Lengths 0-11). The values within each cell are as follows (approximated to one decimal place): * **Type 1:** * Length 0: 0.0 * Length 1: 0.0 * Length 2: 18.7 * Length 3: 28.3 * Length 4: 44.7 * **Type 2:** * Length 0: 69.0 * Length 1: 88.7 * Length 2: 95.7 * Length 3: 90.3 * Length 4: 86.0 * **Type 3:** * Length 0: 0.0 * Length 1: 53.7 * Length 2: 75.0 * Length 3: 81.7 * Length 4: 73.7 * **Type 4:** * Length 0: 47.7 * Length 1: 59.7 * Length 2: 68.7 * Length 3: 67.7 * Length 4: 65.7 * **Type 5:** * Length 5: 46.0 * Length 6: 50.7 * Length 7: 55.3 * Length 8: 63.0 * Length 9: 60.7 * **Type 6:** * Length 0: 0.3 * Length 1: 78.7 * Length 2: 97.0 * Length 3: 96.3 * Length 4: 96.3 * **Type 7:** * Length 0: 0.0 * Length 1: 18.7 * Length 2: 53.7 * Length 3: 73.3 * Length 4: 78.7 **Trends:** * For Type 1, accuracy increases with length up to length 4. * For Type 2, accuracy is generally high, with a slight decrease at length 4. * For Type 3, accuracy increases with length up to length 3, then decreases slightly at length 4. * For Type 4, accuracy increases slightly with length up to length 2, then plateaus. * For Type 5, accuracy increases with length from 5 to 9. * For Type 6, accuracy is high for lengths 2-4, with a significant jump from length 0 to 1. * For Type 7, accuracy increases with length up to length 4. ### Key Observations * Type 1 consistently exhibits the lowest accuracy values, particularly at shorter lengths (0 and 1). * Type 6 generally shows high accuracy, especially for lengths 2-4. * Type 2 has consistently high accuracy across all lengths. * The model seems to perform better with increasing length for most types, but this trend isn't universal. * There's a noticeable difference in performance between different types, suggesting the model is more sensitive to certain input characteristics. ### Interpretation The heatmap provides a visual representation of the Qwen-2.5 1.5B model's performance on a Long-to-Short task, broken down by 'Type' and 'Length'. The data suggests that the model's accuracy is heavily influenced by both the type of input and its length. The varying performance across types indicates that the model may struggle with certain input characteristics or require more data for those specific types. The general trend of increasing accuracy with length suggests that the model benefits from more context, but this isn't a consistent pattern. The significant differences in accuracy highlight areas where the model could be improved, potentially through further training or architectural modifications. The heatmap is a valuable tool for understanding the model's strengths and weaknesses and guiding future development efforts. The "Baseline" in the title suggests this is a starting point for comparison with other models or configurations.

## Heatmap: Baseline - Long-to-Short - Qwen-2.5 1.5B ### Overview This image is a heatmap visualizing the accuracy (in percentage) of a model named "Qwen-2.5 1.5B" on a "Long-to-Short" task. The performance is broken down by two categorical variables: "Type" (y-axis) and "Length" (x-axis). The color intensity represents accuracy, with a scale from 0% (lightest) to 100% (darkest green). ### Components/Axes * **Title:** "Baseline - Long-to-Short - Qwen-2.5 1.5B" (centered at the top). * **Y-Axis (Vertical):** Labeled "Type". It contains 7 discrete categories, numbered 1 through 7 from top to bottom. * **X-Axis (Horizontal):** Labeled "Length". It contains discrete numerical markers: 0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11. Note that length 6 is absent from the axis. * **Legend/Color Bar:** Located on the right side of the chart. It is a vertical gradient bar labeled "Accuracy (%)". The scale has tick marks at 0, 20, 40, 60, 80, and 100. The color transitions from a very light, almost white green (0%) to a deep, dark forest green (100%). * **Data Cells:** The main chart area is a grid where each cell corresponds to a specific (Type, Length) pair. The cell's background color corresponds to the accuracy value, which is also printed as a number within the cell. Not all (Type, Length) combinations are present; the data is sparse. ### Detailed Analysis The following table reconstructs the data from the heatmap. Empty cells indicate no data point for that (Type, Length) combination. | Type | Length 0 | Length 1 | Length 2 | Length 3 | Length 4 | Length 5 | Length 7 | Length 8 | Length 9 | Length 10 | Length 11 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **1** | 0.0 | 0.0 | 18.7 | 28.3 | 44.7 | | | | | | | | **2** | | 69.0 | 88.7 | 95.7 | 90.3 | 86.0 | | | | | | | **3** | 0.0 | 53.7 | 75.0 | 81.7 | 73.7 | | | | | | | | **4** | | 47.7 | 59.7 | 68.7 | 67.7 | 65.7 | | | | | | | **5** | | | | | | | 46.0 | 50.7 | 55.3 | 63.0 | 60.7 | | **6** | 0.3 | 78.7 | 97.0 | 96.3 | 96.3 | | | | | | | | **7** | 0.0 | 18.7 | 53.7 | 73.3 | 78.7 | | | | | | | **Trend Verification by Type:** * **Type 1:** Accuracy starts at 0.0 for lengths 0-1, then shows a steady upward trend with increasing length (18.7 → 28.3 → 44.7). * **Type 2:** Shows high accuracy overall. It increases sharply from length 1 (69.0) to a peak at length 3 (95.7), then slightly decreases at lengths 4 and 5. * **Type 3:** Starts at 0.0 for length 0, jumps to 53.7 at length 1, peaks at length 3 (81.7), and then dips at length 4. * **Type 4:** Shows a moderate, relatively stable accuracy across lengths 1-5, peaking at length 3 (68.7). * **Type 5:** This type is isolated to longer lengths (7-11). It shows a gradual upward trend from length 7 (46.0) to a peak at length 10 (63.0), with a slight drop at length 11. * **Type 6:** Exhibits very high accuracy. After a near-zero start at length 0 (0.3), it jumps to 78.7 at length 1 and maintains very high values (>96) for lengths 2-4. * **Type 7:** Starts at 0.0 for length 0, then shows a consistent and strong upward trend with increasing length, reaching 78.7 at length 4. ### Key Observations 1. **Performance at Length 0:** Types 1, 3, and 7 have 0.0% accuracy at length 0. Type 6 has a negligible 0.3%. This suggests the model fails completely on these task types when the "Length" parameter is 0. 2. **High-Performing Types:** Type 6 is the strongest performer, achieving near-perfect accuracy (97.0%) at length 2 and maintaining >96% for longer lengths. Type 2 also shows excellent performance, peaking at 95.7%. 3. **Length Specialization:** Type 5 is unique, with data only for lengths 7 through 11. This may indicate a task category inherently associated with longer sequences. 4. **General Trend:** For most types (1, 3, 6, 7), accuracy improves as the "Length" value increases from 0 or 1. Performance often peaks around length 3 or 4 before plateauing or slightly declining. 5. **Sparse Data Grid:** The heatmap is not a complete matrix. The absence of data for certain (Type, Length) pairs (e.g., Type 1 at length 5, Type 2 at length 0) is a significant feature of the dataset. ### Interpretation This heatmap provides a diagnostic view of the Qwen-2.5 1.5B model's capabilities on a "Long-to-Short" task, which likely involves condensing or summarizing information. The "Type" axis probably represents different categories or formats of this task (e.g., summarizing a paragraph vs. extracting a key phrase), while "Length" could refer to the input length, output length, or a complexity parameter. The data suggests the model's performance is highly dependent on both the task type and the length parameter. The complete failure at Length 0 for several types indicates a fundamental limitation or a specific edge case in the model's design or training for those scenarios. The strong performance of Types 2 and 6 identifies them as areas of relative strength. The isolated data for Type 5 hints at a specialized subset of the task. Overall, the chart reveals that the model is not uniformly proficient. Its accuracy is contingent on the specific combination of task type and length, with clear patterns of strength (high accuracy at moderate lengths for certain types) and weakness (failure at minimal lengths). This information would be crucial for developers to understand the model's boundaries and guide further fine-tuning or evaluation.

# Technical Document Extraction: Baseline - Long-to-Short - Qwen-2.5 1.5B ## Chart Type Heatmap visualization of accuracy percentages across different model configurations. ## Axes and Labels - **X-axis (Horizontal):** "Length" (0 to 11) - **Y-axis (Vertical):** "Type" (1 to 7) - **Color Scale:** Accuracy (%) from 0% (lightest green) to 100% (darkest green) - **Legend:** Located on the right side of the chart, showing the gradient from light to dark green corresponding to accuracy percentages. ## Data Structure The heatmap represents a 7x12 matrix (Types 1-7 vs Lengths 0-11). Key observations: 1. **Highest Accuracy:** - Type 2, Length 3: 95.7% - Type 6, Length 4: 96.3% 2. **Lowest Accuracy:** - Type 1, Length 0: 0.0% - Type 3, Length 0: 0.0% 3. **Notable Trends:** - Accuracy generally increases with Length for most Types until reaching a peak, then declines - Type 2 shows the most consistent high performance (86.0-95.7% range) - Type 6 demonstrates peak performance at mid-lengths (78.7-96.3%) - Type 7 shows gradual improvement with Length (18.7-78.7%) ## Data Table Reconstruction | Type | Length | Accuracy (%) | |------|--------|--------------| | 1 | 0 | 0.0 | | 1 | 1 | 0.0 | | 1 | 2 | 18.7 | | 1 | 3 | 28.3 | | 1 | 4 | 44.7 | | 2 | 1 | 69.0 | | 2 | 2 | 88.7 | | 2 | 3 | 95.7 | | 2 | 4 | 90.3 | | 2 | 5 | 86.0 | | 3 | 1 | 53.7 | | 3 | 2 | 75.0 | | 3 | 3 | 81.7 | | 3 | 4 | 73.7 | | 4 | 1 | 47.7 | | 4 | 2 | 59.7 | | 4 | 3 | 68.7 | | 4 | 4 | 67.7 | | 4 | 5 | 65.7 | | 5 | 7 | 46.0 | | 5 | 8 | 50.7 | | 5 | 9 | 55.3 | | 5 | 10 | 63.0 | | 5 | 11 | 60.7 | | 6 | 1 | 78.7 | | 6 | 2 | 97.0 | | 6 | 3 | 96.3 | | 6 | 4 | 96.3 | | 7 | 1 | 18.7 | | 7 | 2 | 53.7 | | 7 | 3 | 73.3 | | 7 | 4 | 78.7 | ## Color Legend Verification - All data points match the legend's color gradient: - Light green (0-20%): Type 1, Length 0-1 - Medium green (20-60%): Type 1, Length 2-4; Type 5, Length 7-11 - Dark green (60-100%): All other data points ## Spatial Grounding - Legend position: Right side of chart - Data point verification: Type 6, Length 4 (96.3%) matches darkest green in legend ## Trend Verification 1. **Type 2:** Peaks at Length 3 (95.7%), then declines 2. **Type 6:** Maintains high accuracy (96.3%) across Lengths 2-4 3. **Type 5:** Shows gradual improvement from 46.0% (Length 7) to 63.0% (Length 10) 4. **Type 7:** Steady increase from 18.7% (Length 1) to 78.7% (Length 4) ## Language Note All text appears in English. No non-English content detected.