## Heatmap: Baseline - Long-to-Short - Qwen-2.5 7B ### Overview The image is a heatmap displaying accuracy percentages for a model named "Qwen-2.5 7B" under "Baseline - Long-to-Short" conditions. The heatmap visualizes the relationship between "Type" (categorical, 7 types) and "Length" (numerical, ranging from 0 to 11). The color intensity represents the accuracy percentage, with darker green indicating higher accuracy and lighter green indicating lower accuracy. ### Components/Axes * **Title:** Baseline - Long-to-Short - Qwen-2.5 7B * **X-axis:** Length, with values ranging from 0 to 11. * **Y-axis:** Type, with values ranging from 1 to 7. * **Color Bar (Right):** Accuracy (%), ranging from 0 to 100. The color gradient goes from light green (0%) to dark green (100%). ### Detailed Analysis The heatmap presents accuracy values for each combination of "Type" and "Length." The values are as follows: * **Type 1:** * Length 0: 0.0% * Length 1: 1.7% * Length 2: 25.7% * Length 3: 51.7% * Length 4: 73.3% * **Type 2:** * Length 1: 71.0% * Length 2: 94.3% * Length 3: 98.7% * Length 4: 98.7% * Length 5: 97.0% * **Type 3:** * Length 0: 16.7% * Length 1: 88.7% * Length 2: 94.7% * Length 3: 94.7% * Length 4: 94.3% * **Type 4:** * Length 0: 57.3% * Length 1: 72.0% * Length 2: 81.7% * Length 3: 88.3% * Length 4: 89.0% * **Type 5:** * Length 7: 84.0% * Length 8: 89.0% * Length 9: 85.7% * Length 10: 92.0% * Length 11: 93.7% * **Type 6:** * Length 0: 16.3% * Length 1: 98.3% * Length 2: 99.3% * Length 3: 99.7% * Length 4: 99.0% * **Type 7:** * Length 0: 0.0% * Length 1: 24.0% * Length 2: 56.0% * Length 3: 72.0% * Length 4: 89.3% ### Key Observations * Types 2, 3, and 6 generally exhibit high accuracy (above 70%) for the lengths they cover. * Types 1 and 7 show lower accuracy, especially at shorter lengths. * Type 5 only has data for lengths 7-11, with relatively high accuracy. * Accuracy tends to increase with length for Types 1, 4, and 7, at least up to length 4. ### Interpretation The heatmap provides a visual representation of the model's performance across different "Types" and "Lengths." The data suggests that the model performs better on certain types and longer lengths. The lower accuracy for Types 1 and 7 at shorter lengths could indicate a weakness in handling those specific types of inputs when the input sequence is short. The high accuracy for Types 2, 3, and 6 suggests that the model is well-suited for those types of inputs. The data for Type 5, which only covers longer lengths, indicates that the model maintains good accuracy for those lengths. The increasing accuracy with length for some types suggests that the model benefits from longer input sequences for those specific types.

## Heatmap: Baseline - Long-to-Short - Qwen-2.5 7B ### Overview This image presents a heatmap visualizing the accuracy of a "Long-to-Short" baseline model, specifically "Qwen-2.5 7B", across different "Type" and "Length" combinations. The heatmap uses a color gradient to represent accuracy, ranging from approximately 0% (lightest color) to 100% (darkest color). ### Components/Axes * **Title:** "Baseline - Long-to-Short - Qwen-2.5 7B" (positioned at the top-center) * **X-axis:** "Length" - Values range from 0 to 11, with markers at each integer value. * **Y-axis:** "Type" - Values range from 1 to 7, with markers at each integer value. * **Color Scale/Legend:** Located on the right side of the heatmap. It represents "Accuracy (%)", ranging from 0 to 100, with a gradient from light green to dark green. * **Data Points:** Each cell in the heatmap represents the accuracy for a specific combination of "Type" and "Length". The accuracy value is displayed within each cell. ### Detailed Analysis The heatmap displays accuracy values for 7 types and lengths ranging from 0 to 11. The color intensity corresponds to the accuracy percentage, as indicated by the legend. Here's a breakdown of the data, reading row by row (Type 1 to Type 7): * **Type 1:** Accuracy increases with length. Values are approximately: 0.0 at Length 0, 1.7 at Length 1, 25.7 at Length 2, 51.7 at Length 3, 73.3 at Length 4. * **Type 2:** Accuracy is generally high and increases with length. Values are approximately: 71.0 at Length 0, 94.3 at Length 1, 98.7 at Length 2, 98.7 at Length 3, 97.0 at Length 4. * **Type 3:** Accuracy increases with length. Values are approximately: 16.7 at Length 0, 88.7 at Length 1, 94.7 at Length 2, 94.7 at Length 3, 94.3 at Length 4. * **Type 4:** Accuracy increases with length. Values are approximately: 57.3 at Length 0, 72.0 at Length 1, 81.7 at Length 2, 88.3 at Length 3, 89.0 at Length 4. * **Type 5:** Accuracy starts at a lower value and increases significantly with length. Values are approximately: 84.0 at Length 7, 89.0 at Length 8, 85.7 at Length 9, 92.0 at Length 10, 93.7 at Length 11. * **Type 6:** Accuracy is high and increases with length. Values are approximately: 16.3 at Length 0, 98.3 at Length 1, 99.3 at Length 2, 99.7 at Length 3, 99.0 at Length 4. * **Type 7:** Accuracy increases with length. Values are approximately: 0.0 at Length 0, 24.0 at Length 1, 56.0 at Length 2, 72.0 at Length 3, 89.3 at Length 4. ### Key Observations * For most types (1, 2, 3, 4, 6, 7), accuracy generally increases as the length increases. * Type 5 shows a delayed increase in accuracy, starting at a lower value and then increasing significantly for lengths 7-11. * Type 2 consistently exhibits the highest accuracy across all lengths. * Type 1 and Type 7 start with very low accuracy at length 0. ### Interpretation The heatmap demonstrates the performance of the Qwen-2.5 7B model on a "Long-to-Short" task, broken down by "Type" and "Length". The consistent positive correlation between length and accuracy for most types suggests that the model performs better when processing longer inputs. The variation in accuracy across different types indicates that the model's performance is sensitive to the specific characteristics of the input data represented by each "Type". The relatively low accuracy for Type 1 and Type 7 at shorter lengths suggests that these types may require longer input sequences to achieve optimal performance. The delayed increase in accuracy for Type 5 could indicate a specific challenge associated with this type that requires a certain input length to overcome. Overall, the heatmap provides a valuable visualization of the model's strengths and weaknesses, allowing for targeted improvements and optimizations.

## Heatmap: Baseline - Long-to-Short - Qwen-2.5 7B ### Overview This image is a heatmap visualizing the accuracy (in percentage) of a model named "Qwen-2.5 7B" on a "Long-to-Short" task. The performance is broken down by two categorical variables: "Type" (y-axis) and "Length" (x-axis). The color intensity represents accuracy, with a scale from light green (0%) to dark green (100%). ### Components/Axes * **Title:** "Baseline - Long-to-Short - Qwen-2.5 7B" (centered at the top). * **Y-Axis (Vertical):** Labeled "Type". It lists 7 distinct categories, numbered 1 through 7. * **X-Axis (Horizontal):** Labeled "Length". It lists discrete numerical values: 0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11. Note the gap between 5 and 7. * **Color Bar/Legend:** Located on the right side. It is a vertical gradient bar labeled "Accuracy (%)". The scale runs from 0 (lightest green) at the bottom to 100 (darkest green) at the top, with tick marks at 20, 40, 60, and 80. * **Data Cells:** The main body of the chart is a grid where each cell corresponds to a specific (Type, Length) pair. The cell's background color corresponds to the accuracy value, which is also printed as a number within the cell. ### Detailed Analysis The following table reconstructs the data from the heatmap. Empty cells indicate no data point for that (Type, Length) combination. | Type | Length 0 | Length 1 | Length 2 | Length 3 | Length 4 | Length 5 | Length 7 | Length 8 | Length 9 | Length 10 | Length 11 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **1** | 0.0 | 1.7 | 25.7 | 51.7 | 73.3 | | | | | | | | **2** | | 71.0 | 94.3 | 98.7 | 98.7 | 97.0 | | | | | | | **3** | 16.7 | 88.7 | 94.7 | 94.7 | 94.3 | | | | | | | | **4** | | 57.3 | 72.0 | 81.7 | 88.3 | 89.0 | | | | | | | **5** | | | | | | | 84.0 | 89.0 | 85.7 | 92.0 | 93.7 | | **6** | 16.3 | 98.3 | 99.3 | 99.7 | 99.0 | | | | | | | | **7** | 0.0 | 24.0 | 56.0 | 72.0 | 89.3 | | | | | | | **Trend Verification per Type:** * **Type 1:** Shows a strong, steady upward trend. Accuracy starts at 0.0% (Length 0) and increases monotonically to 73.3% (Length 4). * **Type 2:** Starts high (71.0% at Length 1), peaks at 98.7% (Lengths 3 & 4), and shows a very slight decrease to 97.0% at Length 5. * **Type 3:** Jumps dramatically from 16.7% (Length 0) to 88.7% (Length 1), then plateaus in the mid-90s. * **Type 4:** Exhibits a consistent upward trend from 57.3% (Length 1) to 89.0% (Length 5). * **Type 5:** Data exists only for longer lengths (7-11). Accuracy fluctuates between 84.0% and 93.7%, with a general upward trend from Length 7 to 11. * **Type 6:** Starts low (16.3% at Length 0) but immediately jumps to near-perfect accuracy (98.3% at Length 1) and remains above 99% for Lengths 2-4. * **Type 7:** Mirrors the trend of Type 1, starting at 0.0% (Length 0) and increasing steadily to 89.3% (Length 4). ### Key Observations 1. **Length-Dependent Performance:** For Types 1, 4, and 7, accuracy improves significantly and consistently as the "Length" value increases. 2. **High Baseline Performance:** Types 2, 3, and 6 achieve very high accuracy (>88%) starting from relatively short lengths (Length 1 or 2). 3. **Outlier - Type 5:** This type has no data for lengths 0-6, suggesting it may represent a different category of task or input that only applies to longer sequences. Its performance is consistently high within its range. 4. **Near-Perfect Accuracy:** Type 6 at Lengths 2, 3, and 4 shows accuracy values of 99.3%, 99.7%, and 99.0%, indicating near-perfect performance for those conditions. 5. **Zero Accuracy Points:** Types 1 and 7 both have an accuracy of 0.0% at Length 0, indicating complete failure for that specific condition. ### Interpretation This heatmap provides a granular view of the Qwen-2.5 7B model's performance on a "Long-to-Short" task, revealing that its effectiveness is highly dependent on both the task "Type" and the input "Length." * **Task Difficulty Spectrum:** The "Type" axis likely represents different sub-tasks or problem categories. The data suggests a spectrum of difficulty: Types 1 and 7 appear to be the most challenging at short lengths, requiring longer inputs to achieve decent accuracy. In contrast, Types 2, 3, and 6 seem to be easier or better-suited to the model, as they yield high accuracy even with short inputs. * **The "Long-to-Short" Mechanism:** The general trend of improving accuracy with increasing length for most types supports the premise of a "Long-to-Short" process—perhaps the model uses longer context or reasoning chains to generate a correct short answer. The plateau or slight dip for Type 2 at Length 5 might indicate a point of diminishing returns or a minor failure mode. * **Model Specialization:** The exceptional performance of Type 6 (near 100% accuracy) suggests the model is particularly adept at that specific type of task. Conversely, the 0% accuracy for Types 1 and 7 at Length 0 highlights a critical failure case for very short inputs of those types. * **Data Gaps:** The absence of data for Type 5 at shorter lengths and for all types at lengths 6 and beyond (except Type 5) is notable. It implies the evaluation was either not performed for those combinations or that those combinations are not applicable, which is important context for understanding the model's full capabilities. In summary, the chart demonstrates that the Qwen-2.5 7B model's accuracy on this task is not uniform but is a complex function of task type and input length, with clear patterns of strength and weakness across different conditions.

# Technical Document Extraction: Heatmap Analysis ## Title **Baseline - Long-to-Short - Qwen-2.5 7B** --- ## Axis Labels - **X-axis (Horizontal):** `Length` (values: 0 to 11) - **Y-axis (Vertical):** `Type` (values: 0 to 7) - **Colorbar (Right):** `Accuracy (%)` (range: 0% to 100%) --- ## Data Structure The heatmap represents accuracy percentages for combinations of `Type` (rows) and `Length` (columns). Each cell contains a numerical value corresponding to accuracy. ### Reconstructed Data Table | Type \ Length | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | |---------------|------|------|------|------|------|------|------|------|------|------|------|------| | **0** | 0.0 | 1.7 | 25.7 | 51.7 | 73.3 | | | | | | | | | **1** | | 71.0 | 94.3 | 98.7 | 98.7 | 97.0 | | | | | | | | **2** | 16.7 | 88.7 | 94.7 | 94.7 | 94.3 | | | | | | | | | **3** | | 57.3 | 72.0 | 81.7 | 88.3 | 89.0 | | | | | | | | **4** | | | | | | | 84.0 | 89.0 | 85.7 | 92.0 | 93.7 | | **5** | | | | | | | | | | | | | | **6** | 16.3 | 98.3 | 99.3 | 99.7 | 99.0 | | | | | | | | | **7** | 0.0 | 24.0 | 56.0 | 72.0 | 89.3 | | | | | | | | --- ## Key Trends 1. **General Pattern:** Accuracy increases with `Length` for most `Type` values, peaking around `Length = 4` before plateauing or declining. 2. **High Accuracy:** - `Type 1` and `Type 6` achieve near-perfect accuracy (97–99%) for `Length ≥ 2`. - `Type 4` and `Type 7` show moderate accuracy (70–90%) across most `Length` values. 3. **Low Accuracy:** - `Type 0` and `Type 7` start with near-zero accuracy at `Length = 0`, improving sharply with increasing `Length`. - `Type 5` has no data for `Length ≤ 6`. --- ## Color Legend Verification - **Lightest Green (0–20%):** Matches `Type 0, Length 0` (0.0%) and `Type 7, Length 0` (0.0%). - **Medium Green (40–60%):** Matches `Type 0, Length 2` (25.7%) and `Type 3, Length 2` (72.0%). - **Dark Green (80–100%):** Matches `Type 1, Length 3` (98.7%) and `Type 6, Length 3` (99.7%). --- ## Spatial Grounding - **Legend Position:** Right side of the heatmap. - **Data Point Alignment:** All cell colors strictly correspond to the colorbar's accuracy scale. --- ## Component Isolation 1. **Header:** Title (`Baseline - Long-to-Short - Qwen-2.5 7B`). 2. **Main Chart:** 8x12 heatmap with labeled axes and embedded numerical values. 3. **Footer:** Colorbar (`Accuracy (%)` from 0% to 100%). --- ## Notes - Missing values (e.g., `Type 5, Length ≤ 6`) are represented as empty cells. - No non-English text is present in the image.