## Heatmap: Few-shot - Core Generalization - GPT-4o ### Overview The image is a heatmap visualizing the accuracy (%) of a model (GPT-4o) across different 'Type' categories (1 to 7) and 'Length' values (0 to 19). The color intensity represents the accuracy, with darker blue indicating higher accuracy and lighter shades indicating lower accuracy. ### Components/Axes * **Title:** Few-shot - Core Generalization - GPT-4o * **Y-axis:** "Type" with categories labeled 1 to 7. * **X-axis:** "Length" with values from 0 to 19. * **Color Legend:** Located on the right side of the heatmap, showing a gradient from light blue (0%) to dark blue (100%) representing "Accuracy (%)". ### Detailed Analysis The heatmap displays accuracy values for each combination of 'Type' and 'Length'. Here's a breakdown of the accuracy values for each 'Type' across different 'Length' values: * **Type 1:** * Length 0: 64.0% * Length 1: 33.0% * Length 2: 27.0% * Length 3: 21.0% * Length 4: 22.0% * Length 5: 23.0% * Length 6: 16.0% * Length 7: 28.0% * Length 8: 27.0% * Length 9: 30.0% * **Type 2:** * Length 1: 73.0% * Length 2: 89.0% * Length 3: 91.0% * Length 4: 86.0% * Length 5: 84.0% * Length 6: 81.0% * Length 7: 78.0% * Length 8: 74.0% * Length 9: 63.0% * Length 10: 66.0% * **Type 3:** * Length 0: 42.0% * Length 1: 53.0% * Length 2: 46.0% * Length 3: 44.0% * Length 4: 35.0% * Length 5: 18.0% * Length 6: 16.0% * Length 7: 25.0% * Length 8: 20.0% * Length 9: 18.0% * Length 10: 13.0% * Length 11: 17.0% * Length 12: 17.0% * Length 13: 18.0% * Length 14: 13.0% * Length 15: 17.0% * Length 16: 11.0% * Length 17: 14.0% * Length 18: 10.0% * Length 19: 11.0% * **Type 4:** * Length 1: 68.0% * Length 2: 67.0% * Length 3: 64.0% * Length 4: 47.0% * Length 5: 45.0% * Length 6: 29.0% * Length 7: 30.0% * Length 8: 37.0% * Length 9: 40.0% * Length 10: 41.0% * Length 11: 35.0% * **Type 5:** * Length 7: 11.0% * Length 8: 25.0% * Length 9: 21.0% * Length 10: 18.0% * Length 11: 17.0% * Length 12: 25.0% * Length 13: 20.0% * Length 14: 25.0% * Length 15: 15.0% * Length 16: 24.0% * Length 17: 26.0% * Length 18: 20.0% * Length 19: 27.0% * **Type 6:** * Length 0: 89.0% * Length 1: 75.0% * Length 2: 66.0% * Length 3: 54.0% * Length 4: 51.0% * Length 5: 48.0% * Length 6: 44.0% * Length 7: 49.0% * Length 8: 42.0% * Length 9: 52.0% * Length 10: 46.0% * Length 11: 51.0% * Length 12: 40.0% * Length 13: 44.0% * Length 14: 32.0% * Length 15: 37.0% * Length 16: 38.0% * Length 17: 32.0% * Length 18: 39.0% * **Type 7:** * Length 0: 91.0% * Length 1: 76.0% * Length 2: 63.0% * Length 3: 53.0% * Length 4: 41.0% * Length 5: 36.0% * Length 6: 34.0% * Length 7: 33.0% * Length 8: 39.0% * Length 9: 26.0% * Length 10: 33.0% * Length 11: 34.0% * Length 12: 32.0% * Length 13: 26.0% ### Key Observations * Types 2, 6, and 7 generally exhibit higher accuracy compared to other types. * Accuracy tends to decrease as 'Length' increases for most 'Type' categories. * Type 5 has the lowest accuracy values overall. * Types 1, 3, 5 have missing data for certain 'Length' values. ### Interpretation The heatmap provides insights into the performance of the GPT-4o model in a few-shot learning scenario. The "Type" likely represents different categories or types of tasks, and "Length" could refer to the length of the input sequence or the number of examples provided in the few-shot setting. The data suggests that the model performs better on certain types of tasks (Types 2, 6, and 7) compared to others (Type 5). The decreasing accuracy with increasing "Length" could indicate that the model struggles with longer sequences or requires more examples to maintain performance. The missing data for some 'Type' and 'Length' combinations might indicate that those specific scenarios were not tested or are not applicable. Overall, the heatmap highlights the strengths and weaknesses of the GPT-4o model in few-shot core generalization, providing valuable information for further analysis and improvement.

## Heatmap: Few-shot - Core Generalization - GPT-4o ### Overview This heatmap visualizes the accuracy of GPT-4o across different 'Type' categories and 'Length' values. The color intensity represents the accuracy percentage, ranging from 0% (white) to 100% (dark blue). The heatmap is designed to show how well the model generalizes based on the length of the input and the type of task. ### Components/Axes * **Title:** "Few-shot - Core Generalization - GPT-4o" (Top-center) * **X-axis:** "Length" - Values range from 0 to 19, in integer increments. (Bottom) * **Y-axis:** "Type" - Categories are: '1', '2', '3', '4', '5', '6', '7' (Left) * **Color Scale:** A gradient from white (0%) to dark blue (100%) representing accuracy. The scale is positioned on the right side of the heatmap, with numerical values indicating the corresponding accuracy percentage. ### Detailed Analysis The heatmap displays accuracy values for each combination of 'Type' and 'Length'. I will analyze each 'Type' row and describe the trends. * **Type 1:** Accuracy starts at approximately 91.0% (Length 0), decreases to around 41.0% (Length 4), then increases slightly to 36.0% (Length 5), and remains relatively stable around 33.0-39.0% for Lengths 6-19. * **Type 2:** Accuracy begins at 89.0% (Length 0), decreases to 51.0% (Length 4), then increases to 48.0% (Length 5), and remains relatively stable around 40.0-49.0% for Lengths 6-19. * **Type 3:** Accuracy starts at 85.0% (Length 0), decreases to 46.0% (Length 4), then increases to 44.0% (Length 5), and remains relatively stable around 32.0-42.0% for Lengths 6-19. * **Type 4:** Accuracy begins at 68.0% (Length 0), decreases to 47.0% (Length 4), then increases to 45.0% (Length 5), and remains relatively stable around 30.0-40.0% for Lengths 6-19. * **Type 5:** Accuracy starts at 11.0% (Length 6), increases to 25.0% (Length 7), then remains relatively stable around 17.0-25.0% for Lengths 8-19. * **Type 6:** Accuracy starts at 64.0% (Length 0), decreases to 27.0% (Length 4), then increases to 23.0% (Length 5), and remains relatively stable around 16.0-30.0% for Lengths 6-19. * **Type 7:** Accuracy begins at 75.0% (Length 0), decreases to 41.0% (Length 4), then increases to 36.0% (Length 5), and remains relatively stable around 32.0-39.0% for Lengths 6-19. Here's a table summarizing some key data points: | Type | Length 0 | Length 4 | Length 5 | Length 10 | Length 19 | |---|---|---|---|---|---| | 1 | 91.0% | 41.0% | 36.0% | 33.0% | 36.0% | | 2 | 89.0% | 51.0% | 48.0% | 42.0% | 44.0% | | 3 | 85.0% | 46.0% | 44.0% | 40.0% | 39.0% | | 4 | 68.0% | 47.0% | 45.0% | 37.0% | 35.0% | | 5 | N/A | N/A | N/A | 21.0% | 20.0% | | 6 | 64.0% | 27.0% | 23.0% | 26.0% | 27.0% | | 7 | 75.0% | 41.0% | 36.0% | 33.0% | 32.0% | ### Key Observations * Accuracy generally decreases as 'Length' increases from 0 to 4 for most 'Type' categories. * After Length 4, the accuracy tends to plateau or fluctuate within a relatively narrow range. * Type 5 has significantly lower accuracy values compared to other types, and only has data starting at Length 6. * Type 1 and Type 2 consistently exhibit the highest accuracy values across most lengths. * There is a noticeable dip in accuracy for all types between Length 0 and Length 4. ### Interpretation The heatmap suggests that GPT-4o performs best on shorter inputs (Length 0) for most task types. As the input length increases, the model's accuracy tends to decline, indicating a potential challenge in maintaining performance with longer sequences. The plateauing of accuracy after Length 4 suggests that the model reaches a limit in its ability to effectively process and generalize from longer inputs. The lower accuracy of Type 5 could indicate that this task type is inherently more difficult for the model, or that the few-shot learning setup is less effective for this type. The consistent high performance of Types 1 and 2 suggests these task types are well-suited to the model's capabilities. The initial drop in accuracy as length increases could be due to the model struggling to maintain context or identify relevant information within longer sequences. This data is valuable for understanding the model's strengths and weaknesses and for guiding the development of strategies to improve its generalization performance on longer and more complex inputs.

## Heatmap: Few-shot - Core Generalization - GPT-4o ### Overview This image is a heatmap visualizing the accuracy (in percentage) of the GPT-4o model on "Few-shot - Core Generalization" tasks. The performance is broken down by two categorical variables: "Type" (vertical axis, rows 1-7) and "Length" (horizontal axis, columns 0-19). The color intensity represents accuracy, with a scale from 0% (lightest blue/white) to 100% (darkest blue). The chart contains numerical data points within each cell, and some cells are empty, indicating no data for that specific Type-Length combination. ### Components/Axes * **Title:** "Few-shot - Core Generalization - GPT-4o" (Top Center). * **Vertical Axis (Y-axis):** Labeled "Type". Categories are numbered 1 through 7 from top to bottom. * **Horizontal Axis (X-axis):** Labeled "Length". Categories are numbered 0 through 19 from left to right. * **Color Bar/Legend:** Located on the right side. It is a vertical gradient bar labeled "Accuracy (%)". The scale runs from 0 at the bottom to 100 at the top, with tick marks at 0, 20, 40, 60, 80, and 100. Darker blue corresponds to higher accuracy. * **Data Cells:** Each cell at the intersection of a Type and Length contains a numerical value representing the accuracy percentage. The background color of the cell corresponds to this value per the color bar. ### Detailed Analysis The following table reconstructs the data from the heatmap. An empty cell indicates no data was recorded for that Type-Length pair. | Type | Length 0 | Length 1 | Length 2 | Length 3 | Length 4 | Length 5 | Length 6 | Length 7 | Length 8 | Length 9 | Length 10 | Length 11 | Length 12 | Length 13 | Length 14 | Length 15 | Length 16 | Length 17 | Length 18 | Length 19 | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **1** | 64.0 | 33.0 | 27.0 | 21.0 | 22.0 | 23.0 | 16.0 | 28.0 | 27.0 | 30.0 | | | | | | | | | | | | **2** | | 73.0 | 89.0 | 91.0 | 86.0 | 84.0 | 81.0 | 78.0 | 74.0 | 63.0 | 66.0 | | | | | | | | | | | **3** | 42.0 | 53.0 | 46.0 | 44.0 | 35.0 | 18.0 | 16.0 | 25.0 | 20.0 | 18.0 | 13.0 | 17.0 | 17.0 | 18.0 | 13.0 | 17.0 | 11.0 | 14.0 | 10.0 | 11.0 | | **4** | | 68.0 | 67.0 | 64.0 | 47.0 | 45.0 | 29.0 | 30.0 | 37.0 | 40.0 | 41.0 | 35.0 | | | | | | | | | | **5** | | | | | | | | 11.0 | 25.0 | 21.0 | 18.0 | 17.0 | 25.0 | 20.0 | 25.0 | 15.0 | 24.0 | 26.0 | 20.0 | 27.0 | | **6** | 89.0 | 75.0 | 66.0 | 54.0 | 51.0 | 48.0 | 44.0 | 49.0 | 42.0 | 52.0 | 46.0 | 51.0 | 40.0 | 44.0 | 32.0 | 37.0 | 38.0 | 32.0 | 39.0 | | | **7** | 91.0 | 76.0 | 63.0 | 53.0 | 41.0 | 36.0 | 34.0 | 33.0 | 39.0 | 26.0 | 33.0 | 34.0 | 32.0 | 26.0 | | | | | | | **Trend Verification by Type:** * **Type 1:** Starts at a moderate 64.0% (Length 0) and shows a general downward trend with fluctuations, ending at 30.0% (Length 9). The line slopes downward overall. * **Type 2:** Begins at 73.0% (Length 1), peaks at 91.0% (Length 3), then follows a steady downward trend to 66.0% (Length 10). The line has an initial rise followed by a decline. * **Type 3:** Starts at 42.0% (Length 0), has a brief rise to 53.0% (Length 1), then exhibits a consistent downward trend across all lengths, reaching a low of 10.0% (Length 18). The line slopes downward. * **Type 4:** Starts at 68.0% (Length 1) and shows a general downward trend with a slight mid-range recovery, ending at 35.0% (Length 11). The line slopes downward. * **Type 5:** Data begins at Length 7 (11.0%). The trend is relatively flat and low, fluctuating between 11.0% and 27.0% with no strong directional slope. * **Type 6:** Starts very high at 89.0% (Length 0) and follows a clear downward trend with some volatility, ending at 39.0% (Length 18). The line slopes downward. * **Type 7:** Starts at the highest observed value of 91.0% (Length 0) and shows a strong, consistent downward trend, ending at 26.0% (Length 13). The line slopes downward. ### Key Observations 1. **Performance Decay with Length:** For most Types (1, 2, 3, 4, 6, 7), there is a clear negative correlation between Length and Accuracy. As the Length increases, the model's accuracy generally decreases. 2. **High Initial Performance:** Types 2, 6, and 7 show very high accuracy (>89%) at the shortest measured lengths (Lengths 0-3). 3. **Low-Performance Cluster:** Type 5 and the latter half of Type 3 consistently show low accuracy, mostly below 30%. 4. **Data Sparsity:** The heatmap is not fully populated. Type 1 has no data beyond Length 9. Type 2 has no data at Length 0 or beyond Length 10. Type 4 has no data at Length 0 or beyond Length 11. Type 5 has no data before Length 7. Type 6 has no data at Length 19. Type 7 has no data beyond Length 13. 5. **Peak Accuracy:** The single highest accuracy value is 91.0%, achieved by both Type 2 (at Length 3) and Type 7 (at Length 0). ### Interpretation This heatmap demonstrates that GPT-4o's ability to generalize in few-shot scenarios is highly dependent on both the specific "Type" of task and the "Length" parameter (which could represent sequence length, number of examples, or another complexity metric). * **Core Finding:** The dominant trend is that performance degrades as Length increases. This suggests the model's core generalization capability is sensitive to scale or complexity; it performs best on shorter, presumably simpler, instances of a task type. * **Task-Type Variability:** The significant difference in starting accuracy and decay rates between Types (e.g., Type 7 starting at 91% vs. Type 3 starting at 42%) indicates that some core generalization tasks are inherently easier for the model than others. * **Practical Implication:** For applications relying on few-shot generalization, this data suggests that keeping the "Length" parameter low is crucial for maintaining high accuracy. The model may require different prompting strategies or fine-tuning for task types that show poor performance even at short lengths (like Type 3 and Type 5). * **Anomaly:** Type 5's data starts only at Length 7 and shows a flat, low-accuracy trend. This could indicate a different experimental setup for this type or a category where the model fails to generalize until a certain length threshold is met, after which it performs poorly but consistently.

# Technical Document Extraction: Few-shot - Core Generalization - GPT-4o ## 1. Labels and Axis Titles - **Title**: "Few-shot - Core Generalization - GPT-4o" - **X-axis**: "Length" (values: 0 to 19) - **Y-axis**: "Type" (values: 1 to 7) - **Colorbar**: "Accuracy (%)" (range: 0% to 100%) ## 2. Data Table Structure The heatmap represents accuracy percentages for different combinations of **Type** (rows) and **Length** (columns). Below is the reconstructed table: | Type \ Length | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | |---------------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------| | 1 | 64.0 | 33.0 | 27.0 | 21.0 | 22.0 | 23.0 | 16.0 | 28.0 | 27.0 | 30.0 | | | | | | | | | | | | 2 | | 73.0 | 89.0 | 91.0 | 86.0 | 84.0 | 81.0 | 78.0 | 74.0 | 63.0 | 66.0 | | | | | | | | | | | 3 | 42.0 | 53.0 | 46.0 | 44.0 | 35.0 | 18.0 | 16.0 | 25.0 | 20.0 | 18.0 | 13.0 | 17.0 | 17.0 | 18.0 | 13.0 | 17.0 | 11.0 | 14.0 | 10.0 | 11.0 | | 4 | 68.0 | 67.0 | | 64.0 | 47.0 | 45.0 | 29.0 | 30.0 | 37.0 | 40.0 | 41.0 | 35.0 | | | | | | | | | | 5 | | | | | | | | 11.0 | 25.0 | 21.0 | 18.0 | 17.0 | 25.0 | 20.0 | 25.0 | 15.0 | 24.0 | 26.0 | 20.0 | 27.0 | | 6 | 89.0 | 75.0 | 66.0 | 54.0 | 51.0 | 48.0 | 44.0 | 49.0 | 42.0 | 52.0 | 46.0 | 51.0 | 40.0 | 44.0 | 32.0 | 37.0 | 38.0 | 32.0 | 39.0 | | | 7 | 91.0 | 76.0 | 63.0 | 53.0 | 41.0 | 36.0 | 34.0 | 33.0 | 39.0 | 26.0 | 33.0 | 34.0 | 32.0 | 26.0 | | | | | | | ## 3. Key Trends and Observations - **Type 1**: Accuracy declines sharply with increasing Length (64.0% at Length 0 → 30.0% at Length 9). - **Type 2**: Peaks at Length 2 (89.0%) and declines steadily after Length 7 (66.0% at Length 10). - **Type 3**: Highest accuracy at Length 0 (42.0%), with significant drops at Lengths 5–19 (11.0% at Length 19). - **Type 4**: Moderate accuracy across Lengths 0–9 (40.0% at Length 9), with no data beyond Length 11. - **Type 5**: Low accuracy overall (11.0–27.0%), with no data for Lengths 0–4. - **Type 6**: High accuracy at Length 0 (89.0%), declining to 32.0% at Length 16. - **Type 7**: Highest accuracy at Length 0 (91.0%), with gradual declines to 26.0% at Length 13. ## 4. Legend and Color Mapping - **Colorbar**: Located on the right side of the heatmap. - **Color Gradient**: - Light blue: Low accuracy (0–20%) - Dark blue: High accuracy (80–100%) - **Example**: - Type 7, Length 0 (91.0%) is dark blue. - Type 5, Length 19 (27.0%) is light blue. ## 5. Spatial Grounding - **Legend Position**: Right side of the heatmap. - **Data Point Verification**: - Type 2, Length 2 (89.0%) matches dark blue. - Type 3, Length 19 (11.0%) matches light blue. ## 6. Missing Data - **Type 5**: No data for Lengths 0–4. - **Type 4**: No data for Lengths 12–19. - **Type 7**: No data for Lengths 14–19. ## 7. Summary The heatmap illustrates how accuracy varies with **Type** and **Length** for GPT-4o's few-shot core generalization. High accuracy is observed for shorter lengths (0–10) across most types, with significant declines for longer lengths (11–19). Type 7 consistently shows the highest accuracy at Length 0 (91.0%), while Type 5 exhibits the lowest performance overall.