## Heatmap: Classification Accuracies ### Overview The image is a heatmap displaying classification accuracies for different categories using four different methods: TTPD, LR, CCS, and MM. The heatmap uses a color gradient from dark blue (0.0) to bright yellow (1.0) to represent the accuracy values. Each cell contains the accuracy value and its associated uncertainty (± value). ### Components/Axes * **Title:** Classification accuracies * **Columns (Methods):** TTPD, LR, CCS, MM * **Rows (Categories):** cities, neg\_cities, sp\_en\_trans, neg\_sp\_en\_trans, inventors, neg\_inventors, animal\_class, neg\_animal\_class, element\_symb, neg\_element\_symb, facts, neg\_facts * **Colorbar:** Ranges from 0.0 (dark blue) to 1.0 (bright yellow), representing classification accuracy. ### Detailed Analysis Here's a breakdown of the data for each category and method: * **cities:** * TTPD: 71 ± 2 * LR: 92 ± 7 * CCS: 77 ± 18 * MM: 60 ± 1 * **neg\_cities:** * TTPD: 100 ± 0 * LR: 100 ± 0 * CCS: 87 ± 20 * MM: 100 ± 0 * **sp\_en\_trans:** * TTPD: 99 ± 0 * LR: 99 ± 1 * CCS: 71 ± 21 * MM: 98 ± 0 * **neg\_sp\_en\_trans:** * TTPD: 98 ± 1 * LR: 95 ± 6 * CCS: 77 ± 23 * MM: 99 ± 1 * **inventors:** * TTPD: 88 ± 4 * LR: 93 ± 2 * CCS: 74 ± 18 * MM: 88 ± 5 * **neg\_inventors:** * TTPD: 94 ± 0 * LR: 86 ± 6 * CCS: 64 ± 16 * MM: 94 ± 1 * **animal\_class:** * TTPD: 99 ± 0 * LR: 99 ± 1 * CCS: 79 ± 21 * MM: 99 ± 1 * **neg\_animal\_class:** * TTPD: 99 ± 0 * LR: 99 ± 1 * CCS: 82 ± 17 * MM: 98 ± 1 * **element\_symb:** * TTPD: 95 ± 1 * LR: 98 ± 1 * CCS: 76 ± 19 * MM: 79 ± 4 * **neg\_element\_symb:** * TTPD: 86 ± 3 * LR: 90 ± 6 * CCS: 66 ± 19 * MM: 97 ± 2 * **facts:** * TTPD: 87 ± 0 * LR: 89 ± 1 * CCS: 69 ± 15 * MM: 86 ± 1 * **neg\_facts:** * TTPD: 73 ± 0 * LR: 73 ± 3 * CCS: 65 ± 13 * MM: 67 ± 1 ### Key Observations * LR and TTPD generally have higher accuracy scores compared to CCS. * MM performs well, often close to LR and TTPD. * CCS has the largest uncertainty (± values) in its accuracy scores. * For "neg\_cities", all methods except CCS achieve 100% accuracy. * The "cities" category shows the largest variation in accuracy across the four methods. ### Interpretation The heatmap visualizes the performance of four different classification methods across twelve categories. The color gradient allows for a quick comparison of accuracy scores. The data suggests that LR and TTPD are generally more accurate than CCS, while MM provides competitive results. The high uncertainty associated with CCS indicates that its performance may be less consistent. The perfect accuracy achieved by multiple methods for "neg\_cities" suggests that this category is relatively easy to classify. The variation in accuracy for "cities" indicates that this category may be more challenging.

## Heatmap: Classification Accuracies ### Overview This image presents a heatmap displaying classification accuracies for four different models (TTPD, LR, CCS, MM) across eleven different categories and their negations. The color intensity represents the accuracy, with yellow indicating higher accuracy and red indicating lower accuracy. A colorbar on the right indicates the accuracy scale from 0.0 to 1.0. ### Components/Axes * **Title:** "Classification accuracies" (centered at the top) * **Columns:** Representing the four models: TTPD, LR, CCS, MM. * **Rows:** Representing the eleven categories and their negations: cities, neg\_cities, sp\_en\_trans, neg\_sp\_en\_trans, inventors, neg\_inventors, animal\_class, neg\_animal\_class, element\_symb, neg\_element\_symb, facts, neg\_facts. * **Colorbar:** Located on the right side, ranging from 0.0 (red) to 1.0 (yellow), indicating classification accuracy. * **Data Points:** Each cell in the heatmap represents the accuracy of a specific model on a specific category, displayed as "value ± uncertainty". ### Detailed Analysis The heatmap contains 44 data points (4 models x 11 categories). Each cell shows the accuracy and its standard deviation. Here's a breakdown of the values, row by row, and column by column: **TTPD (First Column)** * cities: 71 ± 2 * neg\_cities: 100 ± 0 * sp\_en\_trans: 99 ± 0 * neg\_sp\_en\_trans: 98 ± 1 * inventors: 88 ± 4 * neg\_inventors: 94 ± 0 * animal\_class: 99 ± 0 * neg\_animal\_class: 99 ± 0 * element\_symb: 95 ± 1 * neg\_element\_symb: 86 ± 3 * facts: 87 ± 0 * neg\_facts: 73 ± 0 **LR (Second Column)** * cities: 92 ± 7 * neg\_cities: 100 ± 0 * sp\_en\_trans: 99 ± 1 * neg\_sp\_en\_trans: 95 ± 6 * inventors: 93 ± 2 * neg\_inventors: 86 ± 6 * animal\_class: 99 ± 1 * neg\_animal\_class: 99 ± 1 * element\_symb: 98 ± 1 * neg\_element\_symb: 90 ± 6 * facts: 89 ± 1 * neg\_facts: 73 ± 3 **CCS (Third Column)** * cities: 77 ± 18 * neg\_cities: 87 ± 20 * sp\_en\_trans: 71 ± 21 * neg\_sp\_en\_trans: 77 ± 23 * inventors: 74 ± 18 * neg\_inventors: 64 ± 16 * animal\_class: 79 ± 21 * neg\_animal\_class: 82 ± 17 * element\_symb: 76 ± 19 * neg\_element\_symb: 66 ± 19 * facts: 69 ± 15 * neg\_facts: 65 ± 13 **MM (Fourth Column)** * cities: 60 ± 1 * neg\_cities: 100 ± 0 * sp\_en\_trans: 98 ± 0 * neg\_sp\_en\_trans: 99 ± 1 * inventors: 88 ± 5 * neg\_inventors: 94 ± 1 * animal\_class: 99 ± 1 * neg\_animal\_class: 98 ± 1 * element\_symb: 79 ± 4 * neg\_element\_symb: 97 ± 2 * facts: 86 ± 1 * neg\_facts: 67 ± 1 ### Key Observations * **High Accuracy on Negations:** All models achieve very high accuracy (close to 1.0) on the "neg\_" categories (neg\_cities, neg\_sp\_en\_trans, neg\_inventors, etc.). This suggests the models are very good at identifying the *absence* of these features. * **Low Accuracy on Cities (MM):** The MM model performs significantly worse on the "cities" category (60 ± 1) compared to the other models. * **CCS Consistently Lower:** The CCS model generally exhibits lower accuracies across most categories compared to TTPD, LR, and MM. * **TTPD and LR Similar:** TTPD and LR models show relatively similar performance across most categories. * **Uncertainty:** The uncertainty values (±) are generally small, indicating relatively consistent performance. However, CCS has larger uncertainties in several categories. ### Interpretation This heatmap demonstrates the performance of four different classification models on a set of categories and their negations. The consistently high accuracy on negated categories suggests that these models are adept at identifying when a particular feature is *not* present. This could be due to the nature of the data or the specific algorithms used. The significant difference in performance on the "cities" category for the MM model is a notable outlier. This could indicate a specific weakness of the MM model in handling data related to cities, or a peculiarity in the dataset itself. Further investigation would be needed to determine the cause. The lower overall performance of the CCS model suggests it may not be as well-suited for this particular classification task compared to the other models. The larger uncertainty values associated with CCS also indicate less stable performance. The data suggests that the models are more confident in identifying the *absence* of features than their presence, which could be a valuable insight for improving the models or understanding the underlying data distribution. The heatmap provides a clear visual comparison of the models' strengths and weaknesses, allowing for informed decisions about which model to use for specific applications.

\n ## Heatmap: Classification Accuracies ### Overview The image is a heatmap titled "Classification accuracies" that compares the performance of four different methods or models (TTPD, LR, CCS, MM) across twelve distinct classification tasks or datasets. Each cell displays a mean accuracy percentage followed by a standard deviation (±). The color of each cell corresponds to its accuracy value, mapped to a vertical color bar on the right side of the chart that ranges from 0.0 (dark purple) to 1.0 (bright yellow). ### Components/Axes * **Title:** "Classification accuracies" (centered at the top). * **Column Headers (Methods/Models):** Four columns labeled from left to right: `TTPD`, `LR`, `CCS`, `MM`. * **Row Labels (Tasks/Datasets):** Twelve rows labeled from top to bottom: 1. `cities` 2. `neg_cities` 3. `sp_en_trans` 4. `neg_sp_en_trans` 5. `inventors` 6. `neg_inventors` 7. `animal_class` 8. `neg_animal_class` 9. `element_symb` 10. `neg_element_symb` 11. `facts` 12. `neg_facts` * **Color Scale/Legend:** A vertical bar on the right side of the chart. The scale runs from 0.0 at the bottom (dark purple) to 1.0 at the top (bright yellow). Intermediate colors include red/orange around 0.5-0.7 and yellow-green above 0.8. * **Data Cells:** Each cell contains text in the format `[Accuracy] ± [Standard Deviation]`. The background color of the cell is determined by the accuracy value. ### Detailed Analysis The following table reconstructs the data presented in the heatmap. Values are percentages. | Task/Dataset | TTPD Accuracy | LR Accuracy | CCS Accuracy | MM Accuracy | | :--- | :--- | :--- | :--- | :--- | | **cities** | 71 ± 2 | 92 ± 7 | 77 ± 18 | 60 ± 1 | | **neg_cities** | 100 ± 0 | 100 ± 0 | 87 ± 20 | 100 ± 0 | | **sp_en_trans** | 99 ± 0 | 99 ± 1 | 71 ± 21 | 98 ± 0 | | **neg_sp_en_trans** | 98 ± 1 | 95 ± 6 | 77 ± 23 | 99 ± 1 | | **inventors** | 88 ± 4 | 93 ± 2 | 74 ± 18 | 88 ± 5 | | **neg_inventors** | 94 ± 0 | 86 ± 6 | 64 ± 16 | 94 ± 1 | | **animal_class** | 99 ± 0 | 99 ± 1 | 79 ± 21 | 99 ± 1 | | **neg_animal_class** | 99 ± 0 | 99 ± 1 | 82 ± 17 | 98 ± 1 | | **element_symb** | 95 ± 1 | 98 ± 1 | 76 ± 19 | 79 ± 4 | | **neg_element_symb** | 86 ± 3 | 90 ± 6 | 66 ± 19 | 97 ± 2 | | **facts** | 87 ± 0 | 89 ± 1 | 69 ± 15 | 86 ± 1 | | **neg_facts** | 73 ± 0 | 73 ± 3 | 65 ± 13 | 67 ± 1 | **Visual Trend Verification by Column:** * **TTPD:** Predominantly high accuracy (yellow cells), with notable dips for `cities` (71%) and `neg_facts` (73%). * **LR:** Consistently high accuracy (yellow cells), with the lowest scores for `neg_inventors` (86%) and `neg_facts` (73%). * **CCS:** Shows the lowest overall performance and highest variability (more orange/red cells). Accuracies are generally 10-30 percentage points lower than the other methods, with very high standard deviations (often ±15 to ±23). * **MM:** High accuracy across most tasks (yellow cells), similar to TTPD and LR. The lowest scores are for `cities` (60%) and `neg_facts` (67%). ### Key Observations 1. **Performance Disparity:** The `CCS` method is a clear outlier, performing significantly worse and with much higher uncertainty (larger standard deviations) than `TTPD`, `LR`, and `MM` across all tasks. 2. **Task Difficulty:** The `neg_facts` task appears to be the most challenging, yielding the lowest or near-lowest scores for all four methods (73%, 73%, 65%, 67%). The `cities` task is also relatively difficult for `TTPD` and `MM`. 3. **Near-Perfect Performance:** The `neg_cities` task is solved with perfect or near-perfect accuracy (100 ± 0) by `TTPD`, `LR`, and `MM`. The `animal_class` and `neg_animal_class` tasks also show near-perfect results for these three methods. 4. **High Variability in CCS:** The standard deviations for `CCS` are an order of magnitude larger than for the other methods, indicating its performance is highly unstable or sensitive to the specific data split or run. 5. **Negation Pattern:** There is no consistent pattern where "neg_" (negation) tasks are universally harder. For example, `neg_cities` is easier than `cities` for all methods, while `neg_facts` is harder than `facts` for all methods. ### Interpretation This heatmap provides a comparative benchmark of four classification methods. The data strongly suggests that **TTPD, LR, and MM are robust, high-performing, and stable methods** for the given set of tasks, achieving accuracies often above 90% with minimal variance. They appear to be reliable choices. In contrast, **CCS is demonstrably inferior** for this specific evaluation. Its low mean accuracies and high standard deviations suggest it may be an ill-suited model for these tasks, suffers from high variance in training, or was perhaps evaluated under different, less favorable conditions. The high uncertainty makes its reported accuracy less trustworthy. The variation in task difficulty (e.g., `neg_cities` vs. `neg_facts`) implies that the underlying datasets or problem definitions differ significantly in complexity or the models' familiarity with the concepts. The perfect scores on `neg_cities` might indicate a trivial or highly predictable pattern in that specific dataset. Overall, this chart would guide a researcher to prefer TTPD, LR, or MM for deployment on similar tasks and to investigate the causes of CCS's poor and unstable performance.

## Heatmap: Classification Accuracies ### Overview The image is a heatmap visualizing classification accuracy across four machine learning models (TTPD, LR, CCS, MM) for 12 distinct categories. Accuracy values are presented with uncertainty (±σ), and colors range from purple (0.0) to yellow (1.0) based on a gradient scale. ### Components/Axes - **X-axis (Models)**: TTPD, LR, CCS, MM (left to right) - **Y-axis (Categories)**: 1. cities 2. neg_cities 3. sp_en_trans 4. neg_sp_en_trans 5. inventors 6. neg_inventors 7. animal_class 8. neg_animal_class 9. element_symbol 10. neg_element_symbol 11. facts 12. neg_facts - **Legend**: Vertical color bar on the right (0.0 = purple, 1.0 = yellow) - **Title**: "Classification accuracies" at the top center ### Detailed Analysis #### Data Table Reconstruction | Category | TTPD | LR | CCS | MM | |------------------------|------------|------------|-------------|------------| | cities | 71 ± 2 | 92 ± 7 | 77 ± 18 | 60 ± 1 | | neg_cities | 100 ± 0 | 100 ± 0 | 87 ± 20 | 100 ± 0 | | sp_en_trans | 99 ± 0 | 99 ± 1 | 71 ± 21 | 98 ± 0 | | neg_sp_en_trans | 98 ± 1 | 95 ± 6 | 77 ± 23 | 99 ± 1 | | inventors | 88 ± 4 | 93 ± 2 | 74 ± 18 | 88 ± 5 | | neg_inventors | 94 ± 0 | 86 ± 6 | 64 ± 16 | 94 ± 1 | | animal_class | 99 ± 0 | 99 ± 1 | 79 ± 21 | 99 ± 1 | | neg_animal_class | 99 ± 0 | 99 ± 1 | 82 ± 17 | 98 ± 1 | | element_symbol | 95 ± 1 | 98 ± 1 | 76 ± 19 | 79 ± 4 | | neg_element_symbol | 86 ± 3 | 90 ± 6 | 66 ± 19 | 97 ± 2 | | facts | 87 ± 0 | 89 ± 1 | 69 ± 15 | 86 ± 1 | | neg_facts | 73 ± 0 | 73 ± 3 | 65 ± 13 | 67 ± 1 | #### Spatial Grounding - **Legend**: Right-aligned vertical color bar (0.0–1.0) - **Title**: Centered at the top - **Axes**: - X-axis labels (models) at the top - Y-axis labels (categories) on the left - **Cell Colors**: Match legend gradient (e.g., 71 ± 2 = reddish-orange, 100 ± 0 = bright yellow) ### Key Observations 1. **Model Performance**: - **LR (Logistic Regression)** and **MM (Meta-Models)** consistently achieve the highest accuracies (90–100%). - **CCS** underperforms in most categories (64–87%), with significant uncertainty (±13–23). - **TTPD** shows moderate performance (67–95%) with lower uncertainty (±0–7). 2. **Category Trends**: - **neg_* categories** (e.g., neg_cities, neg_inventors) achieve near-perfect accuracy (94–100%) across models, suggesting easier classification for negative classes. - **element_symbol** and **neg_element_symbol** show high variability (76–98%), with CCS struggling most (±19). - **facts** and **neg_facts** have the lowest accuracies (65–89%), particularly for CCS (±13–15). 3. **Uncertainty Patterns**: - CCS exhibits the highest uncertainty (e.g., 77 ± 23 for sp_en_trans). - LR and MM demonstrate the lowest uncertainty (e.g., 100 ± 0 for neg_cities). ### Interpretation The data suggests: - **LR and MM** are robust models with high accuracy and low variability, likely due to their simplicity (LR) or ensemble approaches (MM). - **CCS** struggles with complex or imbalanced categories (e.g., neg_inventors, neg_facts), possibly due to overfitting or insufficient feature representation. - **neg_* categories** are consistently easier to classify, indicating potential class imbalance or distinct separability in negative classes. - The **element_symbol** and **facts** categories show the greatest model divergence, highlighting challenges in symbolic or factual reasoning. This heatmap underscores the importance of model selection based on task complexity and data characteristics, with LR/MM being preferable for high-stakes applications requiring reliability.