## Heatmap: Classification Accuracies ### Overview The image is a heatmap displaying classification accuracies for different models (TTPD, LR, CCS, MM) across various categories (cities_conj, cities_disj, etc.). The heatmap uses a color gradient from dark blue (low accuracy) to bright yellow (high accuracy) to represent the accuracy values. Each cell contains the accuracy value and its associated uncertainty (± value). ### Components/Axes * **Title:** Classification accuracies * **Columns (Models):** TTPD, LR, CCS, MM * **Rows (Categories):** cities\_conj, cities\_disj, sp\_en\_trans\_conj, sp\_en\_trans\_disj, inventors\_conj, inventors\_disj, animal\_class\_conj, animal\_class\_disj, element\_symb\_conj, element\_symb\_disj, facts\_conj, facts\_disj, common\_claim\_true\_false, counterfact\_true\_false * **Colorbar:** Ranges from 0.0 (dark blue) to 1.0 (bright yellow), representing the classification accuracy score. ### Detailed Analysis or ### Content Details Here's a breakdown of the accuracy values for each model and category: * **TTPD:** * cities\_conj: 61 ± 1 * cities\_disj: 55 ± 1 * sp\_en\_trans\_conj: 78 ± 1 * sp\_en\_trans\_disj: 72 ± 1 * inventors\_conj: 64 ± 1 * inventors\_disj: 54 ± 1 * animal\_class\_conj: 80 ± 2 * animal\_class\_disj: 55 ± 1 * element\_symb\_conj: 60 ± 2 * element\_symb\_disj: 61 ± 1 * facts\_conj: 63 ± 1 * facts\_disj: 57 ± 0 * common\_claim\_true\_false: 68 ± 1 * counterfact\_true\_false: 64 ± 1 * **LR:** * cities\_conj: 75 ± 8 * cities\_disj: 58 ± 6 * sp\_en\_trans\_conj: 73 ± 8 * sp\_en\_trans\_disj: 61 ± 5 * inventors\_conj: 68 ± 5 * inventors\_disj: 51 ± 7 * animal\_class\_conj: 84 ± 6 * animal\_class\_disj: 54 ± 3 * element\_symb\_conj: 81 ± 5 * element\_symb\_disj: 59 ± 7 * facts\_conj: 70 ± 3 * facts\_disj: 57 ± 3 * common\_claim\_true\_false: 75 ± 2 * counterfact\_true\_false: 76 ± 2 * **CCS:** * cities\_conj: 79 ± 9 * cities\_disj: 67 ± 6 * sp\_en\_trans\_conj: 71 ± 11 * sp\_en\_trans\_disj: 62 ± 8 * inventors\_conj: 71 ± 6 * inventors\_disj: 56 ± 6 * animal\_class\_conj: 89 ± 9 * animal\_class\_disj: 59 ± 4 * element\_symb\_conj: 79 ± 10 * element\_symb\_disj: 59 ± 11 * facts\_conj: 69 ± 5 * facts\_disj: 55 ± 4 * common\_claim\_true\_false: 73 ± 6 * counterfact\_true\_false: 70 ± 7 * **MM:** * cities\_conj: 61 ± 1 * cities\_disj: 54 ± 1 * sp\_en\_trans\_conj: 78 ± 1 * sp\_en\_trans\_disj: 72 ± 0 * inventors\_conj: 64 ± 1 * inventors\_disj: 54 ± 1 * animal\_class\_conj: 79 ± 1 * animal\_class\_disj: 54 ± 1 * element\_symb\_conj: 58 ± 2 * element\_symb\_disj: 61 ± 1 * facts\_conj: 62 ± 1 * facts\_disj: 56 ± 1 * common\_claim\_true\_false: 68 ± 0 * counterfact\_true\_false: 63 ± 1 ### Key Observations * The `animal_class_conj` category generally has high accuracy across all models, with CCS achieving the highest at 89 ± 9. * The `cities_disj` and `inventors_disj` categories tend to have lower accuracy scores compared to other categories across all models. * CCS generally has higher accuracy scores compared to the other models, especially for `animal_class_conj`. * MM has the lowest accuracy for `element_symb_conj` at 58 ± 2. ### Interpretation The heatmap provides a visual comparison of the classification accuracies of four different models across various categories. The color gradient allows for quick identification of high and low performance areas. The data suggests that some categories are inherently easier to classify than others, as evidenced by the consistently high or low scores across all models. The CCS model appears to perform better overall, particularly in the `animal_class_conj` category. The uncertainty values (±) provide a measure of the variability in the accuracy scores, which should be considered when comparing the models. The heatmap highlights the strengths and weaknesses of each model for different types of classification tasks.

## Heatmap: Classification Accuracies ### Overview This image presents a heatmap visualizing classification accuracies for various datasets and methods. The heatmap displays the performance of four different classification methods (TTPD, LR, CCS, MM) across ten different datasets, each represented in both conjunctive ('conj') and disjunctive ('disj') forms. The color intensity represents the accuracy, with warmer colors (yellow) indicating higher accuracy and cooler colors (blue) indicating lower accuracy. A colorbar on the right indicates the accuracy scale from 0.0 to 1.0. ### Components/Axes * **X-axis (Columns):** Represents the classification methods: TTPD, LR, CCS, and MM. * **Y-axis (Rows):** Represents the datasets: * cities\_conj * cities\_disj * sp\_en\_trans\_conj * sp\_en\_trans\_disj * inventors\_conj * inventors\_disj * animal\_class\_conj * animal\_class\_disj * element\_symb\_conj * element\_symb\_disj * facts\_conj * facts\_disj * common\_claim\_true\_false * counterfact\_true\_false * **Colorbar:** Scale from 0.0 (blue) to 1.0 (yellow) representing classification accuracy. * **Title:** "Classification accuracies" (positioned above the heatmap). * **Data Values:** Each cell contains a value in the format "X ± Y", representing the accuracy and its standard deviation. ### Detailed Analysis The heatmap displays accuracy values for each method-dataset combination. I will analyze each row (dataset) and describe the trends across the columns (methods). All values are approximate, based on visual estimation. * **cities\_conj:** TTPD: 61 ± 1, LR: 75 ± 8, CCS: 79 ± 9, MM: 61 ± 1. LR and CCS show higher accuracy than TTPD and MM. * **cities\_disj:** TTPD: 55 ± 1, LR: 58 ± 6, CCS: 67 ± 6, MM: 54 ± 1. CCS has the highest accuracy, followed by LR. * **sp\_en\_trans\_conj:** TTPD: 78 ± 1, LR: 73 ± 8, CCS: 71 ± 11, MM: 78 ± 1. TTPD and MM have the highest accuracy, closely followed by LR. * **sp\_en\_trans\_disj:** TTPD: 72 ± 1, LR: 61 ± 5, CCS: 62 ± 8, MM: 72 ± 0. TTPD and MM have the highest accuracy. * **inventors\_conj:** TTPD: 64 ± 1, LR: 68 ± 5, CCS: 71 ± 6, MM: 64 ± 1. CCS shows the highest accuracy. * **inventors\_disj:** TTPD: 54 ± 1, LR: 51 ± 7, CCS: 56 ± 6, MM: 54 ± 1. CCS has slightly higher accuracy than the others. * **animal\_class\_conj:** TTPD: 80 ± 2, LR: 84 ± 6, CCS: 89 ± 9, MM: 79 ± 1. CCS has the highest accuracy, followed by LR and TTPD. * **animal\_class\_disj:** TTPD: 55 ± 1, LR: 54 ± 3, CCS: 59 ± 4, MM: 54 ± 1. CCS has the highest accuracy. * **element\_symb\_conj:** TTPD: 61 ± 1, LR: 59 ± 7, CCS: 59 ± 11, MM: 61 ± 1. TTPD and MM have the highest accuracy. * **element\_symb\_disj:** TTPD: 63 ± 1, LR: 70 ± 3, CCS: 69 ± 5, MM: 62 ± 1. LR and CCS have the highest accuracy. * **facts\_conj:** TTPD: 63 ± 1, LR: 70 ± 3, CCS: 69 ± 5, MM: 62 ± 1. LR and CCS have the highest accuracy. * **facts\_disj:** TTPD: 57 ± 0, LR: 57 ± 3, CCS: 55 ± 4, MM: 56 ± 1. Accuracy is relatively similar across all methods. * **common\_claim\_true\_false:** TTPD: 68 ± 1, LR: 75 ± 2, CCS: 73 ± 6, MM: 68 ± 0. LR has the highest accuracy. * **counterfact\_true\_false:** TTPD: 64 ± 1, LR: 76 ± 2, CCS: 70 ± 7, MM: 63 ± 1. LR has the highest accuracy. ### Key Observations * **LR consistently performs well:** The LR method frequently achieves high accuracy, particularly on the 'disj' datasets. * **CCS often outperforms on conjunctive datasets:** CCS tends to have higher accuracy on datasets in the 'conj' format. * **TTPD and MM are often comparable:** These two methods often have similar accuracy values. * **Disjunctive vs. Conjunctive:** Accuracy values often differ between the 'conj' and 'disj' versions of the same dataset, suggesting the method's performance is sensitive to the dataset's structure. * **Standard Deviation:** The standard deviations are generally small (mostly ±1 to ±8), indicating relatively consistent performance across runs. ### Interpretation The heatmap demonstrates the performance of different classification methods on a variety of datasets. The varying accuracy levels suggest that the optimal method depends on the specific dataset being used. The consistent strong performance of LR indicates its robustness and general applicability. The differences between 'conj' and 'disj' performance highlight the importance of considering the dataset's logical structure when selecting a classification method. The small standard deviations suggest that the observed differences in accuracy are likely statistically significant. The data suggests that CCS is a strong performer on conjunctive datasets, while LR excels on disjunctive datasets. This could be due to the underlying algorithms of each method and how they handle different types of logical relationships within the data. The heatmap provides a valuable comparative analysis for selecting the most appropriate classification method for a given task.

## Heatmap: Classification Accuracies ### Overview The image is a heatmap titled "Classification accuracies" that compares the performance of four different models (TTPD, LR, CCS, MM) across fourteen distinct classification tasks. The performance metric is accuracy, presented as a mean value with an associated uncertainty (standard deviation or error). The data is encoded using a color gradient, with a corresponding color bar scale on the right side of the chart. ### Components/Axes * **Chart Title:** "Classification accuracies" (top center). * **Column Headers (Models):** Four models are listed horizontally across the top: * TTPD * LR * CCS * MM * **Row Labels (Tasks):** Fourteen tasks are listed vertically on the left side: 1. cities_conj 2. cities_disj 3. sp_en_trans_conj 4. sp_en_trans_disj 5. inventors_conj 6. inventors_disj 7. animal_class_conj 8. animal_class_disj 9. element_symb_conj 10. element_symb_disj 11. facts_conj 12. facts_disj 13. common_claim_true_false 14. counterfact_true_false * **Color Bar/Legend:** Positioned vertically on the far right. It maps color to accuracy values on a scale from 0.0 (dark purple) to 1.0 (bright yellow). Key markers are at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **Data Cells:** Each cell in the grid contains a numerical value in the format "mean ± uncertainty". The background color of each cell corresponds to its mean accuracy value according to the color bar. ### Detailed Analysis The following table reconstructs the data from the heatmap. Values are transcribed exactly as shown. | Task | TTPD Accuracy | LR Accuracy | CCS Accuracy | MM Accuracy | | :--- | :--- | :--- | :--- | :--- | | cities_conj | 61 ± 1 | 75 ± 8 | 79 ± 9 | 61 ± 1 | | cities_disj | 55 ± 1 | 58 ± 6 | 67 ± 6 | 54 ± 1 | | sp_en_trans_conj | 78 ± 1 | 73 ± 8 | 71 ± 11 | 78 ± 1 | | sp_en_trans_disj | 72 ± 1 | 61 ± 5 | 62 ± 8 | 72 ± 0 | | inventors_conj | 64 ± 1 | 68 ± 5 | 71 ± 6 | 64 ± 1 | | inventors_disj | 54 ± 1 | 51 ± 7 | 56 ± 6 | 54 ± 1 | | animal_class_conj | 80 ± 2 | 84 ± 6 | 89 ± 9 | 79 ± 1 | | animal_class_disj | 55 ± 1 | 54 ± 3 | 59 ± 4 | 54 ± 1 | | element_symb_conj | 60 ± 2 | 81 ± 5 | 79 ± 10 | 58 ± 2 | | element_symb_disj | 61 ± 1 | 59 ± 7 | 59 ± 11 | 61 ± 1 | | facts_conj | 63 ± 1 | 70 ± 3 | 69 ± 5 | 62 ± 1 | | facts_disj | 57 ± 0 | 57 ± 3 | 55 ± 4 | 56 ± 1 | | common_claim_true_false | 68 ± 1 | 75 ± 2 | 73 ± 6 | 68 ± 0 | | counterfact_true_false | 64 ± 1 | 76 ± 2 | 70 ± 7 | 63 ± 1 | **Visual Trend Verification:** * **Color Trend:** The heatmap shows a clear pattern where cells for "conj" (conjunctive) tasks are generally lighter (more yellow/orange, indicating higher accuracy) than their "disj" (disjunctive) counterparts, which are darker (more purple/red). * **Model Trend:** The LR and CCS columns contain the brightest cells overall, suggesting they achieve the highest peak accuracies on several tasks. The TTPD and MM columns are more uniformly colored in the mid-range (orange/red). ### Key Observations 1. **Task Difficulty:** "animal_class_conj" is the easiest task, with accuracies ranging from 79% to 89%. "inventors_disj" and "animal_class_disj" appear to be among the hardest, with most accuracies in the low-to-mid 50s. 2. **Conjunctive vs. Disjunctive:** For every task pair (e.g., cities_conj vs. cities_disj), the conjunctive version has a higher mean accuracy than the disjunctive version across all models. This is a consistent and strong pattern. 3. **Model Performance:** * **LR** achieves the single highest accuracy on the chart: 89 ± 9 on "animal_class_conj". It also performs very well on "element_symb_conj" (81 ± 5). * **CCS** shows high performance but with notably larger uncertainty ranges (e.g., 71 ± 11, 79 ± 10), suggesting less consistent results across runs or folds. * **TTPD** and **MM** have very similar performance profiles, often with identical or nearly identical mean accuracies and low uncertainties (±0 or ±1). Their performance is stable but generally not the highest. 4. **Notable Outliers:** The "sp_en_trans" tasks show an interesting reversal. For the conjunctive version, TTPD and MM (78 ± 1) outperform LR and CCS. For the disjunctive version, TTPD and MM (72 ± 0/1) again outperform LR and CCS. This is the only task family where this pattern occurs. ### Interpretation This heatmap provides a comparative analysis of model robustness across different types of logical or factual classification tasks. The data suggests several key insights: 1. **Task Structure is a Primary Determinant of Performance:** The consistent performance gap between conjunctive ("and") and disjunctive ("or") tasks indicates that reasoning about conjunctions is fundamentally easier for these models than reasoning about disjunctions. This could be due to the nature of the training data or the inherent complexity of the logical operations. 2. **Model Specialization:** No single model dominates all tasks. LR and CCS achieve the highest peak accuracies, but with greater variance (higher uncertainty). TTPD and MM offer more consistent, reliable performance, albeit at a slightly lower ceiling. This presents a trade-off between peak performance and stability. 3. **Domain-Specific Strengths:** The outlier in the "sp_en_trans" (likely Spanish-English translation) tasks suggests that TTPD and MM may have an architectural or training advantage for this specific type of linguistic or translational reasoning, which differs from their performance on other knowledge-based tasks (like cities, inventors, elements). 4. **Investigative Reading:** The high uncertainty in CCS's scores (e.g., ±11) warrants further investigation. It could indicate sensitivity to initialization, data splits, or a less stable optimization process. Conversely, the very low uncertainty in TTPD and MM scores suggests highly reproducible results. The choice between models, therefore, depends on the application's need for either maximum potential accuracy (favoring LR/CCS) or guaranteed consistent performance (favoring TTPD/MM).

## Heatmap: Classification accuracies ### Overview The image is a heatmap comparing classification accuracies across 12 different tasks/datasets and 4 methods (TTPD, LR, CCS, MM). Values are represented as percentages with standard deviations (±), using a color gradient from purple (low accuracy) to yellow (high accuracy). The legend on the right maps colors to numerical values (0.0–1.0). ### Components/Axes - **Y-axis (Tasks/Datasets)**: - cities_conj, cities_disj - sp_en_trans_conj, sp_en_trans_disj - inventors_conj, inventors_disj - animal_class_conj, animal_class_disj - element_symb_conj, element_symb_disj - facts_conj, facts_disj - common_claim_true_false, counterfact_true_false - **X-axis (Methods)**: TTPD, LR, CCS, MM - **Legend**: Color gradient from purple (0.0) to yellow (1.0), with intermediate values (0.2, 0.4, 0.6, 0.8). ### Detailed Analysis - **cities_conj**: - TTPD: 61 ± 1 (orange) - LR: 75 ± 8 (orange) - CCS: 79 ± 9 (yellow) - MM: 61 ± 1 (orange) - **cities_disj**: - TTPD: 55 ± 1 (red) - LR: 58 ± 6 (red) - CCS: 67 ± 6 (orange) - MM: 54 ± 1 (red) - **sp_en_trans_conj**: - TTPD: 78 ± 1 (yellow) - LR: 73 ± 8 (orange) - CCS: 71 ± 11 (orange) - MM: 78 ± 1 (yellow) - **sp_en_trans_disj**: - TTPD: 72 ± 1 (orange) - LR: 61 ± 5 (red) - CCS: 62 ± 8 (red) - MM: 72 ± 0 (orange) - **inventors_conj**: - TTPD: 64 ± 1 (orange) - LR: 68 ± 5 (orange) - CCS: 71 ± 6 (orange) - MM: 64 ± 1 (orange) - **inventors_disj**: - TTPD: 54 ± 1 (red) - LR: 51 ± 7 (red) - CCS: 56 ± 6 (red) - MM: 54 ± 1 (red) - **animal_class_conj**: - TTPD: 80 ± 2 (yellow) - LR: 84 ± 6 (yellow) - CCS: 89 ± 9 (bright yellow) - MM: 79 ± 1 (yellow) - **animal_class_disj**: - TTPD: 55 ± 1 (red) - LR: 54 ± 3 (red) - CCS: 59 ± 4 (red) - MM: 54 ± 1 (red) - **element_symb_conj**: - TTPD: 60 ± 2 (red) - LR: 81 ± 5 (orange) - CCS: 79 ± 10 (orange) - MM: 58 ± 2 (red) - **element_symb_disj**: - TTPD: 61 ± 1 (orange) - LR: 59 ± 7 (red) - CCS: 59 ± 11 (red) - MM: 61 ± 1 (orange) - **facts_conj**: - TTPD: 63 ± 1 (orange) - LR: 70 ± 3 (orange) - CCS: 69 ± 5 (orange) - MM: 62 ± 1 (orange) - **facts_disj**: - TTPD: 57 ± 0 (red) - LR: 57 ± 3 (red) - CCS: 55 ± 4 (red) - MM: 56 ± 1 (red) - **common_claim_true_false**: - TTPD: 68 ± 1 (orange) - LR: 75 ± 2 (orange) - CCS: 73 ± 6 (orange) - MM: 68 ± 0 (orange) - **counterfact_true_false**: - TTPD: 64 ± 1 (orange) - LR: 76 ± 2 (orange) - CCS: 70 ± 7 (orange) - MM: 63 ± 1 (orange) ### Key Observations 1. **CCS dominates in animal_class_conj**: Achieves the highest accuracy (89 ± 9) with bright yellow shading, far exceeding other methods. 2. **TTPD and MM parity**: These methods show similar performance across most tasks (e.g., cities_conj, sp_en_trans_conj). 3. **LR underperforms in disjunctive tasks**: Lower accuracies for disjunctive categories (e.g., cities_disj, inventors_disj) compared to conjunctive ones. 4. **CCS variability**: High standard deviations in some tasks (e.g., sp_en_trans_conj: ±11) suggest instability. 5. **MM consistency**: Lowest standard deviations (e.g., sp_en_trans_disj: ±0) indicate stable performance. ### Interpretation The data demonstrates that **CCS** is the most accurate method overall, particularly for conjunctive tasks like `animal_class_conj` and `sp_en_trans_conj`. However, its performance degrades in disjunctive tasks (e.g., `element_symb_disj`). **TTPD** and **MM** show comparable results but lag behind CCS in critical areas. **LR** struggles with disjunctive logic, suggesting limitations in handling negated or exclusive conditions. The standard deviations highlight that CCS’s high accuracy in `animal_class_conj` may come with higher variability, while MM’s consistency (e.g., ±0 in `sp_en_trans_disj`) makes it reliable for specific use cases. The heatmap underscores the importance of method selection based on task structure (conjunctive vs. disjunctive).