## Heatmap: Syllogism Format vs. Predicted Validity ### Overview The image is a heatmap visualizing the number of predicted valid syllogisms for different syllogism formats across four conditions: zh+, zh-, en+, and en-. The color intensity represents the number of predicted valid syllogisms, ranging from 0 (dark purple) to 100 (light yellow). The heatmap is divided into two distinct regions separated by a red line. The top region shows high validity across all conditions, while the bottom region shows varying degrees of validity depending on the syllogism format and condition. ### Components/Axes * **Y-axis:** Syllogism Format. The syllogism formats are listed vertically, with the first 12 formats (AAA-1 to EIO-4) in the top region and the remaining 8 formats (AAI-1 to EAO-4) in the bottom region. * **X-axis:** Conditions. The conditions are zh+, zh-, en+, and en-. * **Color Scale:** The color scale represents the number of predicted valid syllogisms, ranging from 0 (dark purple) to 100 (light yellow). * **Legend:** Located on the right side of the heatmap, showing the color gradient and corresponding numerical values (0, 20, 40, 60, 80, 100). The label for the legend is "The number of predicted VALID". ### Detailed Analysis **Syllogism Formats (Y-axis):** * AAA-1 * EAE-1 * AII-1 * EIO-1 * EAE-2 * AEE-2 * EIO-2 * AOO-2 * AII-3 * IAI-3 * OAO-3 * EIO-3 * AEE-4 * IAI-4 * EIO-4 * AAI-1 * EAO-1 * AEO-2 * EAO-2 * AAI-3 * EAO-3 * AAI-4 * AEO-4 * EAO-4 **Conditions (X-axis):** * zh+ * zh- * en+ * en- **Data Points:** * **Top Region (AAA-1 to EIO-4):** All cells in this region are light yellow, indicating a value close to 100 for all syllogism formats and conditions. * **AAI-1:** zh+ (dark purple, ~0), zh- (dark purple, ~0), en+ (dark purple, ~0), en- (dark purple, ~0) * **EAO-1:** zh+ (dark purple, ~0), zh- (dark purple, ~0), en+ (dark purple, ~0), en- (dark purple, ~0) * **AEO-2:** zh+ (dark purple, ~0), zh- (red-purple, ~30), en+ (dark purple, ~0), en- (dark purple, ~0) * **EAO-2:** zh+ (red-purple, ~30), zh- (orange, ~70), en+ (dark purple, ~0), en- (dark purple, ~0) * **AAI-3:** zh+ (dark purple, ~0), zh- (dark purple, ~0), en+ (dark purple, ~0), en- (dark purple, ~0) * **EAO-3:** zh+ (red-purple, ~30), zh- (orange, ~70), en+ (red-purple, ~30), en- (dark purple, ~0) * **AAI-4:** zh+ (dark purple, ~0), zh- (dark purple, ~0), en+ (dark purple, ~0), en- (dark purple, ~0) * **AEO-4:** zh+ (dark purple, ~0), zh- (red-purple, ~30), en+ (dark purple, ~0), en- (dark purple, ~0) * **EAO-4:** zh+ (red-purple, ~30), zh- (red-purple, ~30), en+ (dark purple, ~0), en- (dark purple, ~0) ### Key Observations * The top 15 syllogism formats (AAA-1 to EIO-4) consistently show high predicted validity across all conditions (zh+, zh-, en+, en-). * The bottom 9 syllogism formats (AAI-1 to EAO-4) show significantly lower predicted validity, with some formats showing higher validity in the zh- condition. * The 'en+' and 'en-' conditions generally show very low predicted validity for the bottom syllogism formats. * A red line separates the two regions of the heatmap, visually highlighting the difference in predicted validity between the two groups of syllogism formats. ### Interpretation The heatmap suggests that certain syllogism formats (AAA-1 to EIO-4) are consistently predicted as valid, regardless of the condition (zh+, zh-, en+, en-). In contrast, other syllogism formats (AAI-1 to EAO-4) are generally predicted as invalid, with some exceptions in the zh- condition. The 'en+' and 'en-' conditions appear to have a negative impact on the predicted validity of these syllogism formats. The separation of the heatmap into two distinct regions indicates a clear difference in the predicted validity of different syllogism formats. This could be due to the inherent logical structure of the syllogisms or the way they are processed under different conditions. The higher validity observed in the zh- condition for some syllogism formats suggests that this condition may be more conducive to valid reasoning for those specific formats. The consistently low validity in the 'en+' and 'en-' conditions warrants further investigation to understand the factors contributing to this effect.

## Heatmap: Syllogism Format Validation Counts ### Overview This image presents a heatmap visualizing the number of predicted valid syllogisms for different syllogism formats and languages. The heatmap uses a color gradient to represent the count, ranging from light yellow (low count) to dark purple (high count). The x-axis represents the language, and the y-axis represents the syllogism format. ### Components/Axes * **X-axis:** Language - with categories "zh+" (Chinese Positive), "zh-" (Chinese Negative), "en+" (English Positive), "en-" (English Negative). * **Y-axis:** Syllogism Format - with the following categories: * AAA-1 * EAE-1 * AII-1 * EIO-1 * EAE-2 * AEE-2 * EIO-2 * AOO-2 * AII-3 * IAI-3 * OAO-3 * EIO-3 * AEE-4 * IAI-4 * EIO-4 * AAI-1 * EAO-1 * AEO-2 * EAO-2 * AAI-3 * EAO-3 * AAI-4 * AEO-4 * EAO-4 * **Color Scale (Right):** "The number of predicted VALID" ranging from 0 (dark purple) to 100 (light yellow). ### Detailed Analysis The heatmap displays the counts for each combination of syllogism format and language. The values are approximate, based on the color gradient. * **zh+ (Chinese Positive):** * AAA-1: ~95 * EAE-1: ~90 * AII-1: ~85 * EIO-1: ~80 * EAE-2: ~75 * AEE-2: ~70 * EIO-2: ~65 * AOO-2: ~60 * AII-3: ~55 * IAI-3: ~50 * OAO-3: ~45 * EIO-3: ~40 * AEE-4: ~35 * IAI-4: ~30 * EIO-4: ~25 * AAI-1: ~20 * EAO-1: ~15 * AEO-2: ~10 * EAO-2: ~10 * AAI-3: ~5 * EAO-3: ~5 * AAI-4: ~0 * AEO-4: ~0 * EAO-4: ~0 * **zh- (Chinese Negative):** * AAA-1: ~85 * EAE-1: ~80 * AII-1: ~75 * EIO-1: ~70 * EAE-2: ~65 * AEE-2: ~60 * EIO-2: ~55 * AOO-2: ~50 * AII-3: ~45 * IAI-3: ~40 * OAO-3: ~35 * EIO-3: ~30 * AEE-4: ~25 * IAI-4: ~20 * EIO-4: ~15 * AAI-1: ~10 * EAO-1: ~5 * AEO-2: ~5 * EAO-2: ~5 * AAI-3: ~0 * EAO-3: ~0 * AAI-4: ~0 * AEO-4: ~0 * EAO-4: ~0 * **en+ (English Positive):** * AAA-1: ~70 * EAE-1: ~65 * AII-1: ~60 * EIO-1: ~55 * EAE-2: ~50 * AEE-2: ~45 * EIO-2: ~40 * AOO-2: ~35 * AII-3: ~30 * IAI-3: ~25 * OAO-3: ~20 * EIO-3: ~15 * AEE-4: ~10 * IAI-4: ~5 * EIO-4: ~5 * AAI-1: ~0 * EAO-1: ~0 * AEO-2: ~0 * EAO-2: ~0 * AAI-3: ~0 * EAO-3: ~0 * AAI-4: ~0 * AEO-4: ~0 * EAO-4: ~0 * **en- (English Negative):** * AAA-1: ~60 * EAE-1: ~55 * AII-1: ~50 * EIO-1: ~45 * EAE-2: ~40 * AEE-2: ~35 * EIO-2: ~30 * AOO-2: ~25 * AII-3: ~20 * IAI-3: ~15 * OAO-3: ~10 * EIO-3: ~5 * AEE-4: ~0 * IAI-4: ~0 * EIO-4: ~0 * AAI-1: ~0 * EAO-1: ~0 * AEO-2: ~0 * EAO-2: ~0 * AAI-3: ~0 * EAO-3: ~0 * AAI-4: ~0 * AEO-4: ~0 * EAO-4: ~0 ### Key Observations * The counts are generally highest for the "zh+" (Chinese Positive) language and decrease as we move to "zh-", "en+", and "en-". * The "AAA-1" format consistently shows the highest counts across all languages. * The "AAI-4", "AEO-4", and "EAO-4" formats consistently show the lowest counts (close to zero) across all languages. * There is a clear trend of decreasing counts as the syllogism format number increases (e.g., from -1 to -4). ### Interpretation The heatmap suggests that the model performs best at predicting valid syllogisms in Chinese (positive polarity) and struggles more with English (especially negative polarity). The performance also varies significantly depending on the syllogism format, with some formats being much easier to validate than others. The consistent high performance for "AAA-1" and low performance for "AAI-4", "AEO-4", and "EAO-4" could indicate inherent differences in the logical structure or complexity of these formats. The difference between positive and negative polarity within each language suggests the model may be sensitive to the phrasing or construction of the syllogisms. This data could be used to improve the model's performance by focusing on the more challenging languages and syllogism formats.

## Heatmap: Syllogism Format Prediction Validity by Language Prompt ### Overview This image is a heatmap visualizing the number of predicted "VALID" outcomes for various syllogism formats under four different language prompt conditions. The data is presented in a grid where color intensity represents the count, with a clear separation between two groups of syllogism formats. ### Components/Axes * **Y-Axis (Vertical):** Labeled **"Syllogism Format"**. It lists 26 distinct syllogism format codes. A horizontal red line separates the list into two distinct groups. * **Top Group (15 formats, above red line):** AAA-1, EAE-1, AII-1, EIO-1, EAE-2, AEE-2, EIO-2, AOO-2, AII-3, IAI-3, OAO-3, EIO-3, AEE-4, IAI-4, EIO-4. * **Bottom Group (11 formats, below red line):** AAI-1, EAO-1, AEO-2, EAO-2, AAI-3, EAO-3, AAI-4, AEO-4, EAO-4. * **X-Axis (Horizontal):** Four categorical labels representing language prompt conditions: * `zh+` (Chinese, positive framing) * `zh-` (Chinese, negative framing) * `en+` (English, positive framing) * `en-` (English, negative framing) * **Color Bar/Legend (Right Side):** A vertical gradient bar titled **"The number of predicted VALID"**. The scale runs from **0** (black/dark purple) at the bottom to **100** (light yellow) at the top, with intermediate markers at 20, 40, 60, and 80. This bar serves as the key for interpreting the cell colors in the heatmap. ### Detailed Analysis The heatmap is divided into two clear regions by a horizontal red line. **1. Top Region (Above Red Line):** * **Trend:** All 15 syllogism formats in this group show uniformly high values across all four language prompt conditions (`zh+`, `zh-`, `en+`, `en-`). * **Data Points:** Every cell in this 15x4 block is colored light yellow, corresponding to the top of the color scale. The number of predicted VALID outcomes is approximately **100** for every combination. There is no visible variation within this group. **2. Bottom Region (Below Red Line):** * **Trend:** This group shows significant variation in values, both between different syllogism formats and across the four language conditions. Values are generally much lower than in the top region. * **Data Points (Approximate values based on color):** * **AAI-1:** `zh+` (~10, dark purple), `zh-` (~20, purple), `en+` (~0, black), `en-` (~5, very dark purple). * **EAO-1:** `zh+` (~15), `zh-` (~30, magenta), `en+` (~10), `en-` (~10). * **AEO-2:** `zh+` (~30), `zh-` (~40, pinkish), `en+` (~0, black), `en-` (~10). * **EAO-2:** `zh+` (~35), `zh-` (~50, salmon), `en+` (~10), `en-` (~25). * **AAI-3:** `zh+` (~15), `zh-` (~30), `en+` (~5), `en-` (~0, black). * **EAO-3:** `zh+` (~30), `zh-` (~60, orange), `en+` (~10), `en-` (~25). * **AAI-4:** `zh+` (~0, black), `zh-` (~0, black), `en+` (~5), `en-` (~5). * **AEO-4:** `zh+` (~5), `zh-` (~25), `en+` (~5), `en-` (~10). * **EAO-4:** `zh+` (~25), `zh-` (~35), `en+` (~10), `en-` (~20). ### Key Observations 1. **Bimodal Distribution:** The red line acts as a stark divider. The 15 formats above it are predicted as VALID nearly 100% of the time regardless of language prompt. The 11 formats below it have much lower and more variable validity prediction rates. 2. **Language Prompt Effect:** Within the bottom group, the `zh-` (Chinese, negative) condition consistently yields the highest number of predicted VALID outcomes for most formats (e.g., EAO-3 peaks at ~60). The `en+` (English, positive) condition often results in the lowest values, frequently near zero. 3. **Format-Specific Patterns:** Certain formats like EAO-2 and EAO-3 show relatively higher validity predictions, especially under Chinese prompts. Others, like AAI-4, show near-zero validity predictions across all conditions. ### Interpretation This heatmap likely presents results from an experiment testing how different language frames (Chinese/English, positive/negative) affect an AI model's judgment of the logical validity of various syllogistic reasoning formats. * **The Red Line's Significance:** The clean separation suggests the top 15 formats are **classically valid** syllogisms (e.g., AAA-1, EAE-2). The model correctly identifies them as valid nearly perfectly. The bottom 11 formats are likely **classically invalid** or "weak" syllogisms (e.g., AAI, EAO forms). The model's ability to predict them as invalid is inconsistent and influenced by the prompt. * **Language and Framing Bias:** The data indicates a potential bias. The model is more likely to incorrectly label an invalid syllogism as "VALID" when prompted in Chinese, especially with negative framing (`zh-`). Conversely, it is more conservative (predicting fewer VALIDs) when prompted in English with positive framing (`en+`). This suggests the model's logical reasoning is not perfectly language- or frame-invariant. * **Practical Implication:** The findings highlight that for robust, unbiased logical reasoning, AI models may require careful prompt engineering or specialized training, as their performance can vary significantly based on superficial linguistic cues, even on formal logic tasks.

## Heatmap: Syllogism Format vs. Language Categories ### Overview The image is a heatmap visualizing the relationship between syllogism formats (y-axis) and language categories (x-axis: zh+, zh-, en+, en-). Color intensity represents the number of predicted VALID outcomes, ranging from 0 (black) to 100 (yellow). A red horizontal line separates the top and bottom halves of the chart. ### Components/Axes - **Y-Axis (Syllogism Format)**: - Categories: AAA-1, EAE-1, AII-1, EIO-1, EAE-2, AEE-2, EIO-2, AOO-2, AII-3, IAI-3, OAO-3, EIO-3, AEE-4, IAI-4, EIO-4 (top to bottom). - **X-Axis (Language Categories)**: - Categories: zh+, zh-, en+, en- (left to right). - **Color Legend**: - Scale: 0 (black) to 100 (yellow), labeled "The number of predicted VALID." - Red line at the midpoint (y-axis) separates high and low-value regions. ### Detailed Analysis - **Top Half (Above Red Line)**: - All cells are uniformly yellow (≈100), indicating maximum predicted VALID for all combinations in this region. - **Bottom Half (Below Red Line)**: - **EAO-2**: - zh+: ≈20 (light purple), zh-: ≈60 (orange), en+: ≈0 (black), en-: ≈10 (dark purple). - **EAO-3**: - zh+: ≈20 (light purple), zh-: ≈60 (orange), en+: ≈0 (black), en-: ≈10 (dark purple). - **EAO-4**: - zh+: ≈0 (black), zh-: ≈20 (light purple), en+: ≈0 (black), en-: ≈10 (dark purple). - **Other Bottom Rows**: - Values cluster between 0 (black) and 20 (light purple), with occasional orange (≈60) in zh- categories. ### Key Observations 1. **High-Value Region**: The top half (above the red line) shows perfect prediction (100% VALID) across all syllogism formats and language categories. 2. **Low-Value Region**: The bottom half exhibits sparse VALID predictions, with most cells near 0 (black) or 10-20 (purple/orange). 3. **Red Line Significance**: The red line likely represents a threshold (e.g., 50% VALID), dividing high-confidence and low-confidence predictions. 4. **Language-Specific Trends**: - zh- categories (e.g., EAO-2, EAO-3) show higher VALID predictions (≈60) compared to zh+ (≈20) and en+ (≈0). - en- consistently shows low VALID predictions (≈10) across most syllogism formats. ### Interpretation The heatmap suggests that syllogism formats in the top half (e.g., AAA-1, EAE-1) are universally predicted as VALID, possibly due to structural simplicity or alignment with training data. In contrast, formats in the bottom half (e.g., EAO-2, EAO-4) exhibit language-dependent performance, with zh- categories outperforming others. The red line may indicate a critical cutoff for model confidence, beyond which predictions become unreliable. The stark contrast between high and low regions implies potential biases in the model’s handling of complex syllogisms or non-native language structures.