## Scatter Plot Grid: Fraction of Variance Explained by PCs ### Overview The image presents a grid of six scatter plots. Each plot shows the fraction of variance explained by principal components (PCs) for different combinations of affirmative, negated, German, conjunction, and disjunction sentence types. The x-axis represents the PC index (from 1 to 10), and the y-axis represents the explained variance (from 0 to approximately 0.6). All plots show a decreasing trend, indicating that the first few PCs explain most of the variance. ### Components/Axes * **Title:** Fraction of variance in centered and averaged activations explained by PCs * **X-axis (all plots):** PC index, with ticks at 2, 4, 6, 8, and 10. * **Y-axis (left plots):** Explained variance, with ticks at 0.0, 0.2, 0.4, and 0.6. * **Y-axis (right and center plots):** Explained variance, with ticks at 0.1, 0.2, and 0.3. * **Plot Titles (from top-left to bottom-right):** * affirmative * affirmative, negated * affirmative, negated, conjunctions * affirmative, affirmative German * affirmative, affirmative German, negated, negated German * affirmative, negated, conjunctions, disjunctions ### Detailed Analysis **Plot 1: affirmative** * Trend: Decreasing. The explained variance drops sharply from PC 1 to PC 2, then decreases more gradually. * PC 1: Approximately 0.61 * PC 2: Approximately 0.14 * PC 10: Approximately 0.01 **Plot 2: affirmative, negated** * Trend: Decreasing. Similar to Plot 1, with a sharp initial drop. * PC 1: Approximately 0.29 * PC 2: Approximately 0.09 * PC 10: Approximately 0.02 **Plot 3: affirmative, negated, conjunctions** * Trend: Decreasing. Again, a sharp initial drop. * PC 1: Approximately 0.32 * PC 2: Approximately 0.24 * PC 10: Approximately 0.02 **Plot 4: affirmative, affirmative German** * Trend: Decreasing. * PC 1: Approximately 0.50 * PC 2: Approximately 0.13 * PC 10: Approximately 0.01 **Plot 5: affirmative, affirmative German, negated, negated German** * Trend: Decreasing. * PC 1: Approximately 0.29 * PC 2: Approximately 0.17 * PC 10: Approximately 0.01 **Plot 6: affirmative, negated, conjunctions, disjunctions** * Trend: Decreasing. * PC 1: Approximately 0.32 * PC 2: Approximately 0.23 * PC 10: Approximately 0.01 ### Key Observations * In all plots, the first principal component explains the largest fraction of variance. * The explained variance decreases rapidly for the first few PCs and then levels off. * The "affirmative" plot has the highest initial explained variance (PC 1 ≈ 0.61), while the other plots have lower initial values (around 0.3 to 0.5). * The explained variance for PC 10 is consistently low (around 0.01-0.02) across all plots. ### Interpretation The plots demonstrate that a small number of principal components can capture a significant portion of the variance in the centered and averaged activations. This suggests that the underlying data has a relatively low intrinsic dimensionality. The "affirmative" condition seems to have a more dominant first principal component compared to the other conditions, which include negations, conjunctions, and disjunctions. This could indicate that affirmative sentences have a more consistent activation pattern across the dataset. The addition of German sentences and their negations does not drastically alter the overall trend, suggesting that the principal components are capturing more general linguistic features rather than language-specific ones. The consistent low variance explained by higher-order PCs (e.g., PC 10) indicates that these components capture noise or less important variations in the data.

## Scatter Plots: Fraction of Variance Explained by PCs ### Overview The image presents six scatter plots, each visualizing the fraction of variance in centered and averaged activations explained by Principal Components (PCs). Each plot corresponds to a different linguistic condition. The x-axis represents the PC index (ranging from 1 to 10), and the y-axis represents the explained variance (ranging from 0 to approximately 0.6). ### Components/Axes * **Title:** "Fraction of variance in centered and averaged activations explained by PCs" (centered at the top) * **X-axis Label:** "PC index" (present on all plots) * **Y-axis Label:** "Explained variance" (present on all plots) * **Plots (from top-left to bottom-right):** 1. "affirmative" 2. "affirmative, negated" 3. "affirmative, negated, conjunctions" 4. "affirmative, affirmative German" 5. "affirmative, affirmative German, negated, negated German" 6. "affirmative, negated, conjunctions, disjunctions" ### Detailed Analysis Each plot shows a distribution of explained variance across the first 10 PCs. The plots share the same x and y axis scales. **Plot 1: "affirmative"** * Trend: The explained variance starts high at PC index 1 and rapidly decreases, then plateaus around 0.05-0.1 for PC indices 3-10. * Data Points (approximate): * PC 1: 0.62 * PC 2: 0.12 * PC 3: 0.07 * PC 4: 0.06 * PC 5: 0.05 * PC 6: 0.04 * PC 7: 0.03 * PC 8: 0.03 * PC 9: 0.03 * PC 10: 0.03 **Plot 2: "affirmative, negated"** * Trend: Similar to the first plot, but with lower overall explained variance. Starts around 0.3 and decreases rapidly, plateauing around 0.05-0.1. * Data Points (approximate): * PC 1: 0.31 * PC 2: 0.14 * PC 3: 0.08 * PC 4: 0.06 * PC 5: 0.05 * PC 6: 0.04 * PC 7: 0.04 * PC 8: 0.04 * PC 9: 0.04 * PC 10: 0.04 **Plot 3: "affirmative, negated, conjunctions"** * Trend: Similar to the previous plots, with a slightly lower initial variance. * Data Points (approximate): * PC 1: 0.28 * PC 2: 0.12 * PC 3: 0.07 * PC 4: 0.06 * PC 5: 0.05 * PC 6: 0.04 * PC 7: 0.04 * PC 8: 0.04 * PC 9: 0.04 * PC 10: 0.04 **Plot 4: "affirmative, affirmative German"** * Trend: Similar to the first plot, with a slightly lower initial variance. * Data Points (approximate): * PC 1: 0.42 * PC 2: 0.11 * PC 3: 0.07 * PC 4: 0.06 * PC 5: 0.05 * PC 6: 0.04 * PC 7: 0.04 * PC 8: 0.04 * PC 9: 0.04 * PC 10: 0.04 **Plot 5: "affirmative, affirmative German, negated, negated German"** * Trend: Similar to the previous plots, with a slightly lower initial variance. * Data Points (approximate): * PC 1: 0.32 * PC 2: 0.13 * PC 3: 0.08 * PC 4: 0.06 * PC 5: 0.05 * PC 6: 0.04 * PC 7: 0.04 * PC 8: 0.04 * PC 9: 0.04 * PC 10: 0.04 **Plot 6: "affirmative, negated, conjunctions, disjunctions"** * Trend: Similar to the previous plots, with a slightly lower initial variance. * Data Points (approximate): * PC 1: 0.27 * PC 2: 0.11 * PC 3: 0.07 * PC 4: 0.06 * PC 5: 0.05 * PC 6: 0.04 * PC 7: 0.04 * PC 8: 0.04 * PC 9: 0.04 * PC 10: 0.04 ### Key Observations * All plots exhibit a similar pattern: a rapid decrease in explained variance for the first few PCs, followed by a plateau. * The "affirmative" condition consistently shows the highest explained variance, particularly for PC 1. * Adding negation, conjunctions, disjunctions, and German language elements generally reduces the explained variance, especially for PC 1. * The explained variance for PCs 3-10 is relatively consistent across all conditions. ### Interpretation The plots suggest that the first PC captures a significant amount of variance in the data, particularly for the "affirmative" condition. This implies that the first PC represents a dominant feature or pattern associated with affirmative statements. As linguistic complexity increases (through negation, conjunctions, disjunctions, or the inclusion of German), the explained variance decreases, indicating that these features introduce more variability and require more PCs to capture the underlying structure. The plateau in explained variance for higher PCs suggests that these components contribute less to the overall variance and may represent noise or less salient features. The consistent pattern across conditions suggests a general principle: simpler linguistic structures are more efficiently represented by fewer PCs, while more complex structures require more components.

## Scatter Plot Grid: Fraction of Variance Explained by Principal Components ### Overview The image displays a 2x3 grid of six scatter plots. The overall title is "Fraction of variance in centered and averaged activations explained by PCs." Each subplot shows the "Explained variance" (y-axis) for the first 10 Principal Components (PCs), indexed on the x-axis. The plots compare how the variance structure changes when the underlying data (likely neural activations) is derived from different sets of linguistic conditions. ### Components/Axes * **Overall Title:** "Fraction of variance in centered and averaged activations explained by PCs" * **X-axis (Common to all plots):** "PC index", with major tick marks at 2, 4, 6, 8, and 10. The index runs from 1 to 10. * **Y-axis (Common to all plots):** "Explained variance". The scale varies slightly between plots to accommodate the data range. * **Data Series:** Each plot contains a single data series represented by blue circular markers. There is no legend, as each plot is defined by its title. * **Subplot Titles (Defining the linguistic conditions):** 1. Top-left: "affirmative" 2. Top-center: "affirmative, negated" 3. Top-right: "affirmative, negated, conjunctions" 4. Bottom-left: "affirmative, affirmative German" 5. Bottom-center: "affirmative, affirmative German, negated, negated German" 6. Bottom-right: "affirmative, negated, conjunctions, disjunctions" ### Detailed Analysis **Trend Verification:** All six plots exhibit the same fundamental trend: a steep, monotonic decrease in explained variance as the PC index increases. The first PC explains the vast majority of the variance, with a sharp drop to the second PC, followed by a more gradual decline. **Data Point Extraction (Approximate values):** * **Plot 1: "affirmative"** * PC1: ~0.61 * PC2: ~0.14 * PC3: ~0.10 * PC4: ~0.08 * PC5: ~0.04 * PC6: ~0.03 * PCs 7-10: All near 0.00 * **Plot 2: "affirmative, negated"** * PC1: ~0.33 * PC2: ~0.28 * PC3: ~0.09 * PC4: ~0.07 * PC5: ~0.06 * PC6: ~0.05 * PC7: ~0.03 * PC8: ~0.02 * PC9: ~0.01 * PC10: ~0.01 * **Plot 3: "affirmative, negated, conjunctions"** * PC1: ~0.33 * PC2: ~0.24 * PC3: ~0.08 * PC4: ~0.07 * PC5: ~0.06 * PC6: ~0.05 * PC7: ~0.04 * PC8: ~0.03 * PC9: ~0.02 * PC10: ~0.02 * **Plot 4: "affirmative, affirmative German"** * PC1: ~0.55 * PC2: ~0.13 * PC3: ~0.08 * PC4: ~0.07 * PC5: ~0.06 * PC6: ~0.03 * PC7: ~0.02 * PC8: ~0.02 * PC9: ~0.01 * PC10: ~0.01 * **Plot 5: "affirmative, affirmative German, negated, negated German"** * PC1: ~0.29 * PC2: ~0.25 * PC3: ~0.09 * PC4: ~0.05 * PC5: ~0.05 * PC6: ~0.04 * PC7: ~0.03 * PC8: ~0.02 * PC9: ~0.02 * PC10: ~0.01 * **Plot 6: "affirmative, negated, conjunctions, disjunctions"** * PC1: ~0.32 * PC2: ~0.23 * PC3: ~0.08 * PC4: ~0.07 * PC5: ~0.06 * PC6: ~0.05 * PC7: ~0.04 * PC8: ~0.03 * PC9: ~0.02 * PC10: ~0.02 ### Key Observations 1. **Dominance of PC1:** In all conditions, the first principal component explains a disproportionately large fraction of the total variance. 2. **Impact of Condition Set Complexity:** As more linguistic conditions are added (moving from "affirmative" alone to combinations including negation, conjunctions, disjunctions, and German translations), the variance explained by PC1 decreases significantly (from ~0.61 to ~0.29-0.33). This suggests the data's structure becomes more distributed across dimensions. 3. **Two-Component Structure in Complex Sets:** In plots 2, 3, 5, and 6, PC1 and PC2 together explain a substantial portion of the variance (e.g., ~0.61 in Plot 2, ~0.57 in Plot 3), indicating a possible two-dimensional core structure when negation is involved. 4. **Rapid Dimensionality Reduction:** Across all plots, the explained variance drops to near zero by PC7-10, indicating that the meaningful variance in these centered and averaged activations is captured by a very low-dimensional subspace (likely 2-6 dimensions). ### Interpretation This analysis reveals how the representational geometry of a system (likely a language model) changes with the complexity of the linguistic input. The "affirmative"-only condition has a highly concentrated representation dominated by a single axis of variation. Introducing negation ("affirmative, negated") splits this variance more evenly between two primary axes, suggesting the model encodes affirmation and negation as somewhat distinct but related dimensions. Adding further logical operations (conjunctions, disjunctions) or cross-lingual data (German) does not dramatically alter this two-axis structure established by negation, but it does slightly redistribute the variance. The consistent, rapid drop-off after the first few PCs is a classic signature of a low-dimensional manifold underlying the high-dimensional activation space. This implies that despite the complexity of language, the core variations in how the model processes these specific sentence types can be understood through a small number of interpretable components. The data suggests that negation is a fundamental axis of variation in the model's internal representations, more so than logical connectives or translation.

## Scatter Plot Grid: Fraction of Variance in Centered and Averaged Activations Explained by PCs ### Overview The image displays a 2x3 grid of scatter plots comparing the relationship between principal component (PC) index and explained variance across six linguistic feature configurations. Each plot uses blue data points to show how variance decreases with increasing PC index, with distinct configurations labeled in the plot titles. ### Components/Axes - **Main Title**: "Fraction of variance in centered and averaged activations explained by PCs" (top center) - **X-axis**: "PC index" (1–10) across all plots - **Y-axis**: "Explained variance" (0–0.6) across all plots - **Subplot Titles** (top of each plot): 1. Top-left: "affirmative" 2. Top-middle: "affirmative, negated" 3. Top-right: "affirmative, negated, conjunctions" 4. Bottom-left: "affirmative, affirmative German" 5. Bottom-middle: "affirmative, affirmative German, negated, negated German" 6. Bottom-right: "affirmative, negated, conjunctions, disjunctions" - **No explicit legend** present ### Detailed Analysis 1. **Affirmative (Top-left)**: - Highest initial variance (~0.6 at PC 1) - Sharp decline to near-zero by PC 10 - Data points: (1, 0.6), (2, 0.15), (3–10, <0.05) 2. **Affirmative, Negated (Top-middle)**: - Initial variance ~0.3 at PC 1 - Gradual decline to ~0.05 by PC 10 - Data points: (1, 0.3), (2, 0.08), (3–10, <0.05) 3. **Affirmative, Negated, Conjunctions (Top-right)**: - Initial variance ~0.3 at PC 1 - Steeper decline than top-middle, reaching ~0.03 by PC 10 - Data points: (1, 0.3), (2, 0.06), (3–10, <0.03) 4. **Affirmative, Affirmative German (Bottom-left)**: - Initial variance ~0.45 at PC 1 - Moderate decline to ~0.03 by PC 10 - Data points: (1, 0.45), (2, 0.1), (3–10, <0.03) 5. **Affirmative, Affirmative German, Negated, Negated German (Bottom-middle)**: - Initial variance ~0.3 at PC 1 - Gradual decline to ~0.02 by PC 10 - Data points: (1, 0.3), (2, 0.07), (3–10, <0.02) 6. **Affirmative, Negated, Conjunctions, Disjunctions (Bottom-right)**: - Initial variance ~0.3 at PC 1 - Steeper decline than top-right, reaching ~0.02 by PC 10 - Data points: (1, 0.3), (2, 0.05), (3–10, <0.02) ### Key Observations - All configurations show **monotonic decrease** in explained variance with increasing PC index - **First PC** consistently explains the largest variance (0.3–0.6 range) - **German-specific configurations** (bottom row) show slightly lower initial variance than English-only configurations - **Complex configurations** (e.g., conjunctions/disjunctions) exhibit faster variance decay - No plot shows variance exceeding 0.6 or falling below 0.0 ### Interpretation The data demonstrates classic principal component analysis (PCA) behavior, where early components capture dominant patterns. The linguistic feature configurations suggest: 1. **Core semantics** (affirmative/negated) explain the largest variance 2. **Syntactic features** (conjunctions/disjunctions) add diminishing explanatory power 3. **Language-specific variants** (German) reduce overall variance explained 4. The consistent decay pattern across all plots implies a shared dimensional structure in the activation data, with linguistic features modulating the strength of early components. The absence of variance beyond PC 10 suggests the dataset may be well-represented by the first 10 principal components.