## Scatter Plot Matrix: Variance Explained by Principal Components ### Overview The image presents a matrix of six scatter plots. Each plot shows the fraction of variance in centered and averaged activations explained by principal components (PCs) for different linguistic conditions. The x-axis represents the PC index (from 1 to 10), and the y-axis represents the explained variance (from 0.0 to either 0.6 or 0.3, depending on the plot). All data points are blue circles. Each plot corresponds to a different combination of affirmative, negated, conjunction, disjunction, and German language conditions. ### Components/Axes * **Title:** "Fraction of variance in centered and averaged activations explained by PCs" * **X-axis (PC index):** Ranges from 1 to 10, with integer markers at each value. * **Y-axis (Explained variance):** Ranges from 0.0 to 0.6 for the "affirmative" plot and from 0.0 to 0.3 for the other plots, with markers at 0.0, 0.1, 0.2, 0.3, 0.4, 0.6. * **Plot Titles (from top-left to bottom-right):** * "affirmative" * "affirmative, negated" * "affirmative, negated, conjunctions" * "affirmative, affirmative German" * "affirmative, affirmative German, negated, negated German" * "affirmative, negated, conjunctions, disjunctions" ### Detailed Analysis Each plot shows a decreasing trend in explained variance as the PC index increases. The first PC explains the most variance, and subsequent PCs explain progressively less. * **Affirmative:** * PC 1 explains approximately 0.61 of the variance. * PC 2 explains approximately 0.14 of the variance. * PC 3 explains approximately 0.11 of the variance. * PC 4 explains approximately 0.07 of the variance. * PC 5 explains approximately 0.05 of the variance. * PC 6 explains approximately 0.03 of the variance. * PC 7 explains approximately 0.01 of the variance. * PC 8 explains approximately 0.01 of the variance. * PC 9 explains approximately 0.00 of the variance. * PC 10 explains approximately 0.00 of the variance. * **Affirmative, Negated:** * PC 1 explains approximately 0.33 of the variance. * PC 2 explains approximately 0.29 of the variance. * PC 3 explains approximately 0.09 of the variance. * PC 4 explains approximately 0.06 of the variance. * PC 5 explains approximately 0.05 of the variance. * PC 6 explains approximately 0.04 of the variance. * PC 7 explains approximately 0.03 of the variance. * PC 8 explains approximately 0.03 of the variance. * PC 9 explains approximately 0.02 of the variance. * PC 10 explains approximately 0.02 of the variance. * **Affirmative, Negated, Conjunctions:** * PC 1 explains approximately 0.33 of the variance. * PC 2 explains approximately 0.24 of the variance. * PC 3 explains approximately 0.07 of the variance. * PC 4 explains approximately 0.07 of the variance. * PC 5 explains approximately 0.05 of the variance. * PC 6 explains approximately 0.04 of the variance. * PC 7 explains approximately 0.03 of the variance. * PC 8 explains approximately 0.03 of the variance. * PC 9 explains approximately 0.02 of the variance. * PC 10 explains approximately 0.02 of the variance. * **Affirmative, Affirmative German:** * PC 1 explains approximately 0.48 of the variance. * PC 2 explains approximately 0.13 of the variance. * PC 3 explains approximately 0.09 of the variance. * PC 4 explains approximately 0.06 of the variance. * PC 5 explains approximately 0.04 of the variance. * PC 6 explains approximately 0.03 of the variance. * PC 7 explains approximately 0.02 of the variance. * PC 8 explains approximately 0.01 of the variance. * PC 9 explains approximately 0.01 of the variance. * PC 10 explains approximately 0.01 of the variance. * **Affirmative, Affirmative German, Negated, Negated German:** * PC 1 explains approximately 0.29 of the variance. * PC 2 explains approximately 0.08 of the variance. * PC 3 explains approximately 0.06 of the variance. * PC 4 explains approximately 0.04 of the variance. * PC 5 explains approximately 0.04 of the variance. * PC 6 explains approximately 0.03 of the variance. * PC 7 explains approximately 0.02 of the variance. * PC 8 explains approximately 0.02 of the variance. * PC 9 explains approximately 0.01 of the variance. * PC 10 explains approximately 0.01 of the variance. * **Affirmative, Negated, Conjunctions, Disjunctions:** * PC 1 explains approximately 0.33 of the variance. * PC 2 explains approximately 0.23 of the variance. * PC 3 explains approximately 0.07 of the variance. * PC 4 explains approximately 0.07 of the variance. * PC 5 explains approximately 0.04 of the variance. * PC 6 explains approximately 0.04 of the variance. * PC 7 explains approximately 0.03 of the variance. * PC 8 explains approximately 0.02 of the variance. * PC 9 explains approximately 0.02 of the variance. * PC 10 explains approximately 0.02 of the variance. ### Key Observations * The "affirmative" condition has the highest explained variance by the first PC (approximately 0.61), indicating that the first principal component captures a larger portion of the variance in this condition compared to the others. * The addition of negation, conjunctions, and disjunctions generally reduces the explained variance by the first PC. * The inclusion of German language data also appears to reduce the explained variance by the first PC. * In all conditions, the explained variance decreases rapidly after the first few PCs, suggesting that the first few components are the most important for capturing the variance in the data. ### Interpretation The data suggests that the linguistic conditions (affirmation, negation, conjunction, disjunction, and language) influence the variance explained by the principal components of the activations. The "affirmative" condition, without any additional modifiers, exhibits the highest explained variance by the first PC, implying that the primary mode of variation in the data is strongly related to affirmative statements. The addition of negation, conjunctions, disjunctions, and German language data introduces more complexity, leading to a distribution of variance across multiple principal components. This could indicate that these conditions introduce additional dimensions of variation in the neural activations. The rapid decrease in explained variance after the first few PCs across all conditions suggests that the underlying data has a relatively low intrinsic dimensionality, with most of the variance captured by the first few principal components.

## Scatter Plots: Fraction of Variance Explained by PCs ### Overview The image presents six scatter plots, each visualizing the fraction of variance in centered and averaged activations explained by Principal Components (PCs). Each plot corresponds to a different linguistic condition. The x-axis represents the PC index (ranging from 1 to 10), and the y-axis represents the explained variance (ranging from 0 to approximately 0.6). ### Components/Axes * **Title:** "Fraction of variance in centered and averaged activations explained by PCs" (centered at the top) * **X-axis Label:** "PC index" (present on all plots) * **Y-axis Label:** "Explained variance" (present on all plots) * **Plots (from top-left to bottom-right):** 1. "affirmative" 2. "affirmative, negated" 3. "affirmative, negated, conjunctions" 4. "affirmative, affirmative German" 5. "affirmative, affirmative German, negated, negated German" 6. "affirmative, negated, conjunctions, disjunctions" * **Axis Scales:** Both axes appear to be linear. The x-axis ranges from 1 to 10. The y-axis ranges from 0 to approximately 0.6. * **Data Points:** Each plot contains approximately 10 data points, represented as blue circles. ### Detailed Analysis or Content Details **Plot 1: "affirmative"** * Trend: The explained variance initially peaks at PC index 1 and then declines rapidly. * Data Points (approximate): * PC 1: 0.55 * PC 2: 0.08 * PC 3: 0.03 * PC 4: 0.02 * PC 5: 0.01 * PC 6: 0.01 * PC 7: 0.01 * PC 8: 0.01 * PC 9: 0.01 * PC 10: 0.01 **Plot 2: "affirmative, negated"** * Trend: Similar to the first plot, the explained variance peaks at PC index 1 and then declines. * Data Points (approximate): * PC 1: 0.32 * PC 2: 0.07 * PC 3: 0.04 * PC 4: 0.03 * PC 5: 0.02 * PC 6: 0.02 * PC 7: 0.01 * PC 8: 0.01 * PC 9: 0.01 * PC 10: 0.01 **Plot 3: "affirmative, negated, conjunctions"** * Trend: Peaks at PC index 1, then declines. * Data Points (approximate): * PC 1: 0.32 * PC 2: 0.06 * PC 3: 0.04 * PC 4: 0.03 * PC 5: 0.02 * PC 6: 0.02 * PC 7: 0.01 * PC 8: 0.01 * PC 9: 0.01 * PC 10: 0.01 **Plot 4: "affirmative, affirmative German"** * Trend: Peaks at PC index 1, then declines. * Data Points (approximate): * PC 1: 0.42 * PC 2: 0.06 * PC 3: 0.03 * PC 4: 0.02 * PC 5: 0.01 * PC 6: 0.01 * PC 7: 0.01 * PC 8: 0.01 * PC 9: 0.01 * PC 10: 0.01 **Plot 5: "affirmative, affirmative German, negated, negated German"** * Trend: Peaks at PC index 1, then declines. * Data Points (approximate): * PC 1: 0.32 * PC 2: 0.08 * PC 3: 0.04 * PC 4: 0.03 * PC 5: 0.02 * PC 6: 0.02 * PC 7: 0.01 * PC 8: 0.01 * PC 9: 0.01 * PC 10: 0.01 **Plot 6: "affirmative, negated, conjunctions, disjunctions"** * Trend: Peaks at PC index 1, then declines. * Data Points (approximate): * PC 1: 0.32 * PC 2: 0.06 * PC 3: 0.04 * PC 4: 0.03 * PC 5: 0.02 * PC 6: 0.02 * PC 7: 0.01 * PC 8: 0.01 * PC 9: 0.01 * PC 10: 0.01 ### Key Observations * All plots exhibit a similar pattern: a steep decline in explained variance after the first PC. * The "affirmative" condition shows the highest explained variance for the first PC (approximately 0.55). * The remaining conditions have lower explained variance for the first PC, ranging from approximately 0.32 to 0.42. * The explained variance for PCs 2 through 10 is consistently low across all conditions. ### Interpretation The data suggests that the first Principal Component captures a significant portion of the variance in the centered and averaged activations for all linguistic conditions. This indicates that there is a dominant underlying factor influencing the data. The differences in explained variance for the first PC across conditions suggest that this dominant factor may be modulated by the specific linguistic context (affirmative, negated, conjunctions, disjunctions, and German). The rapid decline in explained variance for subsequent PCs indicates that these components contribute relatively little to explaining the overall variance in the data. The similarity in the patterns across the different conditions suggests a common underlying structure in how these linguistic elements are represented. The higher variance explained by the first PC in the "affirmative" condition might indicate a simpler or more consistent representation of affirmative statements compared to the other, more complex linguistic structures.

\n ## Scatter Plot Grid: Fraction of Variance Explained by Principal Components (PCs) ### Overview The image displays a 2x3 grid of six scatter plots. The collective title is "Fraction of variance in centered and averaged activations explained by PCs". Each subplot shows the explained variance (y-axis) for the first 10 principal components (x-axis) for different combinations of linguistic conditions. The plots share a common visual style: blue circular data points on a white background with gray grid lines. ### Components/Axes * **Main Title:** "Fraction of variance in centered and averaged activations explained by PCs" * **Common Y-axis Label (Left side of grid):** "Explained variance" * **Common X-axis Label (Bottom of grid):** "PC index" * **Subplot Titles (Top of each plot):** 1. Top-left: "affirmative" 2. Top-middle: "affirmative, negated" 3. Top-right: "affirmative, negated, conjunctions" 4. Bottom-left: "affirmative, affirmative German" 5. Bottom-middle: "affirmative, affirmative German, negated, negated German" 6. Bottom-right: "affirmative, negated, conjunctions, disjunctions" * **Axes Scales:** * **X-axis (PC index):** Linear scale from 1 to 10, with major ticks at 2, 4, 6, 8, 10. * **Y-axis (Explained variance):** Linear scale. The range varies by subplot: * "affirmative": 0.0 to 0.6 * All other subplots: 0.0 to 0.3 or 0.0 to 0.4 (see detailed analysis). ### Detailed Analysis **Trend Verification:** In all six subplots, the data series follows the same fundamental trend: a steep, monotonic decrease in explained variance from PC1 to PC2, followed by a more gradual, asymptotic decline towards zero by PC10. This is the classic "scree plot" pattern expected from PCA. **Subplot 1: "affirmative" (Top-left)** * **Y-axis Range:** 0.0 to 0.6. * **Approximate Data Points:** * PC1: ~0.60 * PC2: ~0.15 * PC3: ~0.11 * PC4: ~0.07 * PC5: ~0.05 * PC6: ~0.03 * PC7: ~0.02 * PC8: ~0.01 * PC9: ~0.01 * PC10: ~0.01 **Subplot 2: "affirmative, negated" (Top-middle)** * **Y-axis Range:** 0.0 to 0.35 (approx). * **Approximate Data Points:** * PC1: ~0.34 * PC2: ~0.30 * PC3: ~0.09 * PC4: ~0.07 * PC5: ~0.06 * PC6: ~0.04 * PC7: ~0.04 * PC8: ~0.03 * PC9: ~0.02 * PC10: ~0.02 **Subplot 3: "affirmative, negated, conjunctions" (Top-right)** * **Y-axis Range:** 0.0 to 0.35 (approx). * **Approximate Data Points:** * PC1: ~0.34 * PC2: ~0.25 * PC3: ~0.08 * PC4: ~0.07 * PC5: ~0.06 * PC6: ~0.05 * PC7: ~0.04 * PC8: ~0.04 * PC9: ~0.03 * PC10: ~0.03 **Subplot 4: "affirmative, affirmative German" (Bottom-left)** * **Y-axis Range:** 0.0 to 0.5 (approx). * **Approximate Data Points:** * PC1: ~0.50 * PC2: ~0.13 * PC3: ~0.10 * PC4: ~0.07 * PC5: ~0.05 * PC6: ~0.03 * PC7: ~0.03 * PC8: ~0.02 * PC9: ~0.02 * PC10: ~0.02 **Subplot 5: "affirmative, affirmative German, negated, negated German" (Bottom-middle)** * **Y-axis Range:** 0.0 to 0.3. * **Approximate Data Points:** * PC1: ~0.29 * PC2: ~0.28 * PC3: ~0.09 * PC4: ~0.06 * PC5: ~0.05 * PC6: ~0.04 * PC7: ~0.03 * PC8: ~0.03 * PC9: ~0.02 * PC10: ~0.02 **Subplot 6: "affirmative, negated, conjunctions, disjunctions" (Bottom-right)** * **Y-axis Range:** 0.0 to 0.35 (approx). * **Approximate Data Points:** * PC1: ~0.33 * PC2: ~0.24 * PC3: ~0.08 * PC4: ~0.07 * PC5: ~0.05 * PC6: ~0.05 * PC7: ~0.04 * PC8: ~0.04 * PC9: ~0.03 * PC10: ~0.03 ### Key Observations 1. **Dominance of PC1:** The first principal component (PC1) consistently explains the largest fraction of variance in every condition, ranging from ~0.29 to ~0.60. 2. **Impact of Condition Complexity:** Adding more linguistic conditions (negation, conjunctions, disjunctions, translations) generally reduces the variance explained by PC1. The "affirmative" only condition has the highest PC1 value (~0.60), while the most complex condition (bottom-middle) has the lowest (~0.29). 3. **Two-Component Structure:** In several plots ("affirmative, negated"; "affirmative, affirmative German, negated, negated German"), PC1 and PC2 explain nearly equal, substantial portions of variance, suggesting a strong two-dimensional structure in the underlying data for those conditions. 4. **Rapid Drop-off:** After the first 2-3 components, the explained variance per component becomes very small (<0.10) and decays slowly, indicating that most meaningful variance is captured by the top few PCs. ### Interpretation This figure presents a **scree analysis** of principal components applied to neural activation patterns under different linguistic manipulations. The data suggests: * **Core Semantic Dimension:** The high variance explained by PC1, especially in the simple "affirmative" condition, likely corresponds to a primary, dominant axis of meaning or representation in the model's activations (e.g., a general "semantic strength" or "activation magnitude" dimension). * **Effect of Linguistic Operations:** Introducing negation ("affirmative, negated") dramatically splits the variance between PC1 and PC2. This implies that negation creates a second major, orthogonal axis of variation in the activation space, possibly representing a "truth value" or "polarity" dimension. * **Cross-Linguistic Stability:** The pattern for "affirmative, affirmative German" closely mirrors the English-only "affirmative" plot, suggesting the core representational structure is stable across these two languages for affirmative statements. * **Increased Dimensionality with Complexity:** As more logical operations (conjunctions, disjunctions) and language variants are combined, the variance becomes slightly more distributed across the first few components, but the overall scree shape remains. This indicates that while the representational space becomes more nuanced, it is still dominated by a small number of principal directions. In essence, the visualization demonstrates how the intrinsic dimensionality of a model's semantic representation, as captured by PCA, expands and reorganizes when processing increasingly complex linguistic constructs.

## Grid of Scatter Plots: Fraction of Variance in Centered and Averaged Activations Explained by PCs ### Overview The image contains six scatter plots arranged in two rows of three, each visualizing the relationship between principal component (PC) index and explained variance under different linguistic conditions. All plots share consistent axes but vary in their experimental conditions (e.g., "affirmative," "affirmative, negated," etc.). Data points are represented by blue dots, with no visible legend or colorbar. --- ### Components/Axes - **X-axis**: "PC index" (ranges from 2 to 10 in all plots) - **Y-axis**: "Explained variance" (scales vary: 0–0.6 in top-left, 0–0.3 in others) - **Chart Titles**: Positioned at the top of each plot, specifying conditions (e.g., "affirmative," "affirmative, negated, conjunctions"). - **Data Points**: Blue dots with no explicit markers or error bars. --- ### Detailed Analysis #### Top Row 1. **affirmative** - Highest explained variance at PC 2 (~0.6). - Sharp decline to near-zero by PC 6. - Minimal variance for PCs 8–10. 2. **affirmative, negated** - Explained variance starts at ~0.3 (PC 2). - Gradual decline to ~0.05 by PC 6. - Flattens near zero for PCs 8–10. 3. **affirmative, negated, conjunctions** - Starts at ~0.3 (PC 2). - Steeper decline to ~0.05 by PC 4. - Remains near zero for PCs 6–10. #### Bottom Row 4. **affirmative, affirmative German** - Explained variance at PC 2 (~0.45). - Gradual decline to ~0.05 by PC 6. - Flattens near zero for PCs 8–10. 5. **affirmative, affirmative German, negated German** - Starts at ~0.3 (PC 2). - Slightly steeper decline than Plot 4. - Flattens near zero by PC 6. 6. **affirmative, negated, conjunctions, disjunctions** - Lowest starting variance (~0.3 at PC 2). - Gradual decline to ~0.05 by PC 4. - Flattens near zero for PCs 6–10. --- ### Key Observations 1. **Decline with PC Index**: All plots show a consistent drop in explained variance as PC index increases, suggesting diminishing returns in variance explanation with higher PCs. 2. **Condition Complexity**: - Simplest condition ("affirmative") explains the most variance (~0.6). - Adding negations, conjunctions, or disjunctions reduces explained variance. - German language variants show slightly lower variance than English equivalents. 3. **Conjunctions/Disjunctions Impact**: The final plot (most complex conditions) has the lowest variance, indicating these syntactic structures contribute minimally to activation patterns. --- ### Interpretation The data suggests that neural activations are most strongly driven by simple affirmative statements, with complexity (negation, conjunctions, disjunctions) reducing the proportion of variance explained by principal components. This implies that higher-order syntactic structures have a smaller impact on the neural representations captured by PCA. The German language variants follow a similar trend but with slightly lower variance, potentially reflecting language-specific processing differences. The consistent decline across all plots highlights the dominance of low-dimensional PCs in explaining activation patterns, with higher PCs contributing minimally.