## Scatter Plot Matrix: Fraction of Variance Explained by PCs ### Overview The image presents a 2x3 matrix of scatter plots. Each plot shows the fraction of variance in centered and averaged activations explained by Principal Components (PCs) for different linguistic conditions. The x-axis represents the PC index, ranging from 1 to 10, and the y-axis represents the explained variance, ranging from 0 to 0.4 or 0.3 depending on the plot. Each plot corresponds to a different combination of affirmative, negated, conjunction, disjunction, and German language conditions. ### Components/Axes * **Title:** Fraction of variance in centered and averaged activations explained by PCs * **X-axis (PC index):** * Label: PC index * Scale: 1 to 10, with tick marks at each integer value. * **Y-axis (Explained variance):** * Label: Explained variance * Scale: Varies slightly between plots. Some go from 0 to 0.4, others from 0 to 0.3. Tick marks are present at intervals of 0.1. * **Plot Titles (Conditions):** * Top-left: affirmative * Top-middle: affirmative, negated * Top-right: affirmative, negated, conjunctions * Bottom-left: affirmative, affirmative German * Bottom-middle: affirmative, affirmative German, negated, negated German * Bottom-right: affirmative, negated, conjunctions, disjunctions ### Detailed Analysis Each scatter plot displays a similar trend: the explained variance decreases as the PC index increases. The first few PCs explain a significant portion of the variance, while the later PCs explain progressively less. **Plot 1: affirmative** * The explained variance for PC 1 is approximately 0.48. * The explained variance for PC 2 is approximately 0.28. * The explained variance for PC 3 is approximately 0.11. * The explained variance for PC 4 is approximately 0.04. * The explained variance for PC 10 is approximately 0.00. **Plot 2: affirmative, negated** * The explained variance for PC 1 is approximately 0.29. * The explained variance for PC 2 is approximately 0.24. * The explained variance for PC 3 is approximately 0.14. * The explained variance for PC 4 is approximately 0.06. * The explained variance for PC 10 is approximately 0.01. **Plot 3: affirmative, negated, conjunctions** * The explained variance for PC 1 is approximately 0.28. * The explained variance for PC 2 is approximately 0.24. * The explained variance for PC 3 is approximately 0.13. * The explained variance for PC 4 is approximately 0.06. * The explained variance for PC 10 is approximately 0.01. **Plot 4: affirmative, affirmative German** * The explained variance for PC 1 is approximately 0.46. * The explained variance for PC 2 is approximately 0.30. * The explained variance for PC 3 is approximately 0.10. * The explained variance for PC 4 is approximately 0.04. * The explained variance for PC 10 is approximately 0.01. **Plot 5: affirmative, affirmative German, negated, negated German** * The explained variance for PC 1 is approximately 0.29. * The explained variance for PC 2 is approximately 0.28. * The explained variance for PC 3 is approximately 0.14. * The explained variance for PC 4 is approximately 0.05. * The explained variance for PC 10 is approximately 0.01. **Plot 6: affirmative, negated, conjunctions, disjunctions** * The explained variance for PC 1 is approximately 0.31. * The explained variance for PC 2 is approximately 0.24. * The explained variance for PC 3 is approximately 0.12. * The explained variance for PC 4 is approximately 0.06. * The explained variance for PC 10 is approximately 0.01. ### Key Observations * The first principal component (PC1) consistently explains the largest fraction of variance across all conditions. * The explained variance decreases rapidly for the first few PCs and then plateaus. * The "affirmative" and "affirmative, affirmative German" conditions show the highest explained variance by PC1, around 0.48 and 0.46 respectively. * The addition of negation, conjunctions, and disjunctions generally reduces the explained variance by PC1. ### Interpretation The plots demonstrate how much of the variance in neural activations can be captured by a small number of principal components under different linguistic conditions. The fact that the first few PCs explain most of the variance suggests that the neural representations are relatively low-dimensional. The differences in explained variance across conditions indicate that certain linguistic features (e.g., affirmation vs. negation) influence the structure of the neural representations. Specifically, affirmative sentences seem to have a more structured representation (higher variance explained by PC1) compared to sentences with negation, conjunctions, or disjunctions. The inclusion of German language data also appears to affect the explained variance, suggesting cross-linguistic differences in neural representation.

## Scatter Plots: Fraction of Variance Explained by PCs ### Overview The image presents six scatter plots, each representing the fraction of variance in centered and averaged activations explained by Principal Components (PCs). Each plot corresponds to a different linguistic condition. The x-axis represents the PC index (ranging from 1 to 10), and the y-axis represents the explained variance (ranging from 0 to approximately 0.4). ### Components/Axes * **Title:** "Fraction of variance in centered and averaged activations explained by PCs" (centered at the top) * **X-axis Label:** "PC index" (present on all plots) * **Y-axis Label:** "Explained variance" (present on all plots) * **Plots (from top-left to bottom-right):** 1. "affirmative" 2. "affirmative, negated" 3. "affirmative, negated, conjunctions" 4. "affirmative, affirmative German" 5. "affirmative, affirmative German, negated, negated German" 6. "affirmative, negated, conjunctions, disjunctions" * **Gridlines:** Present in all plots, providing a visual reference for data point positioning. ### Detailed Analysis or Content Details **Plot 1: "affirmative"** * Trend: The explained variance initially decreases rapidly, then plateaus. * Data Points (approximate): * PC Index 1: 0.38 * PC Index 2: 0.26 * PC Index 3: 0.14 * PC Index 4: 0.08 * PC Index 5: 0.04 * PC Index 6: 0.02 * PC Index 7: 0.01 * PC Index 8: 0.01 * PC Index 9: 0.005 * PC Index 10: 0.005 **Plot 2: "affirmative, negated"** * Trend: Similar to Plot 1, with a rapid initial decrease followed by a plateau, but generally lower variance explained. * Data Points (approximate): * PC Index 1: 0.32 * PC Index 2: 0.16 * PC Index 3: 0.08 * PC Index 4: 0.05 * PC Index 5: 0.03 * PC Index 6: 0.02 * PC Index 7: 0.01 * PC Index 8: 0.01 * PC Index 9: 0.005 * PC Index 10: 0.005 **Plot 3: "affirmative, negated, conjunctions"** * Trend: A similar decreasing trend, but with even lower explained variance. * Data Points (approximate): * PC Index 1: 0.35 * PC Index 2: 0.12 * PC Index 3: 0.06 * PC Index 4: 0.04 * PC Index 5: 0.03 * PC Index 6: 0.02 * PC Index 7: 0.01 * PC Index 8: 0.01 * PC Index 9: 0.005 * PC Index 10: 0.005 **Plot 4: "affirmative, affirmative German"** * Trend: Similar to Plot 1, with a rapid initial decrease followed by a plateau. * Data Points (approximate): * PC Index 1: 0.42 * PC Index 2: 0.28 * PC Index 3: 0.08 * PC Index 4: 0.04 * PC Index 5: 0.02 * PC Index 6: 0.01 * PC Index 7: 0.01 * PC Index 8: 0.005 * PC Index 9: 0.005 * PC Index 10: 0.005 **Plot 5: "affirmative, affirmative German, negated, negated German"** * Trend: Similar to Plot 2, with a rapid initial decrease followed by a plateau, but generally lower variance explained. * Data Points (approximate): * PC Index 1: 0.34 * PC Index 2: 0.18 * PC Index 3: 0.07 * PC Index 4: 0.05 * PC Index 5: 0.03 * PC Index 6: 0.02 * PC Index 7: 0.01 * PC Index 8: 0.01 * PC Index 9: 0.005 * PC Index 10: 0.005 **Plot 6: "affirmative, negated, conjunctions, disjunctions"** * Trend: A similar decreasing trend, but with even lower explained variance. * Data Points (approximate): * PC Index 1: 0.36 * PC Index 2: 0.14 * PC Index 3: 0.06 * PC Index 4: 0.04 * PC Index 5: 0.03 * PC Index 6: 0.02 * PC Index 7: 0.01 * PC Index 8: 0.01 * PC Index 9: 0.005 * PC Index 10: 0.005 ### Key Observations * The first PC consistently explains the largest fraction of variance across all conditions. * The explained variance generally decreases rapidly with increasing PC index, indicating that a small number of PCs capture most of the variance. * The "affirmative, affirmative German" condition shows the highest explained variance overall. * The conditions involving negation, conjunctions, and disjunctions tend to have lower explained variance, suggesting that these linguistic features introduce more complexity. ### Interpretation These plots demonstrate how much of the variability in the activations of a neural network (or similar model) can be captured by a set of principal components for different linguistic conditions. The fact that the first PC explains a substantial portion of the variance suggests that there is a dominant mode of variation in the activations. The decreasing explained variance with increasing PC index indicates diminishing returns – each subsequent PC captures less and less of the remaining variability. The differences in explained variance between the conditions suggest that different linguistic features have varying degrees of impact on the model's internal representations. The higher variance explained by the "affirmative, affirmative German" condition might indicate that this condition is more easily represented by the model, or that the model has learned more robust features for this type of input. Conversely, the lower variance explained by conditions involving negation, conjunctions, and disjunctions could suggest that these features require more complex representations, or that the model has not fully learned to capture their nuances. The plots provide insights into the model's ability to encode and process different linguistic structures, and could be used to identify areas where the model might be struggling or where further training is needed.

## Scatter Plot Grid: Fraction of Variance Explained by Principal Components (PCs) ### Overview The image displays a 2x3 grid of six scatter plots. The overall title is "Fraction of variance in centered and averaged activations explained by PCs." Each subplot shows the explained variance (y-axis) for the first 10 principal components (x-axis) for different combinations of linguistic conditions. The data points are blue circles. The plots collectively analyze how variance in activation data is distributed across principal components under varying experimental conditions. ### Components/Axes * **Overall Title:** "Fraction of variance in centered and averaged activations explained by PCs" * **Y-axis Label (Common to all plots):** "Explained variance" * **X-axis Label (Common to all plots):** "PC index" * **X-axis Scale:** Linear scale from 1 to 10, with major ticks at 2, 4, 6, 8, and 10. * **Y-axis Scale:** Linear scale from 0.0 to approximately 0.5, with major ticks at 0.0, 0.1, 0.2, 0.3, and 0.4. The exact upper limit varies slightly per subplot. * **Subplot Titles (Positioned above each plot):** 1. Top-left: "affirmative" 2. Top-center: "affirmative, negated" 3. Top-right: "affirmative, negated, conjunctions" 4. Bottom-left: "affirmative, affirmative German" 5. Bottom-center: "affirmative, affirmative German, negated, negated German" 6. Bottom-right: "affirmative, negated, conjunctions, disjunctions" * **Data Series:** Each plot contains a single data series represented by blue dots. There is no legend, as the title of each subplot defines the data series. ### Detailed Analysis The following table reconstructs the approximate data points for each subplot. Values are estimated from the grid lines. | PC Index | "affirmative" | "affirmative, negated" | "affirmative, negated, conjunctions" | "affirmative, affirmative German" | "affirmative, affirmative German, negated, negated German" | "affirmative, negated, conjunctions, disjunctions" | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **1** | ~0.48 | ~0.33 | ~0.33 | ~0.48 | ~0.30 | ~0.31 | | **2** | ~0.29 | ~0.29 | ~0.26 | ~0.30 | ~0.29 | ~0.25 | | **3** | ~0.10 | ~0.15 | ~0.14 | ~0.09 | ~0.14 | ~0.13 | | **4** | ~0.04 | ~0.06 | ~0.06 | ~0.04 | ~0.06 | ~0.06 | | **5** | ~0.02 | ~0.05 | ~0.04 | ~0.02 | ~0.05 | ~0.05 | | **6** | ~0.02 | ~0.03 | ~0.03 | ~0.02 | ~0.03 | ~0.04 | | **7** | ~0.00 | ~0.02 | ~0.02 | ~0.01 | ~0.03 | ~0.03 | | **8** | ~0.00 | ~0.02 | ~0.02 | ~0.01 | ~0.02 | ~0.03 | | **9** | ~0.00 | ~0.01 | ~0.02 | ~0.00 | ~0.01 | ~0.03 | | **10** | ~0.00 | ~0.01 | ~0.01 | ~0.00 | ~0.01 | ~0.03 | **Trend Verification:** In all six plots, the data series follows the same fundamental trend: a steep, monotonic decrease in explained variance from PC1 to PC3 or PC4, followed by a long tail where the explained variance approaches zero for higher-index PCs. This is the classic "scree plot" pattern expected from PCA. ### Key Observations 1. **Dominance of First PCs:** Across all conditions, the first 2-3 principal components capture the vast majority of the variance (often over 70% combined). The explained variance drops sharply after PC2. 2. **Condition-Dependent Variance Distribution:** * The "affirmative" only condition (top-left) shows the highest variance for PC1 (~0.48) and a very steep drop, suggesting a simpler, more dominant primary pattern. * Adding more conditions (negated, conjunctions, German translations) generally reduces the variance explained by PC1 (to ~0.30-0.33) and slightly flattens the curve, indicating a more complex variance structure spread across more components. * The plot with the most conditions (bottom-right: "affirmative, negated, conjunctions, disjunctions") shows a slightly more gradual decline, with PC10 still explaining a non-negligible fraction (~0.03). 3. **Similarity Between Related Conditions:** The two plots involving German translations (bottom-left and bottom-center) show very similar variance profiles to their English-only counterparts (top-left and top-center, respectively), suggesting the principal components capture language-invariant patterns. ### Interpretation This set of plots performs a **Peircean investigative** analysis of the dimensional structure of neural or computational activation data. The "explained variance" is a sign of how much information (or structure) each principal component captures. * **What the data suggests:** The data demonstrates that the core variance in "centered and averaged activations" for these linguistic tasks is low-dimensional. A few principal components (likely representing fundamental features like sentence polarity, presence of negation, or logical connective type) account for most of the systematic variation. * **How elements relate:** The progression from simple ("affirmative") to complex ("...disjunctions") conditions acts as a controlled experiment. As the linguistic complexity of the input data increases, the variance becomes less concentrated in the very first component and more distributed, though still dominated by the first few. This implies the underlying representational space expands to accommodate the new distinctions. * **Notable anomalies/trends:** The near-zero variance for PCs 7-10 in the simple "affirmative" case is notable; it suggests that beyond a few key features, the remaining dimensions capture noise or irrelevant variation. In contrast, the more complex conditions maintain small but measurable variance in these higher components, indicating they encode meaningful, albeit subtle, information necessary to distinguish the broader set of conditions. * **Why it matters:** This analysis is crucial for dimensionality reduction and understanding model representations. It tells a researcher that they can likely project their high-dimensional activation data into a 3- to 5-dimensional space (using the top PCs) while retaining most of the meaningful signal for these tasks. The differences between plots guide how many components are needed for different experimental scopes.

## Scatter Plot Grid: Fraction of Variance in Centered and Averaged Activations Explained by PCs ### Overview The image contains six scatter plots arranged in a 2x3 grid, each visualizing the relationship between principal component (PC) indices and explained variance for different linguistic conditions. All plots share identical axes but vary in their categorical labels, which describe combinations of linguistic features (e.g., "affirmative," "negated," "conjunctions," "disjunctions") and language-specific terms ("German"). ### Components/Axes - **X-axis**: "PC index" (ranges from 2 to 10 in integer increments). - **Y-axis**: "Explained variance" (ranges from 0 to 0.4 in 0.1 increments). - **Legend**: No explicit legend is present, but all data points are uniformly blue, suggesting a single data series per plot. - **Plot Titles**: Each plot is labeled with a unique combination of linguistic conditions (see "Content Details" below). ### Detailed Analysis #### Plot 1: "affirmative" - **Trend**: A sharp decline in explained variance after PC index 2. - **Data Points**: - PC 2: ~0.45 - PC 3: ~0.1 - PC 4–10: ~0.01–0.02 #### Plot 2: "affirmative, negated" - **Trend**: Moderate decline after PC index 2. - **Data Points**: - PC 2: ~0.3 - PC 3: ~0.15 - PC 4–10: ~0.03–0.05 #### Plot 3: "affirmative, negated, conjunctions" - **Trend**: Similar to Plot 2 but with slightly lower values. - **Data Points**: - PC 2: ~0.3 - PC 3: ~0.15 - PC 4–10: ~0.03–0.05 #### Plot 4: "affirmative, affirmative German" - **Trend**: Steeper decline than Plot 1. - **Data Points**: - PC 2: ~0.45 - PC 3: ~0.25 - PC 4–10: ~0.01–0.02 #### Plot 5: "affirmative, affirmative German, negated, negated German" - **Trend**: Gradual decline after PC index 2. - **Data Points**: - PC 2: ~0.3 - PC 3: ~0.15 - PC 4–10: ~0.03–0.05 #### Plot 6: "affirmative, negated, conjunctions, disjunctions" - **Trend**: Consistent with Plots 2–3 but with slightly lower values. - **Data Points**: - PC 2: ~0.3 - PC 3: ~0.15 - PC 4–10: ~0.03–0.05 ### Key Observations 1. **Dominance of PC 2**: Across all plots, PC index 2 consistently explains the highest variance (~0.3–0.45), suggesting it captures the most significant linguistic patterns. 2. **Impact of Linguistic Complexity**: Adding features like negation, conjunctions, or disjunctions reduces the variance explained by PC 2 and 3, indicating these features may be distributed across later components. 3. **Language-Specific Effects**: Including "German" terms (e.g., "affirmative German") does not significantly alter the trend but slightly lowers variance in PC 3 compared to non-German conditions. 4. **Consistency in Decline**: All plots show a sharp drop in explained variance after PC 3, with minimal variance in PCs 4–10. ### Interpretation The data suggests that the first principal component (PC 2) is the strongest predictor of variance in centered and averaged activations, particularly for affirmative statements. The inclusion of additional linguistic features (negation, conjunctions, disjunctions) or language-specific terms (German) reduces the explanatory power of the first few PCs, implying these elements may be encoded in later components or require more complex representations. The consistent pattern across plots highlights the robustness of PC 2 in capturing core linguistic structures, while subsequent PCs reflect incremental or nuanced variations.