## Heatmap Grid: Attention Weight Modification ### Overview The image presents a 3x3 grid of heatmaps visualizing the attention weight after modifying the Query, Key, and Value components of a model. Each row represents a different component being modified (Query, Key, Value), and each column represents a different value of epsilon (ɛ = 5e-1, 1e-3, 1e-10). The heatmaps display the attention weights, with color intensity ranging from blue (low weight) to red (high weight), as indicated by the colorbar on the right. ### Components/Axes * **Title:** "The Attention Weight after Modifying Query, Key and Value" * **Colorbar (Right):** * Label: "Attention Weight" * Scale: 0.0 to 1.0, incrementing by 0.2. * **Rows (Top to Bottom):** * Row 1: "Modifying Query with ɛ = 5e-1", "Modifying Query with ɛ = 1e-3", "Modifying Query with ɛ = 1e-10" * Row 2: "Modifying Key with ɛ = 5e-1", "Modifying Key with ɛ = 1e-3", "Modifying Key with ɛ = 1e-10" * Row 3: "Modifying Value with ɛ = 5e-1", "Modifying Value with ɛ = 1e-3", "Modifying Value with ɛ = 1e-10" ### Detailed Analysis Each heatmap is a square matrix, presumably representing the attention weights between different elements. * **Modifying Query:** * ɛ = 5e-1: The attention weights are distributed across the matrix, with higher weights along the first column and the diagonal. * ɛ = 1e-3: The attention weights are concentrated in the top-left corner, decreasing towards the bottom-right. * ɛ = 1e-10: The attention weights are even more concentrated in the top-left corner, with a sharper decrease towards the bottom-right. * **Modifying Key:** * ɛ = 5e-1: Similar to modifying the query, the attention weights are distributed, but with a stronger emphasis on the first column. * ɛ = 1e-3: The attention weights are concentrated in the top-left corner, decreasing towards the bottom-right. * ɛ = 1e-10: The attention weights are highly concentrated in the top-left corner. * **Modifying Value:** * ɛ = 5e-1: The attention weights are primarily concentrated in the first column. * ɛ = 1e-3: The attention weights are almost exclusively concentrated in the first column. * ɛ = 1e-10: The attention weights are almost exclusively concentrated in the first column. ### Key Observations * As epsilon (ɛ) decreases (5e-1 to 1e-10), the attention weights become more concentrated. * Modifying the Value component results in a strong focus on the first column, regardless of the epsilon value. * Modifying the Query and Key components show a transition from distributed attention weights to concentrated attention weights in the top-left corner as epsilon decreases. ### Interpretation The heatmaps illustrate how different modifications to the Query, Key, and Value components, controlled by the epsilon parameter, affect the attention weights within a model. The concentration of attention weights in the top-left corner or the first column suggests that certain elements are becoming more dominant in the attention mechanism as epsilon decreases. This could indicate that the model is becoming more selective in its attention, focusing on specific features or elements. The behavior when modifying the Value component suggests that the value representation plays a crucial role in directing attention towards the initial elements. The parameter epsilon (ɛ) seems to control the "sharpness" or focus of the attention mechanism.

\n ## Heatmap: Attention Weight Modification ### Overview This image presents a 3x3 grid of heatmaps, visualizing the attention weight after modifying the Query, Key, and Value components with different epsilon (ε) values. The epsilon values are 5e-1, 1e-3, and 1e-10. Each heatmap displays attention weights on a scale from 0.0 to 1.0, represented by a color gradient from blue to red. ### Components/Axes The image consists of nine individual heatmaps arranged in a 3x3 grid. - **Rows:** Represent the modification applied: "Modifying Query", "Modifying Key", "Modifying Value". - **Columns:** Represent the epsilon (ε) value used for modification: "ε = 5e-1", "ε = 1e-3", "ε = 1e-10". - **Color Scale (Right Side):** Represents the "Attention Weight", ranging from 0.0 (blue) to 1.0 (red). The scale is linear. - **Heatmap Cells:** Each cell represents the attention weight between two elements. The axes of the heatmaps are not explicitly labeled, but appear to represent indices or positions within a sequence. ### Detailed Analysis Each heatmap is a square grid, approximately 10x10 cells. The attention weights are visualized using color intensity. **1. Modifying Query:** - **ε = 5e-1:** The heatmap shows a strong diagonal pattern with high attention weights (red) along the main diagonal. Attention weights decrease as you move away from the diagonal. Approximate values: Diagonal ~ 0.9-1.0, Upper/Lower off-diagonal ~ 0.2-0.4. - **ε = 1e-3:** The diagonal pattern is still present, but less pronounced. Attention weights are generally lower than with ε = 5e-1. Approximate values: Diagonal ~ 0.7-0.9, Upper/Lower off-diagonal ~ 0.3-0.5. - **ε = 1e-10:** The diagonal pattern is significantly weakened. Attention weights are more evenly distributed, with a general value around 0.5-0.6. Approximate values: Diagonal ~ 0.6-0.7, Upper/Lower off-diagonal ~ 0.4-0.6. **2. Modifying Key:** - **ε = 5e-1:** Similar to modifying the query with ε = 5e-1, a strong diagonal pattern is observed. Approximate values: Diagonal ~ 0.8-1.0, Upper/Lower off-diagonal ~ 0.2-0.4. - **ε = 1e-3:** The diagonal pattern is less pronounced, with lower overall attention weights. Approximate values: Diagonal ~ 0.6-0.8, Upper/Lower off-diagonal ~ 0.3-0.5. - **ε = 1e-10:** The diagonal pattern is significantly weakened, with attention weights more evenly distributed. Approximate values: Diagonal ~ 0.5-0.6, Upper/Lower off-diagonal ~ 0.4-0.6. **3. Modifying Value:** - **ε = 5e-1:** A strong diagonal pattern is visible, similar to the query and key modifications with ε = 5e-1. Approximate values: Diagonal ~ 0.8-1.0, Upper/Lower off-diagonal ~ 0.2-0.4. - **ε = 1e-3:** The diagonal pattern is less pronounced, with lower attention weights. Approximate values: Diagonal ~ 0.6-0.8, Upper/Lower off-diagonal ~ 0.3-0.5. - **ε = 1e-10:** The diagonal pattern is significantly weakened, with attention weights more evenly distributed. Approximate values: Diagonal ~ 0.5-0.6, Upper/Lower off-diagonal ~ 0.4-0.6. ### Key Observations - As epsilon (ε) decreases, the strength of the diagonal pattern in the heatmaps diminishes. This suggests that smaller perturbations to the Query, Key, or Value components lead to a more diffuse attention distribution. - The diagonal pattern indicates that the model initially focuses on the relationship between elements at the same position (self-attention). - The color scale shows that attention weights are generally higher when the Query, Key, or Value are modified with larger epsilon values (5e-1). ### Interpretation The data suggests that modifying the Query, Key, or Value components with different epsilon values impacts the attention mechanism. Larger epsilon values (5e-1) preserve a strong self-attention pattern, where the model primarily attends to elements at the same position. Smaller epsilon values (1e-10) disrupt this pattern, leading to a more uniform attention distribution. This could indicate that the model becomes less focused and more sensitive to all elements in the sequence when the components are perturbed with smaller values. The weakening of the diagonal pattern with decreasing epsilon suggests a trade-off between stability and sensitivity. A strong diagonal pattern indicates a stable attention distribution, while a more diffuse pattern suggests a greater sensitivity to input variations. The choice of epsilon value could therefore influence the model's robustness and generalization ability. The image demonstrates the effect of adding noise to the Query, Key, and Value vectors, and how this noise impacts the attention weights.

## Heatmap Grid: Attention Weight after Modifying Query, Key and Value ### Overview The image displays a 3x3 grid of square heatmaps, each visualizing an attention weight matrix. The overall title at the bottom reads: "The Attention Weight after Modifying Query, Key and Value". A vertical color bar on the right side of the grid serves as a legend for the "Attention Weight" scale. ### Components/Axes * **Grid Structure:** 3 rows by 3 columns of individual heatmap plots. * **Row Labels (Modification Type):** * Top Row: "Modifying Query" * Middle Row: "Modifying Key" * Bottom Row: "Modifying Value" * **Column Labels (Epsilon Value):** * Left Column: `ε = 5e-1` (0.5) * Middle Column: `ε = 1e-3` (0.001) * Right Column: `ε = 1e-10` (0.0000000001) * **Color Bar/Legend:** Located on the far right, spanning the full height of the grid. * **Title:** "Attention Weight" * **Scale:** Continuous gradient from 0.0 (bottom) to 1.0 (top). * **Color Mapping:** * 0.0: Dark Blue * ~0.2: Medium Blue * ~0.4: Light Blue / Grayish-Blue * ~0.6: Light Orange / Peach * ~0.8: Orange * 1.0: Dark Red * **Heatmap Axes:** The individual heatmaps do not have labeled x or y axes. They represent a matrix where both dimensions likely correspond to sequence positions (e.g., token indices in a self-attention mechanism). The pattern is a lower-triangular matrix, indicating a causal or autoregressive attention mask where a position can only attend to itself and previous positions. ### Detailed Analysis Each heatmap is a lower-triangular matrix. The color of each cell represents the attention weight from a "query" position (y-axis, row) to a "key" position (x-axis, column). **Row 1: Modifying Query** * **ε = 5e-1 (Top-Left):** Shows a strong, sharp diagonal of high attention weights (red/orange) from the top-left corner. The first column (all rows) also shows moderately high weights (light orange). The rest of the lower triangle is a gradient of blue, with weights decreasing as you move away from the diagonal and the first column. * **ε = 1e-3 (Top-Middle):** The sharp diagonal persists but is slightly less intense. The high-weight region expands into a broader band along the diagonal. The first column remains prominent. The overall pattern is smoother than the ε=5e-1 case. * **ε = 1e-10 (Top-Right):** Very similar to the ε=1e-3 plot. The diagonal band is well-defined and smooth. The distinction between this and the middle plot is minimal, suggesting a saturation effect for very small epsilon. **Row 2: Modifying Key** * **ε = 5e-1 (Middle-Left):** Pattern is strikingly similar to "Modifying Query, ε=5e-1". A sharp diagonal and a prominent first column are visible. * **ε = 1e-3 (Middle-Middle):** Similar to its counterpart in the Query row. A smooth, broad diagonal band of higher attention weights. * **ε = 1e-10 (Middle-Right):** Again, nearly identical to the Query row's ε=1e-10 plot. A well-defined diagonal band. **Row 3: Modifying Value** * **ε = 5e-1 (Bottom-Left):** **This pattern is fundamentally different.** The entire first column is a solid, dark red band (attention weight ≈ 1.0). The rest of the lower triangle is almost entirely dark blue (weight ≈ 0.0), with only a very faint, sparse diagonal of slightly lighter blue cells. * **ε = 1e-3 (Bottom-Middle):** Identical pattern to the ε=5e-1 case for Value modification. Solid red first column, near-zero weights elsewhere. * **ε = 1e-10 (Bottom-Right):** Identical pattern to the other Value modification plots. No visible change with decreasing epsilon. ### Key Observations 1. **Two Distinct Patterns:** The grid reveals two primary attention patterns. Modifications to **Query** and **Key** produce a **diagonal-band pattern**, where attention is focused on recent tokens (the diagonal) and, to a lesser extent, the very first token. Modifications to **Value** produce a **first-column-only pattern**, where all attention is concentrated solely on the first token in the sequence. 2. **Effect of Epsilon (ε):** For Query and Key modifications, decreasing epsilon from 5e-1 to 1e-3 sharpens and smooths the diagonal attention pattern. Further decrease to 1e-10 shows negligible change, indicating the effect plateaus. For Value modifications, epsilon has no visible effect on the resulting pattern. 3. **Causal Mask:** All heatmaps are strictly lower-triangular, confirming the use of a causal (autoregressive) attention mask. No information flows from future positions. 4. **First Token Bias:** Even in the diagonal patterns (Query/Key mods), the first column (attention to the first token) shows consistently higher weights than other non-diagonal positions. ### Interpretation This visualization demonstrates the sensitivity of a transformer's self-attention mechanism to targeted modifications of its core components (Query, Key, Value projections). The results suggest: * **Query/Key Modifications Control "Where to Look":** Altering the Query or Key vectors primarily influences the *distribution* of attention across the sequence. The diagonal pattern indicates that these modifications preserve or enhance the model's tendency for local, sequential processing (attending to the most recent token). The persistence of the first-column bias suggests the first token (often a `[CLS]` or start token) holds inherent importance. * **Value Modifications Control "What is Attended To":** Modifying the Value vectors has a drastic and categorical effect. It collapses the attention distribution, forcing the model to attend *exclusively* to the first token, regardless of the query. This implies that the Value projection is critical for determining the *content* that is aggregated, and disrupting it can lead to a degenerate attention state where only a single, fixed position is used. * **Robustness and Saturation:** The system shows robustness to very small perturbations (ε=1e-3 vs. 1e-10), as the patterns stabilize. The stark difference between the Value modification results and the others highlights a potential asymmetry in how these components contribute to the attention output. This could be relevant for research in model editing, interpretability, or adversarial attacks on transformers.

## Heatmap: Attention Weight Distribution After Parameter Modifications ### Overview The image presents a 3x3 grid of heatmaps visualizing attention weight distributions in a neural network model after modifying three components (Query, Key, Value) with three different perturbation magnitudes (ε = 5e-1, 1e-3, 1e-10). Each panel shows spatial patterns of attention weights using a red-to-blue color gradient, with red indicating higher weights (1.0) and blue indicating lower weights (0.0). ### Components/Axes 1. **Panel Titles** (Top of each heatmap): - Row 1: "Modifying Query with ε = 5e-1", "Modifying Query with ε = 1e-3", "Modifying Query with ε = 1e-10" - Row 2: "Modifying Key with ε = 5e-1", "Modifying Key with ε = 1e-3", "Modifying Key with ε = 1e-10" - Row 3: "Modifying Value with ε = 5e-1", "Modifying Value with ε = 1e-3", "Modifying Value with ε = 1e-10" 2. **Color Scale** (Right side): - Vertical gradient from red (1.0) to blue (0.0) - Label: "Attention Weight" 3. **X-Axis** (Bottom of all panels): - Label: "The Attention Weight after Modifying Query, Key and Value" - Spatial resolution: 10x10 grid (implied by panel structure) ### Detailed Analysis 1. **Query Modifications**: - **ε = 5e-1**: Diagonal red-to-blue gradient (smooth transition) - **ε = 1e-3**: Sharper diagonal red peak with surrounding blue - **ε = 1e-10**: Dominant diagonal red square (near-perfect focus) 2. **Key Modifications**: - Similar pattern to Query but with slightly less intensity in red regions - ε = 5e-1 shows broader red gradient than Query 3. **Value Modifications**: - Most pronounced diagonal focus (especially at ε = 1e-10) - ε = 5e-1 shows strongest red gradient among all panels ### Key Observations 1. **Epsilon Impact**: - Higher ε (5e-1): Uniform distribution (smooth gradients) - Lower ε (1e-10): Sharp diagonal focus (discrete attention) - Intermediate ε (1e-3): Transitional pattern between uniform and focused 2. **Component Sensitivity**: - Value modifications show strongest diagonal focus - Query modifications exhibit most gradual transitions - Key modifications fall between Query and Value in focus intensity 3. **Spatial Patterns**: - All panels show diagonal dominance (positional correlation) - Lower ε values create more pronounced diagonal red squares - Higher ε values produce more diffuse red-to-blue gradients ### Interpretation The data demonstrates that smaller perturbations (lower ε) enable the model to maintain sharper, more focused attention mechanisms, particularly when modifying the Value component. This suggests that parameter stability (low ε) preserves positional specificity in attention weights. Larger perturbations (high ε) introduce noise that regularizes attention distribution, causing more uniform weight allocation across positions. The consistent diagonal patterns across all panels indicate an inherent positional bias in the attention mechanism, which becomes more pronounced under stable parameter conditions. These findings have implications for understanding how parameter regularization affects model interpretability and performance in transformer architectures.