## Heatmap Analysis: Activation Patterns in Llama 3.1 8B and Mistral-NeMo 12B ### Overview This image presents a comparative analysis of activation patterns within the attention mechanisms of two Large Language Models (LLMs): **Llama 3.1 8B** (top row) and **Mistral-NeMo 12B** (bottom row). The data is visualized via heatmaps, segmented into four panels labeled (a), (b), (c), and (d). Each panel compares "Keys" and "Values" projections across "Head Index" (Y-axis) and "Channel Index" (X-axis). The color scale represents normalized values from 0.0 (dark purple) to 1.0 (light yellow). ### Components/Axes * **Common X-Axis:** "Channel Index" ranging from 16 to 128 (increments of 16). * **Y-Axis (Head Index):** * **Llama 3.1 8B (Panels a & b):** Ranges from 1 to 256. * **Mistral-NeMo 12B (Panels c & d):** Ranges from 1 to 320. * **Color Legend:** A gradient bar located below each pair of heatmaps, ranging from 0.0 (dark purple) to 1.0 (light yellow). * **Structure:** * **Left Column (a & c):** "Relative (within head) Absolute Activations." * **Right Column (b & d):** "Relative (within head) Variance of Activations." --- ### Detailed Analysis #### (a) Llama 3.1 8B: Absolute Activations * **Keys:** Displays a distinct, thin vertical column of high activation (yellow/white) consistently located around the 64-80 channel index range. The rest of the heatmap is predominantly dark purple (low activation). * **Values:** Displays horizontal banding patterns. There is no vertical column; instead, activation intensity varies significantly by Head Index (rows), suggesting some heads are consistently more active than others across all channels. #### (b) Llama 3.1 8B: Variance of Activations * **Keys:** Shows a gradient trend where variance increases as the Channel Index increases. The right side of the heatmap (channels 96-128) is significantly brighter (higher variance) than the left side. * **Values:** Similar to the Absolute Activations plot, this shows horizontal banding. The variance is consistent across channels for a given head but varies between heads. #### (c) Mistral-NeMo 12B: Absolute Activations * **Keys:** Similar to Llama 3.1, there is a distinct vertical line of high activation around the 64-80 channel index. The background is largely dark purple. * **Values:** Exhibits horizontal banding. Certain Head Indices show consistently higher activation across all channels, while others remain low. #### (d) Mistral-NeMo 12B: Variance of Activations * **Keys:** Displays a clear gradient trend. Variance is low on the left (channels 16-48) and increases significantly toward the right (channels 96-128). * **Values:** Exhibits horizontal banding, mirroring the pattern seen in the Absolute Activations plot for Values. --- ### Key Observations 1. **Vertical "Spikes" in Keys:** Both models exhibit a consistent vertical feature in the "Keys" heatmaps (Absolute Activations). This suggests that specific channels (likely around the middle of the embedding dimension) are consistently "active" across almost all attention heads. 2. **Horizontal Banding in Values:** The "Values" heatmaps for both models show horizontal stripes. This indicates that activation and variance are determined primarily by the Head Index rather than the Channel Index. In other words, some heads are "high-energy" (high activation/variance) while others are "low-energy," regardless of the channel. 3. **Right-Side Variance Bias in Keys:** In the "Variance" plots (b and d), the "Keys" show a clear trend where variance increases toward the higher channel indices. This suggests that the higher-indexed channels in the Key projection are more dynamic or carry more information variance than the lower-indexed channels. --- ### Interpretation This data provides a window into the internal "specialization" of attention heads in these LLMs. * **The "Key" Specialization:** The vertical lines in the Key projections suggest that these models utilize a specific subset of channels to perform a consistent function across all heads—perhaps a "global" attention feature or a specific positional encoding mechanism that is shared across the entire model architecture. * **The "Value" Specialization:** The horizontal banding in the Value projections suggests that the model architecture assigns different "roles" to different heads. Some heads are consistently more active (or variable) than others. This is a hallmark of multi-head attention, where different heads are expected to attend to different aspects of the input data. * **Variance Gradient:** The fact that variance in Keys increases with the channel index suggests that the models may be using a "coarse-to-fine" or "low-to-high" frequency distribution across their embedding channels. The lower channels appear more static (low variance), while the higher channels are more dynamic (high variance), potentially allowing the model to capture both stable, long-term features and transient, context-specific information simultaneously. In summary, the models appear to have a highly structured internal representation where "Keys" are organized by channel-specific roles (vertical features) and "Values" are organized by head-specific roles (horizontal features).