## Scatter Plots: Top 10 Safety Heads on Undiff Attn. and Scaling Cont. ### Overview The image contains two scatter plots side-by-side. Both plots show the relationship between "Layer" (x-axis) and "Head" (y-axis) for the top 10 safety heads. The left plot is titled "Top 10 Safety Heads on Undiff Attn." and the right plot is titled "Top 10 Safety Heads on Scaling Cont.". Each plot displays data for two models: "Llama-2-7b-chat" (represented by purple circles) and "Vicuna-7b-v1.5" (represented by yellow crosses). A color bar on the right side of each plot indicates "Generalized Ships" values, with the color of each data point corresponding to its "Generalized Ships" value. ### Components/Axes **Left Plot (Undiff Attn.):** * **Title:** Top 10 Safety Heads on Undiff Attn. * **X-axis:** Layer, with ticks from 0 to 30 in increments of 2. * **Y-axis:** Head, with ticks from 0 to 30 in increments of 2. * **Legend (top-right):** * Purple circle: Llama-2-7b-chat * Yellow cross: Vicuna-7b-v1.5 * **Color Bar (right):** Generalized Ships, ranging from approximately 0 to 70. **Right Plot (Scaling Cont.):** * **Title:** Top 10 Safety Heads on Scaling Cont. * **X-axis:** Layer, with ticks from 0 to 30 in increments of 2. * **Y-axis:** Head, with ticks from 0 to 30 in increments of 2. * **Legend (top-right):** * Purple circle: Llama-2-7b-chat * Yellow cross: Vicuna-7b-v1.5 * **Color Bar (right):** Generalized Ships, ranging from approximately 0 to 20. ### Detailed Analysis **Left Plot (Undiff Attn.):** * **Llama-2-7b-chat (Purple Circles):** * Layer ~ 1, Head ~ 2, Generalized Ships ~ 10 * Layer ~ 2, Head ~ 8, Generalized Ships ~ 15 * Layer ~ 3, Head ~ 6, Generalized Ships ~ 10 * Layer ~ 3, Head ~ 15, Generalized Ships ~ 25 * Layer ~ 1, Head ~ 29, Generalized Ships ~ 50 * Layer ~ 26, Head ~ 26, Generalized Ships ~ 60 * **Vicuna-7b-v1.5 (Yellow Crosses):** * Layer ~ 1, Head ~ 8, Generalized Ships ~ 70 * Layer ~ 2, Head ~ 7, Generalized Ships ~ 60 * Layer ~ 2, Head ~ 6, Generalized Ships ~ 60 * Layer ~ 3, Head ~ 1, Generalized Ships ~ 50 * Layer ~ 3, Head ~ 26, Generalized Ships ~ 60 * Layer ~ 6, Head ~ 6, Generalized Ships ~ 40 **Right Plot (Scaling Cont.):** * **Llama-2-7b-chat (Purple Circles):** * Layer ~ 0, Head ~ 15, Generalized Ships ~ 15 * Layer ~ 0, Head ~ 23, Generalized Ships ~ 15 * Layer ~ 0, Head ~ 27, Generalized Ships ~ 15 * Layer ~ 1, Head ~ 8, Generalized Ships ~ 15 * Layer ~ 1, Head ~ 13, Generalized Ships ~ 15 * Layer ~ 13, Head ~ 1, Generalized Ships ~ 5 * Layer ~ 13, Head ~ 4, Generalized Ships ~ 5 * **Vicuna-7b-v1.5 (Yellow Crosses):** * Layer ~ 1, Head ~ 15, Generalized Ships ~ 15 * Layer ~ 5, Head ~ 15, Generalized Ships ~ 15 * Layer ~ 16, Head ~ 0, Generalized Ships ~ 0 * Layer ~ 21, Head ~ 8, Generalized Ships ~ 10 ### Key Observations * In the "Undiff Attn." plot, Llama-2-7b-chat has a few heads with high "Head" values (around 26 and 29) and high "Generalized Ships" values (around 50-60), while most of its other heads are clustered at lower "Head" and "Layer" values. Vicuna-7b-v1.5 heads are more clustered at lower "Head" values (below 10) but have relatively high "Generalized Ships" values (40-70). * In the "Scaling Cont." plot, Llama-2-7b-chat heads are primarily clustered at the beginning of the layers (Layer 0 and 1) with "Head" values between 8 and 27, and "Generalized Ships" values around 15. Vicuna-7b-v1.5 heads are more spread out across the layers, with lower "Generalized Ships" values. ### Interpretation The plots compare the top 10 safety heads of two language models, Llama-2-7b-chat and Vicuna-7b-v1.5, under two different conditions: "Undiff Attn." and "Scaling Cont.". The "Generalized Ships" value, represented by the color of the data points, seems to indicate some measure of safety or generalization capability. The "Undiff Attn." plot suggests that Llama-2-7b-chat has a few specific heads that are highly active and contribute significantly to safety (high "Head" and "Generalized Ships" values), while Vicuna-7b-v1.5 distributes its safety-related attention more evenly across multiple heads, albeit with lower individual head activation. The "Scaling Cont." plot shows that Llama-2-7b-chat relies heavily on the initial layers for safety-related computations, while Vicuna-7b-v1.5 distributes this responsibility across more layers. The lower "Generalized Ships" values in this plot compared to the "Undiff Attn." plot might indicate a different scaling behavior or a different definition of "safety" under this condition.

\n ## Scatter Plots: Top 10 Safety Heads Performance ### Overview The image presents two scatter plots comparing the performance of "Llama-2-7b-chat" and "Vicuna-7b-v1.5" models across different layers. The plots visualize the relationship between "Layer" (x-axis) and "Generalized Ships" (y-axis) for the top 10 safety heads. The first plot focuses on "Undiff Attn." while the second focuses on "Scaling Cont." The color of the data points corresponds to the model being evaluated, as indicated by the legends. Both plots utilize a color gradient to represent the "Generalized Ships" value. ### Components/Axes * **X-axis (Both Plots):** "Layer" - Ranging from 0 to 30, with markers at integer values. * **Y-axis (Both Plots):** "Generalized Ships" - Ranging from 0 to 70 (left plot) and 0 to 20 (right plot). * **Legend (Both Plots):** * Purple Circle: "Llama-2-7b-chat" * Orange Cross: "Vicuna-7b-v1.5" * **Color Scale (Both Plots):** A gradient from dark blue (low values) to yellow/green (high values) representing "Generalized Ships". * **Title (Left Plot):** "Top 10 Safety Heads on Undiff Attn." * **Title (Right Plot):** "Top 10 Safety Heads on Scaling Cont." ### Detailed Analysis or Content Details **Left Plot: Top 10 Safety Heads on Undiff Attn.** The plot shows a scattered distribution of points for both models. * **Llama-2-7b-chat (Purple):** * Trend: Generally clusters between layers 0-24, with a few points extending to layer 30. The values appear to be relatively stable across layers, with some fluctuations. * Data Points (Approximate): * Layer 0: ~28, Generalized Ships ~68 * Layer 2: ~16, Generalized Ships ~55 * Layer 4: ~4, Generalized Ships ~30 * Layer 6: ~2, Generalized Ships ~20 * Layer 8: ~2, Generalized Ships ~20 * Layer 10: ~16, Generalized Ships ~55 * Layer 12: ~24, Generalized Ships ~60 * Layer 16: ~24, Generalized Ships ~60 * Layer 20: ~24, Generalized Ships ~60 * Layer 24: ~28, Generalized Ships ~68 * Layer 30: ~28, Generalized Ships ~68 * **Vicuna-7b-v1.5 (Orange):** * Trend: Points are more dispersed, with a concentration around layers 0-8. * Data Points (Approximate): * Layer 0: ~6, Generalized Ships ~40 * Layer 2: ~6, Generalized Ships ~40 * Layer 4: ~8, Generalized Ships ~45 * Layer 6: ~2, Generalized Ships ~20 * Layer 8: ~2, Generalized Ships ~20 * Layer 10: ~14, Generalized Ships ~45 * Layer 12: ~14, Generalized Ships ~45 * Layer 16: ~10, Generalized Ships ~35 * Layer 20: ~12, Generalized Ships ~40 * Layer 24: ~4, Generalized Ships ~30 **Right Plot: Top 10 Safety Heads on Scaling Cont.** This plot also shows scattered data points for both models. * **Llama-2-7b-chat (Purple):** * Trend: Points are clustered between layers 0-24, with a slight downward trend as the layer number increases. * Data Points (Approximate): * Layer 0: ~24, Generalized Ships ~18 * Layer 2: ~24, Generalized Ships ~18 * Layer 4: ~22, Generalized Ships ~16 * Layer 6: ~12, Generalized Ships ~8 * Layer 8: ~6, Generalized Ships ~4 * Layer 10: ~12, Generalized Ships ~8 * Layer 12: ~22, Generalized Ships ~16 * Layer 16: ~20, Generalized Ships ~14 * Layer 20: ~6, Generalized Ships ~4 * Layer 24: ~24, Generalized Ships ~18 * **Vicuna-7b-v1.5 (Orange):** * Trend: Points are more spread out, with a noticeable concentration around layers 0-16. * Data Points (Approximate): * Layer 0: ~16, Generalized Ships ~12 * Layer 2: ~16, Generalized Ships ~12 * Layer 4: ~14, Generalized Ships ~10 * Layer 6: ~10, Generalized Ships ~6 * Layer 8: ~10, Generalized Ships ~6 * Layer 10: ~16, Generalized Ships ~12 * Layer 12: ~16, Generalized Ships ~12 * Layer 16: ~12, Generalized Ships ~8 * Layer 20: ~12, Generalized Ships ~8 * Layer 24: ~4, Generalized Ships ~2 ### Key Observations * In the "Undiff Attn." plot, Llama-2-7b-chat generally exhibits higher "Generalized Ships" values than Vicuna-7b-v1.5 across most layers. * In the "Scaling Cont." plot, the difference in "Generalized Ships" values between the two models is less pronounced, but Llama-2-7b-chat still tends to perform better. * Both models show some variability in "Generalized Ships" values across different layers, suggesting that performance is not consistent. * The color gradients effectively highlight the relative performance of each model at each layer. ### Interpretation These plots compare the safety performance of two language models, Llama-2-7b-chat and Vicuna-7b-v1.5, across different layers of their architecture. "Generalized Ships" likely represents a metric related to the model's ability to avoid generating unsafe or harmful content. The two plots explore this metric under different conditions: "Undiff Attn." and "Scaling Cont." The consistent higher performance of Llama-2-7b-chat in both scenarios suggests that it is generally more robust to generating unsafe content than Vicuna-7b-v1.5. The variability in performance across layers indicates that certain layers may be more critical for safety than others. The "Scaling Cont." plot shows a more pronounced decrease in performance for both models as the layer number increases, potentially indicating that safety mechanisms become less effective at deeper layers. The data suggests that layer-specific optimization or targeted safety interventions could be beneficial for improving the overall safety of these models. Further investigation is needed to understand the underlying reasons for the observed performance differences and to identify the specific layers that contribute most to safety.

## [Scatter Plot Comparison]: Top 10 Safety Heads on Undiff Attn. vs. Scaling Cont. ### Overview The image displays two side-by-side scatter plots comparing the locations (by Layer and Head) of the top 10 "safety heads" for two different large language models (Llama-2-7b-chat and Vicuna-7b-v1.5) under two different experimental conditions. The left plot is titled "Top 10 Safety Heads on Undiff Attn." and the right plot is titled "Top 10 Safety Heads on Scaling Cont." Each plot uses a color scale to represent a metric called "Generalized Ships." ### Components/Axes **Common Elements for Both Plots:** * **X-axis:** Label: "Layer". Scale: 0 to 30, with major ticks every 2 units. * **Y-axis:** Label: "Head". Scale: 0 to 30, with major ticks every 2 units. * **Legend:** Located in the top-right corner of each plot. * Purple Circle (●): "Llama-2-7b-chat" * Yellow X (✕): "Vicuna-7b-v1.5" * **Color Bar:** Located to the right of each plot, labeled "Generalized Ships". The scale and range differ between plots. **Left Plot Specifics:** * **Title:** "Top 10 Safety Heads on Undiff Attn." * **Color Bar Scale:** Ranges from 0 (dark purple) to 70 (bright yellow). Ticks at 0, 10, 20, 30, 40, 50, 60, 70. **Right Plot Specifics:** * **Title:** "Top 10 Safety Heads on Scaling Cont." * **Color Bar Scale:** Ranges from 0 (dark purple) to ~22 (bright yellow). Ticks at 0, 5, 10, 15, 20. ### Detailed Analysis **Left Plot: "Undiff Attn."** * **Llama-2-7b-chat (Purple Circles):** Points are clustered in the lower-left quadrant (early layers, lower heads) with a few outliers. * (Layer ~1, Head ~1), Color: Dark purple (~5) * (Layer ~2, Head ~15), Color: Dark purple (~5) * (Layer ~2, Head ~26), Color: Dark purple (~5) * (Layer ~2, Head ~29), Color: Dark purple (~5) * (Layer ~3, Head ~2), Color: Dark purple (~5) * (Layer ~3, Head ~6), Color: Dark purple (~5) * (Layer ~3, Head ~8), Color: Dark purple (~5) * (Layer ~4, Head ~7), Color: Dark purple (~5) * (Layer ~28, Head ~26), Color: Dark purple (~5) * **Vicuna-7b-v1.5 (Yellow X's):** Points are more spread across layers 0-8, with heads mostly below 10. * (Layer ~1, Head ~8), Color: Yellow-green (~60) * (Layer ~2, Head ~1), Color: Blue-green (~30) * (Layer ~3, Head ~7), Color: Blue-green (~30) * (Layer ~4, Head ~2), Color: Blue-green (~30) * (Layer ~6, Head ~0), Color: Blue-green (~30) * (Layer ~6, Head ~2), Color: Blue-green (~30) * (Layer ~6, Head ~6), Color: Blue-green (~30) * (Layer ~3, Head ~26), Color: Blue-green (~30) [Note: This point overlaps with a Llama circle.] **Right Plot: "Scaling Cont."** * **Llama-2-7b-chat (Purple Circles):** Points are distributed across layers 0-14, with a concentration in very early layers (0-1) and heads spanning a wide range. * (Layer ~0, Head ~13), Color: Teal (~12) * (Layer ~0, Head ~21), Color: Teal (~12) * (Layer ~0, Head ~25), Color: Blue (~8) * (Layer ~1, Head ~8), Color: Teal (~12) * (Layer ~1, Head ~15), Color: Yellow (~20) * (Layer ~1, Head ~22), Color: Teal (~12) * (Layer ~1, Head ~27), Color: Blue (~8) * (Layer ~13, Head ~1), Color: Blue (~8) * (Layer ~13, Head ~4), Color: Teal (~12) * (Layer ~14, Head ~23), Color: Blue (~8) * **Vicuna-7b-v1.5 (Yellow X's):** Points are scattered, with a cluster around layers 4-5 and single points at layers 16 and 21. * (Layer ~4, Head ~15), Color: Teal (~12) * (Layer ~5, Head ~15), Color: Teal (~12) * (Layer ~16, Head ~0), Color: Teal (~12) * (Layer ~21, Head ~10), Color: Teal (~12) ### Key Observations 1. **Condition-Dependent Distribution:** The spatial distribution of top safety heads changes dramatically between the "Undiff Attn." and "Scaling Cont." conditions for both models. 2. **Model-Specific Patterns:** * Under "Undiff Attn.", Llama's top heads are mostly in very early layers (1-4) with one late-layer outlier (28), while Vicuna's are in layers 1-8. * Under "Scaling Cont.", Llama's heads are concentrated in the first two layers (0-1), while Vicuna's are more dispersed (layers 4, 5, 16, 21). 3. **"Generalized Ships" Metric:** The metric's value range is much higher for the "Undiff Attn." condition (up to 70) compared to "Scaling Cont." (up to ~20). This suggests the metric is sensitive to the experimental condition. 4. **Overlap:** In the left plot, a Vicuna point at (Layer ~3, Head ~26) overlaps with a Llama point, indicating both models identified a similar head as important under that condition. ### Interpretation This visualization is likely from research on mechanistic interpretability or safety in LLMs. "Safety Heads" probably refers to specific attention heads within the model that are crucial for safe or aligned behavior. "Undiff Attn." (Undifferentiated Attention) and "Scaling Cont." (Scaling Context) are likely two different methods or probes used to identify these heads. The data suggests that: * **The location of influential "safety" mechanisms is not fixed** but depends heavily on the evaluation method ("Undiff Attn." vs. "Scaling Cont."). * **Llama-2-7b-chat and Vicuna-7b-v1.5, despite potential architectural similarities, develop different internal circuits for safety.** Llama shows a strong early-layer focus under "Scaling Cont.", while Vicuna's important heads are more scattered. * The "Generalized Ships" metric, whose meaning is not defined in the image, appears to be a stronger signal under the "Undiff Attn." condition. Its higher values there might indicate a more pronounced or easily detectable effect. **In summary, the image demonstrates that the identification of "safety-critical" components in LLMs is highly contingent on the analytical lens applied, and different models learn different internal strategies for handling safety-related tasks.**

## Scatter Plots: Top 10 Safety Heads on Undiff Attention and Scaling Continuity ### Overview The image contains two side-by-side scatter plots comparing safety head distributions across neural network layers for two models: Llama-2-7b-chat (purple circles) and Vicuna-7b-v1.5 (yellow crosses). The left plot focuses on "Undiff Attention" safety heads, while the right plot examines "Scaling Continuity" safety heads. Both plots use a color gradient ("Generalized Ships") to indicate data point intensity, ranging from 0 (dark purple) to 70 (bright yellow). --- ### Components/Axes #### Left Plot (Undiff Attention) - **X-axis**: Layer (0–30, integer increments) - **Y-axis**: Head (0–30, integer increments) - **Legend**: - Purple circles: Llama-2-7b-chat - Yellow crosses: Vicuna-7b-v1.5 - **Color Bar**: "Generalized Ships" (0–70, darker = lower, brighter = higher) #### Right Plot (Scaling Continuity) - **X-axis**: Layer (0–30, integer increments) - **Y-axis**: Head (0–30, integer increments) - **Legend**: Same as left plot - **Color Bar**: Same scale as left plot --- ### Detailed Analysis #### Left Plot (Undiff Attention) - **Llama-2-7b-chat** (purple circles): - Concentrated in upper layers (26–30) with high head numbers (24–28). - One outlier at layer 2 with head 14. - Color intensity varies: darker (lower ships) in upper layers, brighter (higher ships) in lower layers. - **Vicuna-7b-v1.5** (yellow crosses): - Spread across lower layers (0–6) with heads 0–8. - One outlier at layer 4 with head 8. - Color intensity: brighter (higher ships) in lower layers. #### Right Plot (Scaling Continuity) - **Llama-2-7b-chat** (purple circles): - Clustered in middle layers (12–14) with heads 14–16. - Additional points in upper layers (26–30) with heads 24–26. - Color intensity: darker (lower ships) in middle layers, brighter in upper layers. - **Vicuna-7b-v1.5** (yellow crosses): - Concentrated in lower layers (4–6) with heads 8–10. - One outlier at layer 20 with head 12. - Color intensity: brighter (higher ships) in lower layers. --- ### Key Observations 1. **Layer Distribution**: - Llama-2-7b-chat safety heads dominate **upper layers** in both plots. - Vicuna-7b-v1.5 safety heads are concentrated in **lower layers** for undiff attention and **middle-lower layers** for scaling continuity. 2. **Head Numbers**: - Llama-2-7b-chat consistently shows higher head numbers (14–28) compared to Vicuna-7b-v1.5 (0–12). 3. **Color Intensity**: - Vicuna-7b-v1.5 data points generally exhibit brighter colors (higher "Generalized Ships") in lower layers, suggesting stronger safety signals in these regions. 4. **Outliers**: - Vicuna-7b-v1.5 has an outlier at layer 20 (head 12) in the scaling continuity plot, deviating from its lower-layer trend. --- ### Interpretation 1. **Model Behavior**: - Llama-2-7b-chat’s safety heads in upper layers (undiff attention) and middle/upper layers (scaling continuity) may indicate specialized safety mechanisms in later processing stages. - Vicuna-7b-v1.5’s lower-layer dominance suggests safety features are more active in early computational stages. 2. **Generalized Ships**: - The color gradient implies that Vicuna-7b-v1.5’s safety signals are more pronounced (higher ships) in lower layers, while Llama-2-7b-chat’s signals are stronger in upper layers. 3. **Anomalies**: - The Vicuna-7b-v1.5 outlier at layer 20 (scaling continuity) may reflect an unexpected safety mechanism or data artifact. --- ### Conclusion The plots reveal distinct safety head distributions between the two models, with Llama-2-7b-chat favoring higher layers and Vicuna-7b-v1.5 prioritizing lower layers. The "Generalized Ships" metric highlights differences in safety signal strength across layers, offering insights into model architecture and safety design priorities.