## Bar Chart: Attack Success Rate (ASR) vs. Ablating Head Numbers for Llama-2-7b-chat-hf and Vicuna-7b-v1.5
### Overview
The image presents two bar charts comparing the attack success rate (ASR) against ablating head numbers for two language models: Llama-2-7b-chat-hf (top) and Vicuna-7b-v1.5 (bottom). The x-axis represents the number of ablating heads (0 to 5), while the y-axis represents the attack success rate (ASR) ranging from 0.0 to 1.0. The charts compare the performance of different attack methods: Advbench, Jailbreakbench, and Malicious Instruct, using both "use-tem" and "direct" approaches. Additionally, the charts include line plots showing the "Vanilla Average" and "Use-tem Average" ASR across different ablating head numbers.
### Components/Axes
* **Titles:**
* Top Chart: Llama-2-7b-chat-hf
* Bottom Chart: Vicuna-7b-v1.5
* **X-Axis:**
* Label: Ablating Head Numbers
* Scale: 0, 1, 2, 3, 4, 5
* **Y-Axis:**
* Label: Attack Success Rate (ASR) - repeated for both charts
* Scale: 0.0, 0.2, 0.4, 0.6, 0.8, 1.0
* **Legend (Top-Left):**
* Advbench (use-tem): Red bars
* Jailbreakbench (use-tem): Yellow bars with black diagonal lines
* Malicious Instruct (use-tem): Teal bars
* Advbench (direct): Red bars with black diagonal lines
* Jailbreakbench (direct): Yellow bars
* Malicious Instruct (direct): Teal bars with black diagonal lines
* **Legend (Top-Right):**
* Vanilla Average: Pink line with circular markers
* Use-tem Average: Light Purple line with square markers
### Detailed Analysis
#### Llama-2-7b-chat-hf (Top Chart)
* **Advbench (use-tem):** The ASR starts around 0.18 at 0 ablating heads, increases to approximately 0.3 at 1 ablating head, then fluctuates between 0.35 and 0.45 for the remaining ablating head numbers.
* **Jailbreakbench (use-tem):** The ASR starts around 0.22 at 0 ablating heads, increases to approximately 0.4 at 1 ablating head, then fluctuates between 0.35 and 0.5 for the remaining ablating head numbers.
* **Malicious Instruct (use-tem):** The ASR starts around 0.18 at 0 ablating heads, decreases to approximately 0.02 at 1 ablating head, then fluctuates between 0.2 and 0.3 for the remaining ablating head numbers.
* **Advbench (direct):** The ASR starts around 0.02 at 0 ablating heads, increases to approximately 0.02 at 1 ablating head, then fluctuates between 0.02 and 0.04 for the remaining ablating head numbers.
* **Jailbreakbench (direct):** The ASR starts around 0.02 at 0 ablating heads, increases to approximately 0.1 at 1 ablating head, then fluctuates between 0.4 and 0.5 for the remaining ablating head numbers.
* **Malicious Instruct (direct):** The ASR starts around 0.02 at 0 ablating heads, increases to approximately 0.1 at 1 ablating head, then fluctuates between 0.15 and 0.2 for the remaining ablating head numbers.
* **Vanilla Average:** The ASR starts around 0.1 at 0 ablating heads, increases to approximately 0.35 at 1 ablating head, then fluctuates between 0.35 and 0.4 for the remaining ablating head numbers.
* **Use-tem Average:** The ASR starts around 0.1 at 0 ablating heads, increases to approximately 0.2 at 1 ablating head, then fluctuates between 0.35 and 0.45 for the remaining ablating head numbers.
#### Vicuna-7b-v1.5 (Bottom Chart)
* **Advbench (use-tem):** The ASR starts around 0.63 at 0 ablating heads, decreases to approximately 0.52 at 1 ablating head, then fluctuates between 0.5 and 0.6 for the remaining ablating head numbers.
* **Jailbreakbench (use-tem):** The ASR starts around 0.62 at 0 ablating heads, decreases to approximately 0.5 at 1 ablating head, then fluctuates between 0.5 and 0.6 for the remaining ablating head numbers.
* **Malicious Instruct (use-tem):** The ASR starts around 0.45 at 0 ablating heads, increases to approximately 0.65 at 1 ablating head, then fluctuates between 0.5 and 0.7 for the remaining ablating head numbers.
* **Advbench (direct):** The ASR starts around 0.2 at 0 ablating heads, decreases to approximately 0.15 at 1 ablating head, then fluctuates between 0.0 and 0.3 for the remaining ablating head numbers.
* **Jailbreakbench (direct):** The ASR starts around 0.3 at 0 ablating heads, decreases to approximately 0.3 at 1 ablating head, then fluctuates between 0.3 and 0.4 for the remaining ablating head numbers.
* **Malicious Instruct (direct):** The ASR starts around 0.4 at 0 ablating heads, increases to approximately 0.55 at 1 ablating head, then fluctuates between 0.2 and 0.5 for the remaining ablating head numbers.
* **Vanilla Average:** The ASR starts around 0.55 at 0 ablating heads, decreases to approximately 0.5 at 1 ablating head, then fluctuates between 0.5 and 0.55 for the remaining ablating head numbers.
* **Use-tem Average:** The ASR starts around 0.3 at 0 ablating heads, increases to approximately 0.35 at 1 ablating head, then fluctuates between 0.3 and 0.4 for the remaining ablating head numbers.
### Key Observations
* For Llama-2-7b-chat-hf, the "use-tem" attacks generally have a higher success rate compared to the "direct" attacks.
* For Vicuna-7b-v1.5, the initial attack success rates are higher compared to Llama-2-7b-chat-hf, but the impact of ablating head numbers varies across different attack methods.
* The "Vanilla Average" and "Use-tem Average" lines provide a general trend of the attack success rate as the number of ablating heads increases.
### Interpretation
The charts illustrate the vulnerability of language models to different types of attacks and how ablating head numbers can affect the attack success rate. The "use-tem" attacks appear to be more effective against Llama-2-7b-chat-hf, while Vicuna-7b-v1.5 shows a different pattern with higher initial vulnerability. The data suggests that the models have varying degrees of robustness against different attack strategies, and ablating head numbers can influence their susceptibility. The "Vanilla Average" and "Use-tem Average" lines provide a baseline for comparison and highlight the overall trend of attack success rate as the model's architecture is modified.
