## Bar Chart: Attack Success Rate (ASR) by Defense Mechanism and Attack Type
### Overview
This image displays a grid of eight bar charts, each evaluating the "Attack Success Rate (ASR)" of five different AI defense mechanisms against specific adversarial attack methods. The Y-axis represents the ASR, ranging from 0.0 to 1.0. The X-axis for each sub-chart represents the five defense mechanisms, color-coded consistently across all charts.
### Components/Axes
* **Y-Axis:** "Attack Success Rate (ASR)", scaled from 0.0 to 1.0.
* **X-Axis (Implicit):** Five defense mechanisms, represented by bars in the following order (left to right):
1. **Teal:** Azure Prompt Shield
2. **Blue:** Protect AI v1
3. **Light Green:** Meta Prompt Guard
4. **Yellow:** Vijil Prompt Injection
5. **Tan:** NeMo Guard Jailbreak Detect
* **Sub-Chart Titles (Attack Methods):**
* Top Row: BAE, Bert-Attack, Deep Word Bug, Alzantot
* Bottom Row: Pruthi, PWWS, TextBugger, TextFooler
### Detailed Analysis
The following values are approximate estimates based on the visual height of the bars relative to the Y-axis grid lines.
| Attack Method | Azure (Teal) | Protect AI (Blue) | Meta Guard (Lt. Green) | Vijil (Yellow) | NeMo (Tan) |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **BAE** | ~0.12 | ~0.22 | ~0.05 | ~0.25 | ~0.82 |
| **Bert-Attack** | ~0.12 | ~0.25 | ~0.10 | ~0.22 | ~0.50 |
| **Deep Word Bug** | ~0.15 | ~0.22 | ~0.18 | ~0.28 | ~0.95 |
| **Alzantot** | ~0.12 | ~0.15 | ~0.02 | ~0.08 | ~0.55 |
| **Pruthi** | ~0.15 | ~0.12 | ~0.02 | ~0.05 | ~0.58 |
| **PWWS** | ~0.15 | ~0.32 | ~0.22 | ~0.48 | ~0.65 |
| **TextBugger** | ~0.12 | ~0.32 | ~0.20 | ~0.72 | ~0.95 |
| **TextFooler** | ~0.12 | ~0.30 | ~0.28 | ~0.75 | ~0.92 |
### Key Observations
* **Defense Effectiveness:** Azure Prompt Shield (Teal) and Protect AI v1 (Blue) consistently demonstrate the lowest ASR across all attack methods, suggesting they are the most robust defenses among those tested.
* **Performance Outlier:** NeMo Guard Jailbreak Detect (Tan) consistently shows the highest ASR across all categories, indicating it is the least effective defense against these specific adversarial attacks.
* **Attack Sensitivity:** The "Vijil Prompt Injection" (Yellow) defense shows high variance. It performs well against "Alzantot" and "Pruthi" (low ASR) but fails significantly against "TextBugger" and "TextFooler" (high ASR).
* **Attack Difficulty:** "TextBugger" and "TextFooler" appear to be the most potent attack methods, as they result in the highest ASR values across the board, particularly for the Yellow and Tan defense mechanisms.
### Interpretation
This data demonstrates a comparative benchmark of AI security defenses. The "Attack Success Rate" (ASR) is an inverse metric of security: a lower bar is better.
The data suggests a clear stratification in defense capability. Azure Prompt Shield and Protect AI v1 maintain a relatively stable, low ASR, indicating a consistent baseline of protection regardless of the attack vector. Conversely, the NeMo Guard Jailbreak Detect appears highly vulnerable to these specific text-based adversarial attacks, with ASRs frequently exceeding 0.5 and reaching near 1.0 in cases like "Deep Word Bug" and "TextBugger."
The variability in the "Vijil Prompt Injection" (Yellow) bars suggests that this specific defense mechanism may be over-fitted or specialized for certain types of attacks, failing to generalize against more sophisticated methods like "TextFooler" or "TextBugger." This chart serves as a critical decision-making tool for security engineers to determine which defense mechanism is appropriate based on the threat model (i.e., which attacks they are most concerned about mitigating).
