# Technical Document Analysis of Attack Success Rate Chart
## 1. Chart Type and Structure
- **Chart Type**: Grouped bar chart
- **Axes**:
- **X-axis**: Model names (categorical)
- **Y-axis**: Attack Success Rate (%) (numerical, 0-100%)
## 2. Legend and Color Mapping
- **Legend Location**: Top-right corner
- **Color-Coded Methods**:
- **Gray**: Prompt Only
- **Orange**: "Sure, here's"
- **Blue**: GCG (Ours)
- **Light Blue**: GCG Ensemble (Ours)
## 3. Categories and Sub-Categories
- **Models (X-axis)**:
1. Pythia-12B
2. Falcon-7B
3. Guanaco-7B
4. ChatGLM-6B
5. MPT-7B
6. Stable-Vicuna
7. Vicuna-7B
8. Vicuna-13B
9. GPT-3.5
10. GPT-4
## 4. Data Points and Trends
### Key Observations:
- **GCG Ensemble (Light Blue)** consistently shows the highest attack success rates across all models.
- **Prompt Only (Gray)** generally has the lowest success rates.
- **GCG (Blue)** and **"Sure, here's" (Orange)** methods show intermediate performance, with GCG often outperforming "Sure, here's".
### Model-Specific Analysis:
1. **Pythia-12B**:
- Prompt Only: ~85%
- "Sure, here's": ~95%
- GCG: ~98%
- GCG Ensemble: ~99%
2. **Falcon-7B**:
- Prompt Only: ~75%
- "Sure, here's": ~92%
- GCG: ~90%
- GCG Ensemble: ~95%
3. **Guanaco-7B**:
- Prompt Only: ~25%
- "Sure, here's": ~45%
- GCG: ~90%
- GCG Ensemble: ~95%
4. **ChatGLM-6B**:
- Prompt Only: ~5%
- "Sure, here's": ~40%
- GCG: ~45%
- GCG Ensemble: ~75%
5. **MPT-7B**:
- Prompt Only: ~10%
- "Sure, here's": ~50%
- GCG: ~65%
- GCG Ensemble: ~70%
6. **Stable-Vicuna**:
- Prompt Only: ~2%
- "Sure, here's": ~20%
- GCG: ~92%
- GCG Ensemble: ~97%
7. **Vicuna-7B**:
- Prompt Only: ~1%
- "Sure, here's": ~30%
- GCG: ~93%
- GCG Ensemble: ~97%
8. **Vicuna-13B**:
- Prompt Only: ~2%
- "Sure, here's": ~60%
- GCG: ~96%
- GCG Ensemble: ~98%
9. **GPT-3.5**:
- Prompt Only: ~3%
- "Sure, here's": ~5%
- GCG: ~48%
- GCG Ensemble: ~88%
10. **GPT-4**:
- Prompt Only: ~8%
- "Sure, here's": ~12%
- GCG: ~35%
- GCG Ensemble: ~45%
## 5. Spatial Grounding
- **Legend Position**: Top-right corner (confirmed via visual alignment)
- **Color Consistency**: All bars match legend colors (e.g., GCG Ensemble = light blue).
## 6. Trend Verification
- **General Trend**: GCG Ensemble > GCG > "Sure, here's" > Prompt Only across most models.
- **Exceptions**:
- GPT-3.5: GCG (~48%) > "Sure, here's" (~5%).
- GPT-4: GCG (~35%) > "Sure, here's" (~12%).
## 7. Component Isolation
- **Header**: Legend (top-right)
- **Main Chart**: Bar groups for each model (x-axis) with four bars per model.
- **Footer**: No additional components.
## 8. Summary
The chart demonstrates that **GCG Ensemble** consistently achieves the highest attack success rates across all models, significantly outperforming other methods. **Prompt Only** methods show the lowest performance, while **GCG** and **"Sure, here's"** methods exhibit intermediate effectiveness. The largest performance gaps are observed in models like **Guanaco-7B** and **Stable-Vicuna**, where GCG Ensemble achieves near-perfect success rates (~95-97%).
