## Line Chart: PPL Degradation vs Compression Ratio for GPT Models
### Overview
Three line charts compare perplexity (PPL) degradation against compression ratio for three GPT variants: GPT-2 Medium, GPT Neo 1.3B, and GPT Neo 2.7B. Each chart shows multiple data series representing different quantization (q) and grouping (g) parameters, with distinct markers and colors for each series.
### Components/Axes
- **X-axis**: Compression Ratio (0–12, linear scale)
- **Y-axis**: PPL Degradation (-2–10, linear scale)
- **Legends**:
- **q values**: Quantization levels (3, 4, 5, 6) with distinct colors:
- q=3: Red (square)
- q=4: Orange (circle)
- q=5: Blue (triangle)
- q=6: Green (star)
- **g values**: Grouping parameters (8, 16, 32, 64, 128, 256, Row-wise) with distinct markers:
- g=8: Square
- g=16: Circle
- g=32: Triangle
- g=64: Diamond
- g=128: Pentagon
- g=256: Hexagon
- Row-wise: Star
### Detailed Analysis
#### Chart (a) GPT-2 Medium
- **q=3 (Red)**: Steep upward slope, reaching ~8 PPL at compression ratio 10.
- **q=4 (Orange)**: Gradual increase, peaking at ~6 PPL at ratio 10.
- **q=5 (Blue)**: Moderate rise, ~4 PPL at ratio 10.
- **q=6 (Green)**: Slowest increase, ~2 PPL at ratio 10.
- **g values**: All g series (8–256) cluster near 0–2 PPL until ratio ~6, then diverge sharply. Row-wise (star) shows a unique spike at ratio 8 (~8 PPL).
#### Chart (b) GPT Neo 1.3B
- **q=3 (Red)**: Sharp rise, ~7 PPL at ratio 10.
- **q=4 (Orange)**: Steady increase, ~5 PPL at ratio 10.
- **q=5 (Blue)**: Moderate growth, ~3 PPL at ratio 10.
- **q=6 (Green)**: Minimal degradation, ~1 PPL at ratio 10.
- **g values**: Similar clustering as GPT-2, but Row-wise (star) spikes earlier (~6 PPL at ratio 6).
#### Chart (c) GPT Neo 2.7B
- **q=3 (Red)**: Most severe degradation, ~9 PPL at ratio 10.
- **q=4 (Orange)**: Rapid rise, ~6 PPL at ratio 10.
- **q=5 (Blue)**: Gradual increase, ~4 PPL at ratio 10.
- **q=6 (Green)**: Stable, ~1.5 PPL at ratio 10.
- **g values**: Row-wise (star) shows extreme sensitivity, spiking to ~7 PPL at ratio 8.
### Key Observations
1. **Quantization Impact**: Lower q values (e.g., q=3) consistently show higher PPL degradation across all models.
2. **Grouping Sensitivity**: Higher g values (e.g., g=256) exhibit delayed but severe degradation after compression ratio ~6.
3. **Row-wise Anomaly**: The "Row-wise" series (star) demonstrates disproportionate sensitivity in all models, with sharp spikes at mid-to-high compression ratios.
4. **Model-Specific Trends**: GPT Neo 2.7B shows the most severe overall degradation, particularly for q=3 and Row-wise configurations.
### Interpretation
The charts reveal that:
- **Compression Ratio** directly correlates with PPL degradation, with steeper slopes indicating worse performance.
- **Quantization (q)** and **Grouping (g)** parameters significantly influence degradation patterns:
- Lower q values (higher compression) degrade performance more severely.
- Larger g values (e.g., g=256) show delayed but catastrophic failure after moderate compression.
- **Row-wise grouping** behaves anomalously, suggesting it may not scale well with compression in these models.
- **Model Architecture** affects degradation severity: GPT Neo 2.7B (largest) degrades most, while GPT-2 Medium (smallest) shows milder trends.
This data implies that compression strategies must carefully balance q and g parameters to mitigate PPL degradation, with Row-wise grouping requiring special consideration due to its nonlinear behavior.
