## Scatter Plot: Qwen3 (50x Compaction) Performance
### Overview
This scatter plot compares the accuracy and compaction time of various text compression methods for Qwen3 (50x compression). The chart visualizes trade-offs between compression speed (x-axis) and output quality (y-axis), with special emphasis on methods achieving "high quality compaction in <1 minute" (green shaded region).
### Components/Axes
- **X-axis**: Avg Compaction Time per Article (s) [log scale: 10⁰ to 10⁴]
- **Y-axis**: Accuracy [0.35 to 0.70]
- **Legend**: Right-aligned, color-coded for:
- Original Cache (dashed black)
- No-Context (dotted gray)
- Compression methods (solid lines/dots with labels)
- **Special Elements**:
- Green shaded region: "high quality compaction in <1 minute"
- Dotted reference lines at y=0.35 (No-Context) and y=0.70 (Original Cache)
### Detailed Analysis
1. **Original Cache** (dashed black line):
- Horizontal line at y=0.70
- Represents baseline accuracy with no compression
2. **No-Context** (dotted gray line):
- Horizontal line at y=0.35
- Represents minimum acceptable accuracy threshold
3. **Compression Methods**:
- **Cartridges** (teal dashed line):
- Points at (10³.⁵s, 0.55), (10⁴s, 0.58), (10⁴.⁵s, 0.60)
- Shows gradual accuracy improvement with longer compaction times
- **AM-OMP** (purple solid line):
- Points at (10²s, 0.65), (10².⁵s, 0.66)
- High accuracy with moderate compaction time
- **AM-HighestAttnKeys** (red solid line):
- Points at (10¹.⁵s, 0.58), (10²s, 0.65)
- Similar performance to AM-OMP but slightly faster
- **AM-OMP-fast** (purple dot):
- Point at (10¹s, 0.63)
- Faster version with minor accuracy reduction
- **AM-HighestAttnKeys-fast** (red dot):
- Point at (10¹s, 0.57)
- Fastest among high-attention methods
- **Summarization** (blue dot):
- Point at (10⁰s, 0.52)
- Near-instant but lower accuracy
- **H2O+** (brown dot):
- Point at (10⁰.⁵s, 0.48)
- Very fast but significantly lower accuracy
- **KVzip** (dark gray dot):
- Point at (10⁰.⁸s, 0.42)
- Slightly slower than H2O+ with worse accuracy
- **PyramidKV** (light gray dot):
- Point at (10¹s, 0.41)
- Similar to SnapKV but slightly faster
- **SnapKV** (lightest gray dot):
- Point at (10¹s, 0.40)
- Fastest among KV-based methods
### Key Observations
1. **Accuracy-Time Tradeoff**:
- Faster methods (10⁰-10¹s) cluster near No-Context accuracy (0.35-0.52)
- Methods achieving >0.60 accuracy require >10²s compaction time
- "High quality" region (<1 minute) contains 4 methods with 0.57-0.66 accuracy
2. **Performance Clusters**:
- **High-Performance**: AM-OMP/AM-HighestAttnKeys (0.65-0.66 accuracy at 10²-10².⁵s)
- **Balanced**: Cartridges (0.55-0.60 accuracy at 10³-10⁴.⁵s)
- **Speed-Focused**: Fast variants (0.57-0.63 accuracy at 10¹s)
3. **Outliers**:
- H2O+ (0.48 accuracy at 10⁰.⁵s) shows worst accuracy among fast methods
- Original Cache (0.70 accuracy) remains unattainable by all compression methods
### Interpretation
The chart demonstrates that Qwen3 compression methods exhibit a clear Pareto frontier between accuracy and speed. Methods in the green "high quality" region (<1 minute) achieve 0.57-0.66 accuracy, representing optimal trade-offs for practical deployment. The AM-OMP and AM-HighestAttnKeys methods show the best balance of speed and quality, while Cartridges offers more gradual improvements for applications tolerating longer compaction times. The Original Cache remains the theoretical upper bound for accuracy, while No-Context represents the lower limit for acceptable output quality. The fast variants (AM-OMP-fast, AM-HighestAttnKeys-fast) provide viable options for latency-sensitive applications requiring <10¹s compaction times with moderate accuracy sacrifices.
