## Scatter Plot: LongBench Accuracy vs KV Cache Size for TriviaQA Dataset
### Overview
The chart visualizes the relationship between KV cache size (in GB) and LongBench accuracy for five different methods (H₂O, KIVI, SnapKV, MiniKV) on the TriviaQA dataset. A Pareto Frontier line connects optimal trade-off points between cache size and accuracy.
### Components/Axes
- **X-axis**: KV Cache Size (GB) ranging from 0.00 to 2.00 in 0.25 increments
- **Y-axis**: LongBench Accuracy (81 to 85.5) in 0.5 increments
- **Legend**: Located in top-right corner with color/symbol mappings:
- Red squares: H₂O
- Blue circles: KIVI
- Green diamonds: SnapKV
- Yellow triangles: MiniKV
- Black crosses: Pareto Frontier
### Detailed Analysis
1. **H₂O (Red Squares)**
- Points at:
- 0.25 GB → ~84.0 accuracy
- 0.5 GB → ~84.2 accuracy
- 1.0 GB → ~84.0 accuracy
- Trend: Slight fluctuation with no clear upward trend
2. **KIVI (Blue Circle)**
- Single point at:
- 0.5 GB → ~84.8 accuracy
- Trend: Isolated high-performance outlier
3. **SnapKV (Green Diamonds)**
- Points at:
- 0.25 GB → ~85.2 accuracy
- 0.5 GB → ~85.0 accuracy
- 1.0 GB → ~85.5 accuracy
- 1.25 GB → ~85.5 accuracy
- Trend: Steady increase followed by plateau
4. **MiniKV (Yellow Triangles)**
- Points at:
- 0.1 GB → ~82.7 accuracy
- 0.3 GB → ~82.0 accuracy
- 0.5 GB → ~82.5 accuracy
- Trend: Initial drop followed by slight recovery
5. **Pareto Frontier (Black Line with Crosses)**
- Connects optimal points:
- Starts at ~0.1 GB, 82.7 accuracy
- Peaks at ~1.0 GB, 85.5 accuracy
- Flattens at ~1.25 GB, 85.5 accuracy
### Key Observations
- **Performance Leaders**: SnapKV and KIVI dominate in accuracy, with SnapKV achieving 85.5% at 1.0-1.25 GB cache size
- **Resource Efficiency**: MiniKV uses smallest cache (0.1-0.5 GB) but lags in accuracy (~82-82.5%)
- **Diminishing Returns**: Pareto Frontier flattens after 1.0 GB, indicating limited accuracy gains from larger caches
- **Anomaly**: KIVI's single point at 0.5 GB (84.8%) sits below SnapKV's performance at same cache size
### Interpretation
The data demonstrates a clear trade-off between computational resources and question-answering performance. SnapKV emerges as the most efficient method, achieving highest accuracy (85.5%) with moderate cache usage (1.0-1.25 GB). The Pareto Frontier reveals that beyond 1.0 GB, additional cache provides minimal accuracy improvements, suggesting diminishing returns in this configuration.
Notably, MiniKV's lower performance despite smaller cache footprint indicates potential architectural limitations. KIVI's isolated high-performance point suggests specialized optimization for specific cache sizes. These findings highlight the importance of cache efficiency over raw size in knowledge-intensive QA systems, with SnapKV representing the optimal balance for TriviaQA.
