## Line Graphs: Accuracy vs. Compacted Size Across Datasets and Methods ### Overview The image contains six line graphs comparing the accuracy of various methods across three datasets (Qwen3-4B, Llama3-1.8B, Gemma3-12B) under two evaluation metrics: QUALITY and LongHealth. Each graph plots accuracy against compacted size (0.01 to 0.20), with methods differentiated by color. The graphs show trends in performance as models are compressed. ### Components/Axes - **X-axis**: Compacted Size (0.01, 0.02, 0.05, 0.10, 0.20) - **Y-axis**: Accuracy (0.35 to 0.80) - **Legend**: Located at the bottom center, mapping colors to methods: - Purple: AM-OMP - Red: AM-HighestAttnKeys - Green: Cartridges - Blue: Summarization (various prompts) - Gray: SnapKV - Light Gray: H2O+ - Dark Gray: PyramidKV - Blue Dotted: KVzip - Dashed Line: Original Cache - Dotted Line: No Context ### Detailed Analysis #### Qwen3-4B QUALITY - **AM-OMP (Purple)**: Starts at ~0.65 (0.01) and rises to ~0.75 (0.20), maintaining the highest accuracy. - **AM-HighestAttnKeys (Red)**: Begins at ~0.60 (0.01), peaks at ~0.72 (0.10), then plateaus. - **Cartridges (Green)**: Flat line at ~0.62 across all compacted sizes. - **Summarization (Blue)**: Starts at ~0.45 (0.01), rises to ~0.55 (0.10), then drops to ~0.50 (0.20). - **SnapKV (Gray)**: Gradual increase from ~0.40 (0.01) to ~0.55 (0.20). - **H2O+ (Light Gray)**: Starts at ~0.35 (0.01), rises to ~0.50 (0.10), then declines to ~0.45 (0.20). - **PyramidKV (Dark Gray)**: Flat at ~0.40 (0.01–0.20). - **KVzip (Blue Dotted)**: Starts at ~0.40 (0.01), peaks at ~0.55 (0.10), then drops to ~0.50 (0.20). - **Original Cache (Dashed)**: Flat at ~0.65. - **No Context (Dotted)**: Flat at ~0.35. #### Qwen3-4B LongHealth - **AM-OMP (Purple)**: Starts at ~0.60 (0.01), rises to ~0.70 (0.10), then plateaus at ~0.72 (0.20). - **AM-HighestAttnKeys (Red)**: Begins at ~0.55 (0.01), peaks at ~0.68 (0.10), then drops to ~0.65 (0.20). - **Cartridges (Green)**: Flat at ~0.65 (0.01–0.20). - **Summarization (Blue)**: Starts at ~0.35 (0.01), rises to ~0.45 (0.10), then drops to ~0.40 (0.20). - **SnapKV (Gray)**: Gradual increase from ~0.30 (0.01) to ~0.50 (0.20). - **H2O+ (Light Gray)**: Starts at ~0.30 (0.01), rises to ~0.45 (0.10), then declines to ~0.40 (0.20). - **PyramidKV (Dark Gray)**: Flat at ~0.35 (0.01–0.20). - **KVzip (Blue Dotted)**: Starts at ~0.35 (0.01), peaks at ~0.50 (0.10), then drops to ~0.45 (0.20). - **Original Cache (Dashed)**: Flat at ~0.60. - **No Context (Dotted)**: Flat at ~0.30. #### Llama3-1.8B QUALITY - **AM-OMP (Purple)**: Starts at ~0.55 (0.01), rises to ~0.65 (0.10), then plateaus at ~0.68 (0.20). - **AM-HighestAttnKeys (Red)**: Begins at ~0.50 (0.01), peaks at ~0.62 (0.10), then drops to ~0.60 (0.20). - **Cartridges (Green)**: Flat at ~0.55 (0.01–0.20). - **Summarization (Blue)**: Starts at ~0.40 (0.01), rises to ~0.50 (0.10), then drops to ~0.45 (0.20). - **SnapKV (Gray)**: Gradual increase from ~0.35 (0.01) to ~0.55 (0.20). - **H2O+ (Light Gray)**: Starts at ~0.30 (0.01), rises to ~0.45 (0.10), then declines to ~0.40 (0.20). - **PyramidKV (Dark Gray)**: Flat at ~0.35 (0.01–0.20). - **KVzip (Blue Dotted)**: Starts at ~0.40 (0.01), peaks at ~0.50 (0.10), then drops to ~0.45 (0.20). - **Original Cache (Dashed)**: Flat at ~0.60. - **No Context (Dotted)**: Flat at ~0.35. #### Llama3-1.8B LongHealth - **AM-OMP (Purple)**: Starts at ~0.50 (0.01), rises to ~0.60 (0.10), then plateaus at ~0.62 (0.20). - **AM-HighestAttnKeys (Red)**: Begins at ~0.45 (0.01), peaks at ~0.60 (0.10), then drops to ~0.58 (0.20). - **Cartridges (Green)**: Flat at ~0.55 (0.01–0.20). - **Summarization (Blue)**: Starts at ~0.30 (0.01), rises to ~0.40 (0.10), then drops to ~0.35 (0.20). - **SnapKV (Gray)**: Gradual increase from ~0.25 (0.01) to ~0.45 (0.20). - **H2O+ (Light Gray)**: Starts at ~0.25 (0.01), rises to ~0.40 (0.10), then declines to ~0.35 (0.20). - **PyramidKV (Dark Gray)**: Flat at ~0.30 (0.01–0.20). - **KVzip (Blue Dotted)**: Starts at ~0.30 (0.01), peaks at ~0.45 (0.10), then drops to ~0.40 (0.20). - **Original Cache (Dashed)**: Flat at ~0.55. - **No Context (Dotted)**: Flat at ~0.25. #### Gemma3-12B QUALITY - **AM-OMP (Purple)**: Starts at ~0.60 (0.01), rises to ~0.70 (0.10), then plateaus at ~0.72 (0.20). - **AM-HighestAttnKeys (Red)**: Begins at ~0.55 (0.01), peaks at ~0.68 (0.10), then drops to ~0.65 (0.20). - **Cartridges (Green)**: Flat at ~0.58 (0.01–0.20). - **Summarization (Blue)**: Starts at ~0.45 (0.01), rises to ~0.55 (0.10), then drops to ~0.50 (0.20). - **SnapKV (Gray)**: Gradual increase from ~0.40 (0.01) to ~0.55 (0.20). - **H2O+ (Light Gray)**: Starts at ~0.35 (0.01), rises to ~0.50 (0.10), then declines to ~0.45 (0.20). - **PyramidKV (Dark Gray)**: Flat at ~0.40 (0.01–0.20). - **KVzip (Blue Dotted)**: Starts at ~0.45 (0.01), peaks at ~0.55 (0.10), then drops to ~0.50 (0.20). - **Original Cache (Dashed)**: Flat at ~0.65. - **No Context (Dotted)**: Flat at ~0.35. #### Gemma3-12B LongHealth - **AM-OMP (Purple)**: Starts at ~0.55 (0.01), rises to ~0.65 (0.10), then plateaus at ~0.68 (0.20). - **AM-HighestAttnKeys (Red)**: Begins at ~0.50 (0.01), peaks at ~0.65 (0.10), then drops to ~0.62 (0.20). - **Cartridges (Green)**: Flat at ~0.58 (0.01–0.20). - **Summarization (Blue)**: Starts at ~0.40 (0.01), rises to ~0.50 (0.10), then drops to ~0.45 (0.20). - **SnapKV (Gray)**: Gradual increase from ~0.35 (0.01) to ~0.50 (0.20). - **H2O+ (Light Gray)**: Starts at ~0.30 (0.01), rises to ~0.45 (0.10), then declines to ~0.40 (0.20). - **PyramidKV (Dark Gray)**: Flat at ~0.35 (0.01–0.20). - **KVzip (Blue Dotted)**: Starts at ~0.40 (0.01), peaks at ~0.50 (0.10), then drops to ~0.45 (0.20). - **Original Cache (Dashed)**: Flat at ~0.60. - **No Context (Dotted)**: Flat at ~0.30. ### Key Observations 1. **AM-OMP** consistently outperforms other methods across all datasets and metrics, maintaining the highest accuracy as compacted size increases. 2. **AM-HighestAttnKeys** shows strong performance but declines at larger compacted sizes (e.g., 0.20). 3. **Cartridges** maintains stable but suboptimal accuracy compared to AM-OMP. 4. **Summarization** methods exhibit a "U-shaped" trend, peaking at mid-compacted sizes before declining. 5. **SnapKV** and **H2O+** show gradual improvement but lag behind AM-OMP. 6. **PyramidKV** and **KVzip** underperform, with flat or declining trends. 7. **Original Cache** (dashed line) serves as a baseline, outperforming most methods except AM-OMP. 8. **No Context** (dotted line) consistently has the lowest accuracy. ### Interpretation The data suggests that **AM-OMP** is the most robust method for maintaining accuracy under compression, likely due to its adaptive optimization. Methods like **AM-HighestAttnKeys** and **Cartridges** offer trade-offs between compression and performance, while **Summarization** and **KVzip** struggle with larger compacted sizes. The **Original Cache** baseline highlights the importance of context retention, as "No Context" methods fail to retain meaningful accuracy. The decline in performance at larger compacted sizes (e.g., 0.20) across most methods indicates a critical threshold where compression harms utility. This analysis is critical for optimizing model deployment in resource-constrained environments.