## Line Graphs: Recall@1k vs Top-k for Different Models and Quantization Methods
### Overview
The image contains three line graphs comparing the performance of different quantization methods (TurboQuant-prod and EDEN-unbiased) across three datasets: GloVe (d=200), OpenAI3 (d=1536), and OpenAI3 (d=3072). Each graph plots **Recall@1k** (y-axis) against **Top-k** (x-axis), with performance trends visualized for 2-bit and 4-bit quantization configurations.
---
### Components/Axes
1. **X-axis (Top-k)**:
- Values: 1, 2, 4, 8, 16, 32, 64 (logarithmic scale).
- Label: "Top-k" (positioned at the bottom of each graph).
2. **Y-axis (Recall@1k)**:
- Graph (a): 0.4–1.0 (increments of 0.1).
- Graph (b): 0.80–1.00 (increments of 0.05).
- Graph (c): 0.825–1.000 (increments of 0.025).
- Label: "Recall@1k" (positioned on the left of each graph).
3. **Legends**:
- Located at the bottom-right of each graph.
- Entries:
- **TurboQuant-prod 2 bits** (light blue).
- **TurboQuant-prod 4 bits** (dark blue).
- **EDEN-unbiased 2 bits** (light green).
- **EDEN-unbiased 4 bits** (dark green).
---
### Detailed Analysis
#### Graph (a): GloVe - d=200
- **Trends**:
- **TurboQuant-prod 2 bits** (light blue): Starts at ~0.38 (Top-k=1) and rises steeply to ~0.92 (Top-k=64).
- **TurboQuant-prod 4 bits** (dark blue): Begins at ~0.72 (Top-k=1) and plateaus near 1.0.
- **EDEN-unbiased 2 bits** (light green): Starts at ~0.62 (Top-k=1) and increases to ~0.98 (Top-k=64).
- **EDEN-unbiased 4 bits** (dark green): Begins at ~0.88 (Top-k=1) and stabilizes at 1.0.
- **Key Data Points**:
- At Top-k=1: TurboQuant-prod 2 bits (0.38), TurboQuant-prod 4 bits (0.72), EDEN-unbiased 2 bits (0.62), EDEN-unbiased 4 bits (0.88).
- At Top-k=64: All 4-bit methods reach ~1.0; TurboQuant-prod 2 bits reaches 0.92.
#### Graph (b): OpenAI3 - d=1536
- **Trends**:
- **TurboQuant-prod 2 bits** (light blue): Starts at ~0.80 (Top-k=1) and rises to ~0.98 (Top-k=64).
- **TurboQuant-prod 4 bits** (dark blue): Begins at ~0.90 (Top-k=1) and plateaus near 1.0.
- **EDEN-unbiased 2 bits** (light green): Starts at ~0.88 (Top-k=1) and increases to ~0.99 (Top-k=64).
- **EDEN-unbiased 4 bits** (dark green): Begins at ~0.95 (Top-k=1) and stabilizes at 1.0.
- **Key Data Points**:
- At Top-k=1: TurboQuant-prod 2 bits (0.80), TurboQuant-prod 4 bits (0.90), EDEN-unbiased 2 bits (0.88), EDEN-unbiased 4 bits (0.95).
- At Top-k=64: All 4-bit methods reach ~1.0; TurboQuant-prod 2 bits reaches 0.98.
#### Graph (c): OpenAI3 - d=3072
- **Trends**:
- **TurboQuant-prod 2 bits** (light blue): Starts at ~0.825 (Top-k=1) and rises sharply to ~0.975 (Top-k=2), then plateaus.
- **TurboQuant-prod 4 bits** (dark blue): Begins at ~0.925 (Top-k=1) and plateaus near 1.0.
- **EDEN-unbiased 2 bits** (light green): Starts at ~0.90 (Top-k=1) and increases to ~0.99 (Top-k=64).
- **EDEN-unbiased 4 bits** (dark green): Begins at ~0.975 (Top-k=1) and stabilizes at 1.0.
- **Key Data Points**:
- At Top-k=1: TurboQuant-prod 2 bits (0.825), TurboQuant-prod 4 bits (0.925), EDEN-unbiased 2 bits (0.90), EDEN-unbiased 4 bits (0.975).
- At Top-k=64: All 4-bit methods reach ~1.0; TurboQuant-prod 2 bits plateaus at 0.975.
---
### Key Observations
1. **Quantization Impact**:
- 4-bit quantization consistently outperforms 2-bit across all models and datasets.
- EDEN-unbiased methods outperform TurboQuant-prod in all cases, especially at lower Top-k values.
2. **Performance Plateaus**:
- Higher Top-k values (e.g., 16–64) show diminishing returns, with most lines converging near 1.0.
3. **Dataset-Specific Behavior**:
- GloVe (d=200) shows the steepest improvement for TurboQuant-prod 2 bits.
- OpenAI3 (d=3072) exhibits the most dramatic jump for TurboQuant-prod 2 bits between Top-k=1 and 2.
---
### Interpretation
The data demonstrates that:
- **Higher quantization (4 bits)** reduces performance degradation compared to 2-bit methods, likely due to better preservation of model precision.
- **EDEN-unbiased models** achieve superior recall, suggesting their training or architecture mitigates quantization errors more effectively than TurboQuant-prod.
- **Dataset dimensionality** influences performance: Lower-dimensional GloVe (d=200) allows faster convergence, while higher-dimensional OpenAI3 (d=3072) requires larger Top-k to reach peak performance.
Notably, the sharp rise in TurboQuant-prod 2 bits for OpenAI3 (d=3072) at Top-k=2 implies a critical threshold where recall improves significantly, highlighting the importance of tuning Top-k for high-dimensional models.
