## Line Graphs: Algorithm Performance Comparison Across Datasets
### Overview
Three line graphs compare algorithm performance (Recall@1@k) across different datasets (GloVe-d=200, OpenAI3-d=1536, OpenAI3-d=3072) as Top-k values increase from 1 to 64. Each graph shows five algorithms with varying bit configurations, with performance metrics plotted against Top-k values.
### Components/Axes
- **X-axis**: Top-k (logarithmic scale from 1 to 64)
- **Y-axis**: Recall@1@k (linear scale from 0.5 to 1.0 for GloVe, 0.85 to 1.0 for OpenAI3-d=1536, 0.85 to 1.0 for OpenAI3-d=3072)
- **Legends**: Positioned at bottom-right of each graph, with color-coded labels:
- **Blue**: TurboQuant_mise 2 bits
- **Dark Blue**: TurboQuant_mise 4 bits
- **Light Blue**: TurboQuant_prod 2 bits
- **Brown**: TurboQuant_prod 4 bits
- **Pink**: RaBitQ 4 bits
### Detailed Analysis
#### (a) GloVe-d=200
- **Trend**: All lines converge near Top-k=64, with RaBitQ 4 bits consistently highest (0.98-1.0). TurboQuant_prod 2 bits starts lowest (0.55 at Top-k=1) but improves sharply.
- **Key Data Points**:
- TurboQuant_mise 2 bits: 0.5 (Top-k=1) → 0.98 (Top-k=64)
- RaBitQ 4 bits: 0.85 (Top-k=1) → 1.0 (Top-k=64)
#### (b) OpenAI3-d=1536
- **Trend**: Lines diverge initially but converge at Top-k=64. RaBitQ 4 bits maintains lead (0.99-1.0). TurboQuant_prod 2 bits starts lowest (0.875 at Top-k=1).
- **Key Data Points**:
- TurboQuant_prod 4 bits: 0.89 (Top-k=1) → 0.99 (Top-k=64)
- RaBitQ 4 bits: 0.97 (Top-k=1) → 1.0 (Top-k=64)
#### (c) OpenAI3-d=3072
- **Trend**: Similar convergence pattern. RaBitQ 4 bits dominates (0.99-1.0). TurboQuant_prod 2 bits starts at 0.875 (Top-k=1).
- **Key Data Points**:
- TurboQuant_mise 4 bits: 0.97 (Top-k=1) → 0.99 (Top-k=64)
- RaBitQ 4 bits: 0.97 (Top-k=1) → 1.0 (Top-k=64)
### Key Observations
1. **Convergence**: All algorithms show diminishing performance differences at higher Top-k values.
2. **RaBitQ 4 bits**: Consistently outperforms others across all datasets.
3. **Bit Configuration Impact**: 4-bit configurations generally outperform 2-bit versions.
4. **Dataset Sensitivity**: OpenAI3-d=3072 shows the most pronounced performance gaps at low Top-k values.
### Interpretation
The data demonstrates that:
- **Algorithm Efficiency**: RaBitQ 4 bits achieves highest recall across all datasets, suggesting superior design for sparse retrieval tasks.
- **Precision Tradeoffs**: TurboQuant_prod 2 bits sacrifices initial performance for better scalability at low Top-k values.
- **Dataset Complexity**: Larger datasets (OpenAI3-d=3072) amplify performance differences between algorithms, particularly at low Top-k values.
- **Bit Depth Impact**: 4-bit configurations consistently outperform 2-bit versions, indicating that higher precision improves recall in sparse vector spaces.
The convergence patterns suggest that while algorithmic differences matter at scale, system-level optimizations (like Top-k tuning) can mitigate performance gaps in practical applications.
