## Embedding-based Distillation (L_distill)
### Overview
The image displays three line graphs, each representing different metrics related to embedding-based distillation. The graphs are labeled as "Embedding-based Distillation (L_distill)", "InfoNCE (L_NCE)", and "Score-based Distillation (L_score)" respectively.
### Components/Axes
- **X-axis**: Training Steps
- **Y-axis**: Average nDCG@10
- **Legend**:
- Blue line: 1e-4 lr
- Orange line: 1e-5 lr
### Detailed Analysis or ### Content Details
- **Embedding-based Distillation (L_distill)**: The blue line shows a steady increase in the average nDCG@10 metric as the number of training steps increases. The orange line, representing 1e-5 lr, shows a slightly higher nDCG@10 metric compared to 1e-4 lr throughout the training process.
- **InfoNCE (L_NCE)**: The orange line shows a slight increase in the average nDCG@10 metric as the number of training steps increases. The blue line, representing 1e-4 lr, shows a more significant increase compared to 1e-5 lr.
- **Score-based Distillation (L_score)**: The blue line shows a slight increase in the average nDCG@10 metric as the number of training steps increases. The orange line, representing 1e-4 lr, shows a more significant increase compared to 1e-5 lr.
### Key Observations
- The blue line (1e-4 lr) consistently shows higher nDCG@10 values compared to the orange line (1e-5 lr) across all three metrics.
- The orange line (1e-5 lr) shows a slight increase in nDCG@10 values, but the difference is not as pronounced as with the blue line.
### Interpretation
The data suggests that a learning rate of 1e-4 lr is more effective for embedding-based distillation, InfoNCE, and score-based distillation compared to a learning rate of 1e-5 lr. The slight increase in nDCG@10 values with a higher learning rate in InfoNCE and score-based distillation indicates that a higher learning rate might help in fine-tuning the model more effectively. However, the difference in performance between the two learning rates is not significant enough to warrant a change in the learning rate.
