## Line Chart: Overlap Score Across Layers
### Overview
The chart visualizes the overlap scores (in percentage) of three language models (LLaMA2-7B-Chat, LLaMA3-8B, Mistral-7B-v0.2) across 30 layers. Scores range from 0% to 100%, with distinct trends for each model.
### Components/Axes
- **X-axis (Layer)**: Discrete values from 0 to 30, labeled "Layer."
- **Y-axis (Overlap Score %)**: Continuous scale from 0% to 100%, labeled "Overlap Score (%)".
- **Legend**: Located at the bottom-right corner, mapping:
- Blue circles: LLaMA2-7B-Chat
- Orange squares: LLaMA3-8B
- Green triangles: Mistral-7B-v0.2
### Detailed Analysis
1. **LLaMA2-7B-Chat (Blue Circles)**:
- **Layer 0**: ~17% overlap (lowest initial score).
- **Layer 1**: Sharp rise to ~95%.
- **Layers 2–30**: Stabilizes between ~90–95%, with minor fluctuations (~92–94%).
- **Layer 30**: ~84% (notable drop from peak).
2. **LLaMA3-8B (Orange Squares)**:
- **Layer 0**: ~52% overlap.
- **Layer 1**: Peaks at ~99%.
- **Layers 2–29**: Maintains ~98–99% overlap.
- **Layer 30**: ~92% (slight decline from peak).
3. **Mistral-7B-v0.2 (Green Triangles)**:
- **Layer 0**: ~36% overlap.
- **Layer 1**: Rises to ~99%.
- **Layers 2–29**: Fluctuates between ~95–99%.
- **Layer 30**: ~80% (significant drop from peak).
### Key Observations
- **Initial Layer Disparity**: All models show low overlap at Layer 0, with LLaMA2-7B-Chat having the lowest (~17%) and LLaMA3-8B the highest (~52%).
- **Convergence**: By Layer 1, all models achieve near-100% overlap, suggesting architectural or training similarities in early layers.
- **Divergence at Final Layer**: Mistral-7B-v0.2 experiences the largest drop (~19% from Layer 29 to 30), while LLaMA3-8B retains the highest final score (~92%).
- **Stability**: LLaMA3-8B demonstrates the most consistent performance across layers, maintaining ~98–99% overlap.
### Interpretation
The data suggests that while all three models exhibit high layer overlap after the first layer, their architectural or training differences manifest in the final layers. LLaMA3-8B’s sustained high overlap indicates robust consistency, whereas Mistral-7B-v0.2’s sharp decline at Layer 30 may reflect divergent design choices in later layers. LLaMA2-7B-Chat’s rapid convergence to ~95% overlap by Layer 1 highlights its efficiency in aligning with other models early in processing. These trends could inform model selection based on layer-specific performance requirements.
