## Heatmap: Average JS Divergence Across Layers and Categories
### Overview
The image is a heatmap visualizing the average JS divergence across three categories ("Subj.", "Attn.", "Last.") and 31 layers (0–30). The color intensity represents divergence values, with darker blues indicating higher divergence (0.6) and lighter blues/white indicating lower divergence (0.1).
### Components/Axes
- **Y-Axis (Categories)**:
- "Subj." (Subject)
- "Attn." (Attention)
- "Last." (Last)
- **X-Axis (Layers)**:
- Labeled "Layer" with integer markers from 0 to 30 in increments of 2.
- **Color Bar (Legend)**:
- Positioned on the right, labeled "Avg JS Divergence."
- Gradient from light blue (0.1) to dark blue (0.6).
### Detailed Analysis
- **Subj. (Subject)**:
- Dark blue bars dominate layers 0–10, indicating high divergence (0.4–0.6).
- Gradual lightening from layer 10 onward, with values dropping to ~0.2 by layer 30.
- **Attn. (Attention)**:
- Uniformly light blue across all layers, suggesting consistently low divergence (~0.1–0.2).
- **Last. (Last)**:
- Light blue in layers 0–10, transitioning to medium blue (0.3–0.4) from layer 10–20.
- Peaks at dark blue (~0.5) in layers 20–25, then fades to light blue by layer 30.
### Key Observations
1. **Subj.** shows the highest divergence in early layers (0–10), with a sharp decline afterward.
2. **Attn.** maintains the lowest divergence across all layers, with minimal variation.
3. **Last.** exhibits a mid-layer peak (20–25) with the highest divergence values, followed by a decline.
### Interpretation
- The heatmap suggests that **Subject** variability is most pronounced in early layers, possibly indicating initial processing or feature extraction stages.
- **Attention** remains stable, implying consistent focus or weighting across layers.
- **Last.** divergence peaks in mid-layers (20–25), which could reflect a critical phase of integration or decision-making in the modeled system.
- The divergence patterns may correlate with architectural design choices (e.g., layer depth, attention mechanisms) in neural networks or similar computational models.
