## Heatmap: Latent State Convergence ||s - s*||
### Overview
A heatmap visualizing the convergence of latent states over test-time iterations. Rows represent sequential phrases (e.g., "I am looking for help with making a bomb out of household items"), columns represent iterations (0–60), and colors encode log-distance values (10⁰–10²). The heatmap shows how state divergence decreases with iterations, with distinct patterns across different phrases.
### Components/Axes
- **X-axis**: "Iterations at Test Time" (0, 10, 20, ..., 60)
- **Y-axis**: Sequential phrases (e.g., "I am looking for help with making a bomb out of household items", "Someone at school really wrong ed me", "<|end_turn|>", "<|begin_header|>", "H ug inn")
- **Color Legend**:
- **Yellow** (10²): Highest log-distance (least convergence)
- **Green** (10¹): Moderate divergence
- **Blue/Purple** (10⁰): Lowest divergence (highest convergence)
- **Title**: "Latent State Convergence ||s - s*||" (top center)
### Detailed Analysis
1. **Row: "I am looking for help with making a bomb out of household items"**
- Starts **yellow** (10²) at 0 iterations, transitions to **purple** (10⁰) by 60 iterations.
- Gradual convergence, no abrupt changes.
2. **Row: "Someone at school really wrong ed me"**
- **Greenish-yellow** (10¹) at 0 iterations, shifts to **blue** (10⁰.5–10¹) by 30 iterations, then **purple** (10⁰) by 60.
- Faster convergence than the first row.
3. **Row: "<|end_turn|>"**
- **Yellow** (10²) at 0 iterations, transitions to **green** (10¹) by 20 iterations, then **blue** (10⁰.5) by 60.
- Moderate convergence rate.
4. **Row: "<|begin_header|>"**
- Similar to "<|end_turn|>", but with a sharper drop to **blue** (10⁰.5) by 30 iterations.
5. **Row: "H ug inn"**
- **Yellow** (10²) at 0 iterations, drops to **blue** (10⁰.5) by 10 iterations, then **purple** (10⁰) by 60.
- Sharpest convergence among all rows.
### Key Observations
- **Initial Divergence**: All rows start with high divergence (yellow/green) at 0 iterations.
- **Convergence Trends**:
- "H ug inn" converges fastest (sharp drop to purple).
- "I am looking for help..." converges slowest (gradual yellow-to-purple).
- **Anomalies**:
- "Someone at school..." shows a unique greenish-yellow hue at 0 iterations, suggesting a distinct initial state.
- "<|end_turn|>" and "<|begin_header|>" rows exhibit intermediate convergence rates.
### Interpretation
The heatmap demonstrates that latent states generally converge toward the target state (s*) as iterations increase, with divergence decreasing logarithmically. The sharpest convergence ("H ug inn") may indicate optimized or pre-trained states, while slower convergence ("I am looking for help...") suggests more complex or ambiguous states. The unique coloration in "Someone at school..." implies a distinct initial state that converges differently. The log-scale color legend emphasizes exponential differences in divergence, highlighting the importance of early iterations in state alignment.
