# Technical Document Extraction: Model Architecture and Prediction Flow
## Diagram Overview
The image depicts a neural network architecture for text generation with a Medusa-style multi-head prediction system. The diagram is divided into three primary sections connected by directional arrows, with a central llama icon serving as a visual motif.
---
### **1. Original Model (Left Section)**
- **Color**: Blue gradient background
- **Components**:
- **LM Head**: Rectangular box labeled "LM Head" with an arrow pointing to "Last Hidden"
- **Transformer Layers**: Rectangular box labeled "Transformer Layers" with an upward arrow to "LM Head"
- **Embedding**: Rectangular box labeled "Embedding" with an upward arrow to "Transformer Layers"
- **Input**: Text box containing the prompt:
`"What will happen if Medusa meets a llama?"`
- **Legend**:
- ❄️ (Snowflake) / 🔥 (Fire) icon above the section
---
### **2. Medusa Heads (Center Section)**
- **Color**: Pink gradient background
- **Components**:
- **Medusa Head 1**: Rectangular box labeled "Medusa Head 1" with an arrow to "Candidates"
- **Medusa Head 2**: Rectangular box labeled "Medusa Head 2" with an arrow to "Candidates"
- **Medusa Head 3**: Rectangular box labeled "Medusa Head 3" with an arrow to "Candidates"
- **Candidates Section**:
- Text box with three candidate outputs:
1. `"It is difficult not"` (✅ Green checkmark)
2. `"It' difficult a"` (❌ Red cross)
3. `"It is' not"` (❌ Red cross)
- **Legend**:
- 🔥 (Fire) icon above the section
---
### **3. Top-k Predictions (Right Section)**
- **Color**: Purple gradient background
- **Components**:
- **Top-k Predictions**: Rectangular box labeled "Top-k Predictions" with an upward arrow to "Medusa Heads"
- **Predictions**:
- `"It, I, As"`
- `"is, ', the"`
- `"difficult, is, '"`
- `"not, difficult, a"`
- **Single Step Prediction**: Text box with the output:
`"It is difficult"`
- **Legend**:
- ↑ (Upward arrow) icon labeled "TOP" above the section
---
### **4. Connections and Flow**
1. **Input → Embedding → Transformer Layers → LM Head → Last Hidden**
2. **Last Hidden → Medusa Head 1/2/3**
3. **Medusa Heads → Candidates**
4. **Candidates → Single Step Prediction**
---
### **5. Visual Elements**
- **Central Icon**: Circular logo featuring a llama with:
- Yellow mane
- White horns
- Star-shaped crown
- Black-and-white background
- **Arrows**:
- Black directional arrows connecting components
- Colored icons (❄️, 🔥, ↑) for section identification
---
### **6. Key Observations**
- The Medusa Heads generate multiple candidate outputs, with one correct prediction highlighted.
- The single-step prediction combines elements from the top candidates to form a coherent output.
- The architecture emphasizes parallel processing via multiple Medusa Heads for diverse prediction paths.
---
### **7. Missing Elements**
- No explicit axis titles, legends, or data tables present.
- No numerical data or heatmap categories visible.
- All textual information is embedded within labeled boxes or candidate lists.
---
This extraction captures all textual and structural elements from the diagram. The absence of numerical data or heatmaps limits quantitative analysis to the described architectural flow and prediction logic.
