# Technical Document Extraction: Attention Key Query Score Chart
## Chart Title
**Attention Key Query Score, a=512**
## Axes
- **X-axis (Horizontal):**
- Label: **Hidden Size**
- Range: **0 to 32768**
- Tick Marks: **0, 4096, 8192, 12288, 16384, 20480, 24576, 28672, 32768**
- **Y-axis (Vertical):**
- Label: **Throughput (TFLOP/s)**
- Range: **0 to 50**
- Tick Marks: **0, 10, 20, 30, 40, 50**
## Legend
- **Location:** Top-right corner of the chart
- **Color-Coded Labels:**
- **Blue:** h/a = 1
- **Orange:** h/a = 2
- **Green:** h/a = 4
- **Red:** h/a = 8
- **Purple:** h/a = 16
- **Brown:** h/a = 32
- **Pink:** h/a = 64
## Data Series Trends
1. **h/a = 1 (Blue Line):**
- Starts at **0 TFLOP/s** at Hidden Size = 0.
- Gradually increases to **25 TFLOP/s** at Hidden Size = 32768.
- **Trend:** Steady upward slope with minor fluctuations.
2. **h/a = 2 (Orange Line):**
- Starts at **5 TFLOP/s** at Hidden Size = 4096.
- Peaks at **15 TFLOP/s** at Hidden Size = 16384.
- Drops to **20 TFLOP/s** at Hidden Size = 20480.
- **Trend:** Initial rise, peak, then decline.
3. **h/a = 4 (Green Line):**
- Starts at **0 TFLOP/s** at Hidden Size = 0.
- Peaks at **12 TFLOP/s** at Hidden Size = 16384.
- Increases to **18 TFLOP/s** at Hidden Size = 20480.
- **Trend:** Gradual rise with a peak at 16384.
4. **h/a = 8 (Red Line):**
- Starts at **10 TFLOP/s** at Hidden Size = 4096.
- Rises sharply to **50 TFLOP/s** at Hidden Size = 32768.
- **Trend:** Steep upward slope.
5. **h/a = 16 (Purple Line):**
- Starts at **15 TFLOP/s** at Hidden Size = 8192.
- Peaks at **45 TFLOP/s** at Hidden Size = 24576.
- **Trend:** Rapid increase, then plateaus.
6. **h/a = 32 (Brown Line):**
- Starts at **10 TFLOP/s** at Hidden Size = 8192.
- Peaks at **18 TFLOP/s** at Hidden Size = 16384.
- Increases to **22 TFLOP/s** at Hidden Size = 20480.
- **Trend:** Moderate rise with a peak at 16384.
7. **h/a = 64 (Pink Line):**
- Starts at **12 TFLOP/s** at Hidden Size = 8192.
- Peaks at **20 TFLOP/s** at Hidden Size = 16384.
- Increases to **25 TFLOP/s** at Hidden Size = 20480.
- **Trend:** Steady upward slope with a peak at 16384.
## Key Observations
- **h/a = 8 (Red Line)** achieves the highest throughput (50 TFLOP/s) at the largest hidden size (32768).
- **h/a = 16 (Purple Line)** shows the steepest increase, reaching 45 TFLOP/s at 24576.
- **h/a = 2 (Orange Line)** exhibits a non-monotonic trend, peaking at 16384 before declining.
- **h/a = 1 (Blue Line)** has the lowest throughput across all hidden sizes.
## Spatial Grounding
- **Legend Position:** Top-right corner (coordinates: [x=0.85, y=0.95] relative to chart boundaries).
- **Line Colors:** All lines match their legend labels (e.g., red = h/a=8, blue = h/a=1).
## Notes
- No additional text, tables, or diagrams are present.
- All data points and trends are derived from the chart's visual representation.
- No non-English text is detected.
