## Line Charts: Performance Metrics vs. Draft Length
### Overview
The image presents a series of line charts comparing the performance of different models (Llama 2-70B, Llama 2-Chat-70B, Dolly) on various tasks (WMT, XSum) across different metrics (block efficiency, MBSU, token rate, accuracy) as a function of draft length. Four different methods (SD, SpecTr, RSD-C (ours), RSD-S (ours)) are compared.
### Components/Axes
* **X-axis:** Draft Length, with values 2, 3, 4, and 5.
* **Y-axes:**
* Block Efficiency: Ranges from approximately 1.6 to 4.0.
* MBSU: Ranges from approximately 1.2 to 3.4.
* Token Rate: Ranges from approximately 1.2 to 2.4.
* Accuracy: Ranges from approximately 0.7 to 1.3.
* **Models (Rows):**
* Llama 2-70B, WMT
* Llama 2-70B, XSum
* Llama 2-Chat-70B, WMT
* Llama 2-Chat-70B, XSum
* Dolly
* **Metrics (Columns):** Block efficiency, MBSU, token rate, accuracy.
* **Legend (Bottom):**
* Yellow dotted line: SD
* Red dashed line with plus markers: SpecTr
* Green dashed line with diamond markers: RSD-C (ours)
* Blue solid line with circle markers: RSD-S (ours)
### Detailed Analysis
**Llama 2-70B, WMT**
* **Block Efficiency:**
* SD (Yellow): Increases from ~1.6 to ~1.8.
* SpecTr (Red): Increases from ~1.7 to ~1.9.
* RSD-C (Green): Increases from ~1.8 to ~2.1.
* RSD-S (Blue): Increases from ~1.9 to ~2.3.
* **MBSU:**
* SD (Yellow): Increases from ~1.6 to ~1.8.
* SpecTr (Red): Increases from ~1.7 to ~1.9.
* RSD-C (Green): Increases from ~1.8 to ~2.2.
* RSD-S (Blue): Increases from ~1.9 to ~2.4.
* **Token Rate:**
* SD (Yellow): Increases from ~1.3 to ~1.4.
* SpecTr (Red): Increases from ~1.4 to ~1.45.
* RSD-C (Green): Increases from ~1.5 to ~1.6.
* RSD-S (Blue): Increases from ~1.5 to ~1.7.
* **Accuracy:**
* All methods maintain a constant accuracy of approximately 1.0.
**Llama 2-70B, XSum**
* **Block Efficiency:**
* SD (Yellow): Increases from ~2.2 to ~2.8.
* SpecTr (Red): Increases from ~2.3 to ~2.9.
* RSD-C (Green): Increases from ~2.4 to ~3.3.
* RSD-S (Blue): Increases from ~2.5 to ~3.9.
* **MBSU:**
* SD (Yellow): Increases from ~2.2 to ~2.4.
* SpecTr (Red): Increases from ~2.3 to ~2.6.
* RSD-C (Green): Increases from ~2.5 to ~3.2.
* RSD-S (Blue): Increases from ~2.6 to ~3.8.
* **Token Rate:**
* SD (Yellow): Increases from ~1.6 to ~1.7.
* SpecTr (Red): Increases from ~1.7 to ~1.9.
* RSD-C (Green): Increases from ~1.9 to ~2.2.
* RSD-S (Blue): Increases from ~2.0 to ~2.3.
* **Accuracy:**
* All methods maintain a constant accuracy of approximately 1.0.
**Llama 2-Chat-70B, WMT**
* **Block Efficiency:**
* SD (Yellow): Increases from ~1.6 to ~1.7.
* SpecTr (Red): Increases from ~1.7 to ~1.9.
* RSD-C (Green): Increases from ~1.8 to ~2.1.
* RSD-S (Blue): Increases from ~1.9 to ~2.5.
* **MBSU:**
* SD (Yellow): Increases from ~1.5 to ~1.6.
* SpecTr (Red): Increases from ~1.4 to ~1.5.
* RSD-C (Green): Increases from ~1.4 to ~1.6.
* RSD-S (Blue): Increases from ~1.7 to ~2.2.
* **Token Rate:**
* SD (Yellow): Decreases from ~1.4 to ~1.3.
* SpecTr (Red): Decreases from ~1.4 to ~1.3.
* RSD-C (Green): Decreases from ~1.5 to ~1.4.
* RSD-S (Blue): Increases from ~1.5 to ~1.6.
* **Accuracy:**
* All methods maintain a constant accuracy of approximately 1.0.
**Llama 2-Chat-70B, XSum**
* **Block Efficiency:**
* SD (Yellow): Increases from ~1.8 to ~2.0.
* SpecTr (Red): Increases from ~1.9 to ~2.1.
* RSD-C (Green): Increases from ~2.0 to ~2.4.
* RSD-S (Blue): Increases from ~2.1 to ~2.6.
* **MBSU:**
* SD (Yellow): Increases from ~1.7 to ~1.8.
* SpecTr (Red): Increases from ~1.7 to ~1.8.
* RSD-C (Green): Increases from ~1.8 to ~2.0.
* RSD-S (Blue): Increases from ~1.9 to ~2.4.
* **Token Rate:**
* SD (Yellow): Increases from ~1.4 to ~1.5.
* SpecTr (Red): Increases from ~1.4 to ~1.5.
* RSD-C (Green): Increases from ~1.5 to ~1.7.
* RSD-S (Blue): Increases from ~1.7 to ~2.0.
* **Accuracy:**
* All methods maintain a constant accuracy of approximately 1.0.
**Dolly**
* **Block Efficiency:**
* SD (Yellow): Increases from ~2.0 to ~2.2.
* SpecTr (Red): Increases from ~2.1 to ~2.3.
* RSD-C (Green): Increases from ~2.2 to ~2.5.
* RSD-S (Blue): Increases from ~2.3 to ~2.7.
* **MBSU:**
* SD (Yellow): Increases from ~1.6 to ~1.8.
* SpecTr (Red): Increases from ~1.7 to ~1.8.
* RSD-C (Green): Increases from ~1.8 to ~2.0.
* RSD-S (Blue): Increases from ~1.9 to ~2.2.
* **Token Rate:**
* SD (Yellow): Increases from ~1.4 to ~1.5.
* SpecTr (Red): Increases from ~1.5 to ~1.6.
* RSD-C (Green): Increases from ~1.6 to ~1.8.
* RSD-S (Blue): Increases from ~1.7 to ~2.0.
* **Accuracy:**
* All methods maintain a constant accuracy of approximately 1.0.
### Key Observations
* RSD-S (ours) generally outperforms the other methods (SD, SpecTr, RSD-C) in terms of block efficiency, MBSU, and token rate across all models and tasks.
* Accuracy remains relatively constant across all methods and draft lengths.
* The performance gains from increasing draft length tend to diminish as the draft length increases from 4 to 5.
* The Llama 2-70B model generally achieves higher block efficiency and MBSU scores compared to the Llama 2-Chat-70B model.
### Interpretation
The data suggests that the RSD-S method is more effective at improving block efficiency, MBSU, and token rate compared to the other methods tested. The consistent accuracy across all methods indicates that these improvements are not achieved at the expense of accuracy. The diminishing returns from increasing draft length suggest that there is an optimal draft length beyond which further increases provide minimal benefit. The differences in performance between the Llama 2-70B and Llama 2-Chat-70B models may be attributed to differences in their architectures or training data.
