## Bar Chart: Normalized Decoding Speed Across Verification Widths and Methods ### Overview The image is a grouped bar chart comparing the normalized decoding speed of four methods (Sequential, Medusa, EM+Medusa, Ghidorah) across four benchmarks (MT-bench, GSM8K, MBPP, Human-Eval) at varying verification widths (4, 8, 16, 32, 64). Each subplot (a-d) represents a benchmark, with bars grouped by verification width and colored by method. ### Components/Axes - **X-axis**: Verification Width (4, 8, 16, 32, 64) - **Y-axis**: Normalized Decoding Speed (0–6) - **Legend**: - **Sequential**: Brown (solid) - **Medusa**: Blue (diagonal lines) - **EM+Medusa**: Orange (crosshatch) - **Ghidorah**: Green (diagonal stripes) - **Subplots**: - (a) MT-bench - (b) GSM8K - (c) MBPP - (d) Human-Eval ### Detailed Analysis #### (a) MT-bench - **Trend**: Ghidorah (green) consistently outperforms others, peaking at ~5.5 at width 16. EM+Medusa (orange) follows closely (~5.0 at width 16). Medusa (blue) and Sequential (brown) lag, with values ~3.0–4.0. - **Values**: - Width 4: Ghidorah ~5.0, EM+Medusa ~4.5, Medusa ~3.0, Sequential ~1.0 - Width 64: Ghidorah ~5.2, EM+Medusa ~4.8, Medusa ~3.5, Sequential ~1.2 #### (b) GSM8K - **Trend**: EM+Medusa (orange) leads at width 32 (~5.5), while Ghidorah (green) dominates at width 16 (~5.8). Medusa (blue) and Sequential (brown) remain lower (~3.0–4.0). - **Values**: - Width 8: EM+Medusa ~4.7, Ghidorah ~5.3, Medusa ~3.2, Sequential ~1.1 - Width 64: EM+Medusa ~5.0, Ghidorah ~5.4, Medusa ~3.8, Sequential ~1.3 #### (c) MBPP - **Trend**: Ghidorah (green) maintains the highest speed (~5.5 at width 32), with EM+Medusa (orange) slightly behind (~5.0). Medusa (blue) and Sequential (brown) show minimal improvement with width. - **Values**: - Width 4: Ghidorah ~5.2, EM+Medusa ~4.6, Medusa ~2.8, Sequential ~1.0 - Width 64: Ghidorah ~5.3, EM+Medusa ~4.9, Medusa ~3.5, Sequential ~1.2 #### (d) Human-Eval - **Trend**: Ghidorah (green) peaks at width 32 (~5.7), while EM+Medusa (orange) declines slightly at width 64 (~4.8). Medusa (blue) and Sequential (brown) show weak scaling. - **Values**: - Width 16: Ghidorah ~5.6, EM+Medusa ~5.1, Medusa ~3.0, Sequential ~1.1 - Width 64: Ghidorah ~5.4, EM+Medusa ~4.8, Medusa ~3.7, Sequential ~1.3 ### Key Observations 1. **Ghidorah Dominance**: Outperforms all methods in three benchmarks (MT-bench, GSM8K, MBPP) and matches EM+Medusa in Human-Eval. 2. **EM+Medusa Scalability**: Shows strong performance in GSM8K and Human-Eval but lags in MT-bench. 3. **Medusa Limitations**: Consistently underperforms across benchmarks, with minimal improvement at higher widths. 4. **Sequential Baseline**: Remains the weakest method, with negligible gains as verification width increases. ### Interpretation The data suggests **Ghidorah** is the most efficient method for decoding across diverse tasks, likely due to optimized parallelization or architectural advantages. **EM+Medusa** excels in logic-heavy tasks (GSM8K, Human-Eval) but struggles with MT-bench, indicating task-specific trade-offs. **Medusa** and **Sequential** methods underperform, highlighting their inefficiency in scaling with verification width. The lack of linear scaling for all methods implies computational overhead at higher widths, possibly due to memory or processing constraints. These results underscore the importance of method selection based on task requirements and resource constraints.