## Bar Chart: TPS_sys Performance Across Concurrency Levels ### Overview The chart compares the transaction processing speed (TPS_sys) in tokens per second (tok/s) for three computational methods (BF16, INT4, INT4+BDR) across varying concurrency levels (8, 16, 32, 64, 128). TPS_sys increases with concurrency for all methods, but performance gaps emerge at higher levels. ### Components/Axes - **X-axis (Concurrency)**: Discrete values [8, 16, 32, 64, 128], labeled "Concurrency". - **Y-axis (TPS_sys)**: Continuous scale from 0 to 700 tok/s, labeled "TPS_sys (tok/s)". - **Legend**: - Blue: BF16 - Orange: INT4 - Green: INT4+BDR - **Bar Groups**: Three bars per concurrency level, grouped by method. ### Detailed Analysis 1. **Concurrency = 8**: - BF16: ~410 tok/s - INT4: ~450 tok/s - INT4+BDR: ~450 tok/s - *Observation*: INT4 and INT4+BDR perform identically, outperforming BF16 by ~10%. 2. **Concurrency = 16**: - BF16: ~480 tok/s - INT4: ~570 tok/s - INT4+BDR: ~570 tok/s - *Observation*: INT4/BDR methods scale linearly, while BF16 lags by ~15%. 3. **Concurrency = 32**: - BF16: ~570 tok/s - INT4: ~610 tok/s - INT4+BDR: ~610 tok/s - *Observation*: INT4/BDR maintain parity; BF16 closes the gap slightly but remains ~7% behind. 4. **Concurrency = 64**: - BF16: ~470 tok/s - INT4: ~660 tok/s - INT4+BDR: ~660 tok/s - *Observation*: BF16 drops ~18% from 32 concurrency, while INT4/BDR scale to ~660 tok/s. 5. **Concurrency = 128**: - BF16: ~560 tok/s - INT4: ~700 tok/s - INT4+BDR: ~700 tok/s - *Observation*: INT4/BDR reach peak performance; BF16 recovers slightly but remains ~20% behind. ### Key Observations - **INT4 vs. INT4+BDR**: No significant performance difference across all concurrency levels (max 1% variance). - **BF16 Limitations**: Performance plateaus at 64 concurrency and declines relative to INT4/BDR. - **Scaling Trends**: INT4/BDR exhibit linear scaling up to 128 concurrency, while BF16 shows diminishing returns. ### Interpretation The data suggests that INT4 and INT4+BDR architectures are optimized for high-concurrency workloads, maintaining consistent performance gains as concurrency increases. BF16, while competitive at lower concurrency (8–32), struggles with scalability beyond 32, likely due to architectural bottlenecks (e.g., memory bandwidth, parallelism limits). The near-identical performance of INT4 and INT4+BDR implies that the "BDR" component (possibly a data structure or optimization) provides marginal but consistent benefits. The drop in BF16 at 64 concurrency highlights a critical inefficiency, potentially making it unsuitable for high-throughput applications. This analysis underscores the importance of architectural design in scaling computational efficiency.