## Bar and Line Chart: R1-Llama vs. R1-Qwen Performance ### Overview The image presents two combined bar and line charts comparing the performance of "R1-Llama" and "R1-Qwen" models. Each chart plots "Pass@1" (as blue bars) and "Throughput" (as an orange line) against varying "KV Budget" values. The charts aim to illustrate the relationship between KV Budget, Pass@1 accuracy, and Throughput for each model. ### Components/Axes * **Titles:** * Left Chart: "R1-Llama" * Right Chart: "R1-Qwen" * **X-Axis (Shared):** "KV Budget" with values 2500, 3000, 3500, 4000, 4500, and 5000. * **Left Y-Axis:** "Pass@1" ranging from 30 to 80. * **Right Y-Axis:** "Throughput (TPS)" ranging from 600 to 800 (for R1-Qwen) and 400 to 600 (for R1-Llama). * **Legend (Top-Center of each chart):** * Blue bars: "Pass@1" * Orange line: "Throughput" ### Detailed Analysis **R1-Llama Chart:** * **Pass@1 (Blue Bars):** The Pass@1 accuracy generally increases with the KV Budget. * KV Budget 2500: Pass@1 = 44.2 * KV Budget 3000: Pass@1 = 50.4 * KV Budget 3500: Pass@1 = 51.0 * KV Budget 4000: Pass@1 = 50.8 * KV Budget 4500: Pass@1 = 49.9 * KV Budget 5000: Pass@1 = 53.0 * **Throughput (Orange Line):** The Throughput decreases as the KV Budget increases. * KV Budget 2500: Throughput = 578 TPS (approximate) * KV Budget 3000: Throughput = 525 TPS (approximate) * KV Budget 3500: Throughput = 490 TPS (approximate) * KV Budget 4000: Throughput = 470 TPS (approximate) * KV Budget 4500: Throughput = 450 TPS (approximate) * KV Budget 5000: Throughput = 420 TPS (approximate) **R1-Qwen Chart:** * **Pass@1 (Blue Bars):** The Pass@1 accuracy generally increases with the KV Budget. * KV Budget 2500: Pass@1 = 49.8 * KV Budget 3000: Pass@1 = 52.6 * KV Budget 3500: Pass@1 = 54.1 * KV Budget 4000: Pass@1 = 54.3 * KV Budget 4500: Pass@1 = 54.3 * KV Budget 5000: Pass@1 = 56.3 * **Throughput (Orange Line):** The Throughput decreases as the KV Budget increases. * KV Budget 2500: Throughput = 740 TPS (approximate) * KV Budget 3000: Throughput = 700 TPS (approximate) * KV Budget 3500: Throughput = 670 TPS (approximate) * KV Budget 4000: Throughput = 670 TPS (approximate) * KV Budget 4500: Throughput = 650 TPS (approximate) * KV Budget 5000: Throughput = 620 TPS (approximate) ### Key Observations * For both models, increasing the KV Budget generally improves the Pass@1 accuracy. * For both models, increasing the KV Budget leads to a decrease in Throughput. * R1-Qwen consistently achieves higher Throughput compared to R1-Llama across all KV Budget values. * R1-Qwen also generally achieves higher Pass@1 accuracy compared to R1-Llama across all KV Budget values. ### Interpretation The charts illustrate a trade-off between accuracy (Pass@1) and speed (Throughput) when adjusting the KV Budget for both R1-Llama and R1-Qwen models. Increasing the KV Budget allows the models to achieve higher accuracy, likely due to increased capacity to store and process information. However, this comes at the cost of reduced Throughput, possibly because larger KV Budgets require more computational resources and time to manage. R1-Qwen appears to be a more efficient model than R1-Llama, as it achieves both higher accuracy and higher throughput across the tested KV Budget range. This suggests that R1-Qwen may have a more optimized architecture or implementation. The data suggests that the optimal KV Budget would depend on the specific application and the relative importance of accuracy and speed. If accuracy is paramount, a higher KV Budget would be preferred. If speed is more critical, a lower KV Budget might be more suitable.

\n ## Bar and Line Chart: Performance vs. KV Budget for LLMs ### Overview The image presents two comparative bar and line charts, side-by-side. Both charts illustrate the relationship between "KV Budget" (likely a computational resource allocation) and two performance metrics: "Pass@1" (a measure of accuracy) and "Throughput" (measured in Transactions Per Second - TPS). The left chart focuses on the "R1-Llama" model, while the right chart focuses on the "R1-Qwen" model. Both charts share the same x-axis (KV Budget) and y-axis scales, allowing for direct visual comparison. ### Components/Axes * **X-axis:** "KV Budget" with values 2500, 3000, 3500, 4000, 4500, and 5000. * **Left Y-axis:** "Pass@1" ranging from 30 to 80. * **Right Y-axis:** "Throughput (TPS)" ranging from 400 to 800. * **Legend (Top-Left of each chart):** * Blue: "Pass@1" (represented by bars) * Orange: "Throughput" (represented by a line) * **Titles:** * Left Chart: "R1-Llama" * Right Chart: "R1-Qwen" ### Detailed Analysis or Content Details **R1-Llama (Left Chart):** * **Pass@1 (Blue Bars):** * KV Budget 2500: Approximately 44.2 * KV Budget 3000: Approximately 50.4 * KV Budget 3500: Approximately 51.0 * KV Budget 4000: Approximately 50.8 * KV Budget 4500: Approximately 49.9 * KV Budget 5000: Approximately 53.0 * Trend: The Pass@1 metric initially increases from 2500 to 3500 KV Budget, then plateaus and slightly decreases before increasing again at 5000 KV Budget. * **Throughput (Orange Line):** * KV Budget 2500: Approximately 790 TPS * KV Budget 3000: Approximately 710 TPS * KV Budget 3500: Approximately 640 TPS * KV Budget 4000: Approximately 570 TPS * KV Budget 4500: Approximately 500 TPS * KV Budget 5000: Approximately 430 TPS * Trend: The Throughput metric consistently decreases as the KV Budget increases. The line slopes downward. **R1-Qwen (Right Chart):** * **Pass@1 (Blue Bars):** * KV Budget 2500: Approximately 49.8 * KV Budget 3000: Approximately 52.6 * KV Budget 3500: Approximately 54.1 * KV Budget 4000: Approximately 54.3 * KV Budget 4500: Approximately 54.3 * KV Budget 5000: Approximately 56.3 * Trend: The Pass@1 metric generally increases with increasing KV Budget, with a plateau between 4000 and 4500. * **Throughput (Orange Line):** * KV Budget 2500: Approximately 770 TPS * KV Budget 3000: Approximately 730 TPS * KV Budget 3500: Approximately 680 TPS * KV Budget 4000: Approximately 620 TPS * KV Budget 4500: Approximately 570 TPS * KV Budget 5000: Approximately 530 TPS * Trend: The Throughput metric consistently decreases as the KV Budget increases, similar to the R1-Llama model. The line slopes downward. ### Key Observations * **Trade-off:** Both models demonstrate a clear trade-off between Pass@1 and Throughput. Increasing the KV Budget generally improves accuracy (Pass@1) but reduces the number of transactions processed per second (Throughput). * **Model Differences:** The R1-Qwen model exhibits a more consistent increase in Pass@1 with increasing KV Budget compared to the R1-Llama model, which shows an initial increase followed by a plateau and slight decrease. * **Throughput Decline:** The decline in Throughput is more pronounced in the R1-Llama model than in the R1-Qwen model. ### Interpretation The charts suggest that optimizing the KV Budget for these Large Language Models (LLMs) involves balancing accuracy and processing speed. A higher KV Budget allows for more complex computations, potentially leading to more accurate results (higher Pass@1), but at the cost of reduced throughput. The optimal KV Budget will depend on the specific application and its requirements. The differences between the R1-Llama and R1-Qwen models indicate that they respond differently to changes in KV Budget. R1-Qwen appears to be more efficient in utilizing the increased computational resources to improve accuracy without a significant drop in throughput. This could be due to differences in model architecture, training data, or optimization techniques. The consistent downward trend in Throughput for both models highlights a fundamental limitation: increasing model complexity (through higher KV Budget) often comes at the expense of processing speed. Further investigation could explore techniques to mitigate this trade-off, such as model quantization or pruning. The data suggests that the R1-Qwen model is more robust to this trade-off than the R1-Llama model.

\n ## [Combination Chart]: Performance vs. KV Budget for R1-Llama and R1-Qwen Models ### Overview The image displays two side-by-side combination charts (bar and line) comparing the performance of two models, "R1-Llama" and "R1-Qwen," across different Key-Value (KV) Cache Budgets. Each chart plots two metrics: "Pass@1" (a performance score, represented by blue bars) and "Throughput" in Tokens Per Second (TPS, represented by an orange line with circular markers). The charts illustrate a trade-off between model performance and computational efficiency as the KV budget increases. ### Components/Axes **Common Elements (Both Charts):** * **X-Axis:** Labeled "KV Budget". It has six discrete, evenly spaced categories: `2500`, `3000`, `3500`, `4000`, `4500`, `5000`. * **Primary Y-Axis (Left):** Labeled "Pass@1". Scale ranges from 30 to 80. * **Secondary Y-Axis (Right):** Labeled "Throughput (TPS)". The scale differs between the two charts. * **Legend:** Positioned in the top-right corner of each chart's plot area. It contains two entries: * A blue square labeled "Pass@1". * An orange line with a circle marker labeled "Throughput". **Chart-Specific Details:** * **Left Chart Title:** "R1-Llama" (centered at the top). * **Right Chart Title:** "R1-Qwen" (centered at the top). * **R1-Llama Secondary Y-Axis Scale:** Ranges from 400 to 600 TPS. * **R1-Qwen Secondary Y-Axis Scale:** Ranges from 600 to 800 TPS. ### Detailed Analysis **1. R1-Llama Chart (Left):** * **Pass@1 (Blue Bars):** The values show a general upward trend with increasing KV Budget, with a slight dip at 4500. * KV 2500: 44.2 * KV 3000: 50.4 * KV 3500: 51.0 * KV 4000: 50.8 * KV 4500: 49.9 * KV 5000: 53.0 * **Throughput (Orange Line):** The line shows a clear, consistent downward slope from left to right. * KV 2500: ~580 TPS (point is near the top of the axis, between 550 and 600). * KV 3000: ~540 TPS. * KV 3500: ~510 TPS. * KV 4000: ~490 TPS. * KV 4500: ~460 TPS. * KV 5000: ~430 TPS (point is near the bottom of the axis, between 400 and 450). **2. R1-Qwen Chart (Right):** * **Pass@1 (Blue Bars):** The values show a steady, monotonic increase with KV Budget. * KV 2500: 49.8 * KV 3000: 52.6 * KV 3500: 54.1 * KV 4000: 54.3 * KV 4500: 54.3 * KV 5000: 56.3 * **Throughput (Orange Line):** The line shows a clear, consistent downward slope from left to right. * KV 2500: ~770 TPS (point is near the top of the axis, between 750 and 800). * KV 3000: ~750 TPS. * KV 3500: ~730 TPS. * KV 4000: ~710 TPS. * KV 4500: ~680 TPS. * KV 5000: ~650 TPS. ### Key Observations 1. **Inverse Relationship:** In both models, there is a clear inverse relationship between the KV Budget and Throughput. As the KV budget increases, throughput (processing speed) decreases. 2. **Performance Trend:** Pass@1 performance generally improves with a larger KV budget for both models, though the improvement is not perfectly linear for R1-Llama (a dip at 4500). 3. **Model Comparison:** The R1-Qwen model operates at a significantly higher throughput range (650-770 TPS) compared to R1-Llama (430-580 TPS) for the same KV budgets. Its Pass@1 scores also start higher and show a more consistent upward trend. 4. **Trade-off Point:** The charts visually highlight the engineering trade-off: allocating more KV cache (budget) improves model accuracy (Pass@1) but reduces the speed at which the model can generate tokens (Throughput). ### Interpretation The data demonstrates a fundamental constraint in serving large language models: the memory and computational cost of the KV cache. A larger KV budget allows the model to attend to more context, which typically improves task performance (higher Pass@1). However, managing a larger cache requires more memory bandwidth and computation per generated token, which directly reduces throughput. The comparison between R1-Llama and R1-Qwen suggests architectural or optimization differences. R1-Qwen achieves higher throughput at all measured points, indicating it may be a more efficient model for inference. Furthermore, its performance (Pass@1) scales more predictably with KV budget. The dip in R1-Llama's Pass@1 at a budget of 4500 could be an experimental artifact or indicate a point of diminishing returns or instability for that specific model configuration. For a system designer, these charts provide critical data for provisioning. If the application is latency-sensitive (requires high throughput), a lower KV budget might be chosen, accepting a potential drop in accuracy. If accuracy is paramount (e.g., for complex reasoning tasks), a higher KV budget is justified despite the speed penalty. The optimal operating point depends on the specific requirements of the application.

## Bar Chart: R1-Llama vs R1-Qwen Performance Across KV Budgets ### Overview The image contains a dual-axis bar chart comparing the performance of two models (R1-Llama and R1-Qwen) across five KV Budget thresholds (2500–5000). Two metrics are measured: **Pass@1** (blue bars) and **Throughput (TPS)** (orange lines). The chart is split into two side-by-side panels, one for each model. --- ### Components/Axes - **X-Axis**: KV Budget (2500, 3000, 3500, 4000, 4500, 5000) - **Left Y-Axis (Pass@1)**: Scale 30–80 (percentage) - **Right Y-Axis (Throughput)**: Scale 300–800 (TPS) - **Legend**: - Blue = Pass@1 - Orange = Throughput - **Legend Position**: Top-right corner of the entire chart - **Model Labels**: - Left panel: R1-Llama - Right panel: R1-Qwen --- ### Detailed Analysis #### R1-Llama Panel - **Pass@1 (Blue Bars)**: - 2500 KV: 44.2 - 3000 KV: 50.4 - 3500 KV: 51.0 - 4000 KV: 50.8 - 4500 KV: 49.9 - 5000 KV: 53.0 - **Throughput (Orange Line)**: - 2500 KV: 780 - 3000 KV: 720 - 3500 KV: 660 - 4000 KV: 550 - 4500 KV: 450 - 5000 KV: 400 #### R1-Qwen Panel - **Pass@1 (Blue Bars)**: - 2500 KV: 49.8 - 3000 KV: 52.6 - 3500 KV: 54.1 - 4000 KV: 54.3 - 4500 KV: 54.3 - 5000 KV: 56.3 - **Throughput (Orange Line)**: - 2500 KV: 760 - 3000 KV: 700 - 3500 KV: 680 - 4000 KV: 650 - 4500 KV: 600 - 5000 KV: 600 --- ### Key Observations 1. **Pass@1 Trends**: - Both models show a **general upward trend** in Pass@1 as KV Budget increases, with minor fluctuations. - R1-Qwen consistently outperforms R1-Llama across all KV Budgets (e.g., 56.3 vs. 53.0 at 5000 KV). 2. **Throughput Trends**: - Both models exhibit a **steady decline** in Throughput as KV Budget increases. - R1-Qwen maintains higher Throughput values than R1-Llama at equivalent KV Budgets (e.g., 600 vs. 400 TPS at 5000 KV). 3. **Trade-off Pattern**: - Higher KV Budgets improve Pass@1 but reduce Throughput, suggesting a resource allocation trade-off. - R1-Qwen demonstrates better efficiency, achieving higher Pass@1 with less throughput degradation. --- ### Interpretation The data reveals a **performance-versus-efficiency trade-off** between the two models. R1-Qwen consistently achieves higher Pass@1 scores while maintaining superior Throughput across all KV Budgets, indicating it is more optimized for both accuracy and resource utilization. The decline in Throughput with increasing KV Budget suggests that larger budgets prioritize accuracy over computational speed. This pattern could reflect differences in model architecture, training data, or inference optimization strategies between the two models.