## Bar Charts: VRAM Usage and Average Accuracy vs. Model Parameters ### Overview The image presents two side-by-side bar charts comparing the performance of different model configurations. The left chart shows VRAM (Video RAM) usage in Gigabytes (GB), while the right chart displays Average Accuracy in percentage (%). Both charts compare four configurations: "Ours" (blue), "Ours + GPTQ" (light blue), "Original" (black), and "Original + GPTQ" (dark gray), across varying model sizes defined by the number of parameters (2.7B, 3.7B, 5.5B, and 6.7B). ### Components/Axes * **X-axis (Both Charts):** "# Parameters" with markers at 2.7B, 3.7B, 5.5B, and 6.7B. * **Y-axis (Left Chart):** "VRAM (GB)" ranging from approximately 2 GB to 13 GB, with gridlines at 2, 4, 6, 8, 10, and 12. * **Y-axis (Right Chart):** "Ave Acc (%)" ranging from approximately 52% to 65%, with gridlines at 52.5, 55, 57.5, 60, 62.5, and 65. * **Legend (Top-Left):** * Blue: "Ours" * Light Blue: "Ours + GPTQ" * Black: "Original" * Dark Gray: "Original + GPTQ" ### Detailed Analysis or Content Details **Left Chart: VRAM Usage** * **2.7B Parameters:** * Ours (Blue): Approximately 5.1 GB * Ours + GPTQ (Light Blue): Approximately 3.1 GB * Original (Black): Approximately 6.8 GB * Original + GPTQ (Dark Gray): Approximately 3.3 GB * **3.7B Parameters:** * Ours (Blue): Approximately 7.1 GB * Ours + GPTQ (Light Blue): Approximately 3.6 GB * Original (Black): Approximately 8.5 GB * Original + GPTQ (Dark Gray): Approximately 4.1 GB * **5.5B Parameters:** * Ours (Blue): Approximately 10.3 GB * Ours + GPTQ (Light Blue): Approximately 4.1 GB * Original (Black): Approximately 11.5 GB * Original + GPTQ (Dark Gray): Approximately 4.6 GB * **6.7B Parameters:** * Ours (Blue): Approximately 12.5 GB * Ours + GPTQ (Light Blue): Approximately 4.7 GB * Original (Black): Approximately 12.8 GB * Original + GPTQ (Dark Gray): Approximately 5.2 GB **Right Chart: Average Accuracy** * **2.7B Parameters:** * Ours (Blue): Approximately 54.5% * Ours + GPTQ (Light Blue): Approximately 55.2% * Original (Black): Approximately 54.8% * Original + GPTQ (Dark Gray): Approximately 55.5% * **3.7B Parameters:** * Ours (Blue): Approximately 57.2% * Ours + GPTQ (Light Blue): Approximately 57.5% * Original (Black): Approximately 57.0% * Original + GPTQ (Dark Gray): Approximately 58.0% * **5.5B Parameters:** * Ours (Blue): Approximately 61.5% * Ours + GPTQ (Light Blue): Approximately 61.8% * Original (Black): Approximately 60.5% * Original + GPTQ (Dark Gray): Approximately 62.0% * **6.7B Parameters:** * Ours (Blue): Approximately 64.0% * Ours + GPTQ (Light Blue): Approximately 64.5% * Original (Black): Approximately 63.0% * Original + GPTQ (Dark Gray): Approximately 65.0% ### Key Observations * **VRAM Usage:** VRAM usage increases consistently with the number of parameters for all configurations. "Ours" and "Original" consistently require more VRAM than their respective "+ GPTQ" counterparts. * **Accuracy:** Accuracy generally increases with the number of parameters. "+ GPTQ" configurations show a slight accuracy improvement over their base configurations ("Ours" and "Original"). * **GPTQ Impact:** Applying GPTQ significantly reduces VRAM usage across all model sizes, with a relatively small impact on accuracy. * **Comparison of "Ours" vs "Original":** "Original" models generally require slightly more VRAM than "Ours" models for the same number of parameters, but the accuracy is comparable. ### Interpretation The data suggests that GPTQ is an effective quantization technique for reducing the memory footprint of large language models without substantial performance degradation. The consistent reduction in VRAM usage across all model sizes indicates that GPTQ's benefits scale with model complexity. The slight accuracy improvements observed with "+ GPTQ" configurations could be attributed to the quantization process itself or the specific implementation details. The comparison between "Ours" and "Original" models suggests that there are architectural or implementation differences that affect VRAM usage, but not necessarily accuracy. The overall trend of increasing VRAM usage and accuracy with more parameters highlights the trade-off between model size, computational resources, and performance. The charts provide a clear visual representation of this trade-off, allowing for informed decisions about model selection and optimization.