## Bar Charts: VRAM Usage and Average Accuracy vs. Number of Parameters ### Overview The image presents two bar charts side-by-side. The left chart displays VRAM usage (in GB) as a function of the number of parameters, while the right chart shows the average accuracy (in %) as a function of the number of parameters. Both charts compare four different configurations: "Ours", "Ours + GPTQ", "Original", and "Original + GPTQ". The x-axis for both charts represents the number of parameters, with values at 2.7B, 3.7B, 5.5B, and 6.7B. ### Components/Axes **Left Chart (VRAM Usage):** * **Y-axis:** VRAM (GB), ranging from 2 to 12 in increments of 2. * **X-axis:** # Parameters, with values 2.7B, 3.7B, 5.5B, and 6.7B. * **Legend:** Located at the top of the image. * Blue: Ours * Teal: Ours + GPTQ * Black: Original * Gray: Original + GPTQ **Right Chart (Average Accuracy):** * **Y-axis:** Ave Acc (%), ranging from 52.5 to 65.0 in increments of 2.5. * **X-axis:** # Parameters, with values 2.7B, 3.7B, 5.5B, and 6.7B. * **Legend:** (Same as left chart, located at the top of the image). * Blue: Ours * Teal: Ours + GPTQ * Black: Original * Gray: Original + GPTQ ### Detailed Analysis **Left Chart (VRAM Usage):** * **Ours (Blue):** VRAM usage increases with the number of parameters. * 2.7B: ~5.2 GB * 3.7B: ~7.2 GB * 5.5B: ~10.5 GB * 6.7B: Not present * **Ours + GPTQ (Teal):** VRAM usage also increases with the number of parameters, but at a much lower rate than "Ours". * 2.7B: ~2.7 GB * 3.7B: ~3.1 GB * 5.5B: ~4.1 GB * 6.7B: Not present * **Original (Black):** VRAM usage is only present for 6.7B parameters. * 6.7B: ~12.3 GB * **Original + GPTQ (Gray):** VRAM usage is only present for 6.7B parameters. * 6.7B: ~4.9 GB **Right Chart (Average Accuracy):** * **Ours (Blue):** Average accuracy increases with the number of parameters. * 2.7B: ~55.0 % * 3.7B: ~57.3 % * 5.5B: ~61.3 % * 6.7B: Not present * **Ours + GPTQ (Teal):** Average accuracy increases with the number of parameters, and is slightly lower than "Ours". * 2.7B: ~54.7 % * 3.7B: ~56.8 % * 5.5B: ~60.1 % * 6.7B: Not present * **Original (Black):** Average accuracy is only present for 6.7B parameters. * 6.7B: ~65.5 % * **Original + GPTQ (Gray):** Average accuracy is only present for 6.7B parameters. * 6.7B: ~63.7 % ### Key Observations * For both charts, the "Ours" and "Ours + GPTQ" configurations have data for 2.7B, 3.7B, and 5.5B parameters, while "Original" and "Original + GPTQ" only have data for 6.7B parameters. * GPTQ significantly reduces VRAM usage compared to the original configurations. * The "Original" configuration has the highest average accuracy at 6.7B parameters, but also the highest VRAM usage. * Applying GPTQ to "Original" reduces VRAM usage while only slightly decreasing average accuracy. ### Interpretation The data suggests that GPTQ (GPT Quantization) is an effective technique for reducing VRAM usage in models, as evidenced by the lower VRAM consumption of "Ours + GPTQ" compared to "Ours", and "Original + GPTQ" compared to "Original". The trade-off is a slight decrease in average accuracy when using GPTQ. The "Original" model with 6.7B parameters achieves the highest accuracy, but at the cost of significantly higher VRAM usage. This information is valuable for model optimization, allowing users to balance accuracy and memory footprint based on their specific needs and hardware constraints. The absence of data for "Original" and "Original + GPTQ" at lower parameter counts limits a full comparison across all configurations.

## Bar Charts: VRAM Usage and Average Accuracy vs. Model Parameters ### Overview The image presents two side-by-side bar charts comparing the performance of different model configurations. The left chart shows VRAM (Video RAM) usage in Gigabytes (GB), while the right chart displays Average Accuracy in percentage (%). Both charts compare four configurations: "Ours" (blue), "Ours + GPTQ" (light blue), "Original" (black), and "Original + GPTQ" (dark gray), across varying model sizes defined by the number of parameters (2.7B, 3.7B, 5.5B, and 6.7B). ### Components/Axes * **X-axis (Both Charts):** "# Parameters" with markers at 2.7B, 3.7B, 5.5B, and 6.7B. * **Y-axis (Left Chart):** "VRAM (GB)" ranging from approximately 2 GB to 13 GB, with gridlines at 2, 4, 6, 8, 10, and 12. * **Y-axis (Right Chart):** "Ave Acc (%)" ranging from approximately 52% to 65%, with gridlines at 52.5, 55, 57.5, 60, 62.5, and 65. * **Legend (Top-Left):** * Blue: "Ours" * Light Blue: "Ours + GPTQ" * Black: "Original" * Dark Gray: "Original + GPTQ" ### Detailed Analysis or Content Details **Left Chart: VRAM Usage** * **2.7B Parameters:** * Ours (Blue): Approximately 5.1 GB * Ours + GPTQ (Light Blue): Approximately 3.1 GB * Original (Black): Approximately 6.8 GB * Original + GPTQ (Dark Gray): Approximately 3.3 GB * **3.7B Parameters:** * Ours (Blue): Approximately 7.1 GB * Ours + GPTQ (Light Blue): Approximately 3.6 GB * Original (Black): Approximately 8.5 GB * Original + GPTQ (Dark Gray): Approximately 4.1 GB * **5.5B Parameters:** * Ours (Blue): Approximately 10.3 GB * Ours + GPTQ (Light Blue): Approximately 4.1 GB * Original (Black): Approximately 11.5 GB * Original + GPTQ (Dark Gray): Approximately 4.6 GB * **6.7B Parameters:** * Ours (Blue): Approximately 12.5 GB * Ours + GPTQ (Light Blue): Approximately 4.7 GB * Original (Black): Approximately 12.8 GB * Original + GPTQ (Dark Gray): Approximately 5.2 GB **Right Chart: Average Accuracy** * **2.7B Parameters:** * Ours (Blue): Approximately 54.5% * Ours + GPTQ (Light Blue): Approximately 55.2% * Original (Black): Approximately 54.8% * Original + GPTQ (Dark Gray): Approximately 55.5% * **3.7B Parameters:** * Ours (Blue): Approximately 57.2% * Ours + GPTQ (Light Blue): Approximately 57.5% * Original (Black): Approximately 57.0% * Original + GPTQ (Dark Gray): Approximately 58.0% * **5.5B Parameters:** * Ours (Blue): Approximately 61.5% * Ours + GPTQ (Light Blue): Approximately 61.8% * Original (Black): Approximately 60.5% * Original + GPTQ (Dark Gray): Approximately 62.0% * **6.7B Parameters:** * Ours (Blue): Approximately 64.0% * Ours + GPTQ (Light Blue): Approximately 64.5% * Original (Black): Approximately 63.0% * Original + GPTQ (Dark Gray): Approximately 65.0% ### Key Observations * **VRAM Usage:** VRAM usage increases consistently with the number of parameters for all configurations. "Ours" and "Original" consistently require more VRAM than their respective "+ GPTQ" counterparts. * **Accuracy:** Accuracy generally increases with the number of parameters. "+ GPTQ" configurations show a slight accuracy improvement over their base configurations ("Ours" and "Original"). * **GPTQ Impact:** Applying GPTQ significantly reduces VRAM usage across all model sizes, with a relatively small impact on accuracy. * **Comparison of "Ours" vs "Original":** "Original" models generally require slightly more VRAM than "Ours" models for the same number of parameters, but the accuracy is comparable. ### Interpretation The data suggests that GPTQ is an effective quantization technique for reducing the memory footprint of large language models without substantial performance degradation. The consistent reduction in VRAM usage across all model sizes indicates that GPTQ's benefits scale with model complexity. The slight accuracy improvements observed with "+ GPTQ" configurations could be attributed to the quantization process itself or the specific implementation details. The comparison between "Ours" and "Original" models suggests that there are architectural or implementation differences that affect VRAM usage, but not necessarily accuracy. The overall trend of increasing VRAM usage and accuracy with more parameters highlights the trade-off between model size, computational resources, and performance. The charts provide a clear visual representation of this trade-off, allowing for informed decisions about model selection and optimization.

## [Bar Charts]: VRAM Usage and Average Accuracy vs. Model Parameter Count ### Overview The image displays two side-by-side bar charts comparing the performance of different model variants across four model sizes (2.7B, 3.7B, 5.5B, and 6.7B parameters). The left chart measures Video RAM (VRAM) usage in Gigabytes (GB), and the right chart measures Average Accuracy (Ave Acc) as a percentage. The comparison involves four model configurations: "Ours," "Ours + GPTQ," "Original," and "Original + GPTQ." ### Components/Axes * **Legend (Top-Center, spanning both charts):** * **Blue Bar:** "Ours" * **Teal Bar:** "Ours + GPTQ" * **Black Bar:** "Original" * **Gray Bar:** "Original + GPTQ" * **Left Chart - VRAM (GB):** * **Y-axis:** Label: "VRAM (GB)". Scale: 2 to 12, with major ticks every 2 units. * **X-axis:** Label: "# Parameters". Categories: "2.7B", "3.7B", "5.5B", "6.7B". * **Right Chart - Ave Acc (%):** * **Y-axis:** Label: "Ave Acc (%)". Scale: 52.5 to 65.0, with major ticks every 2.5 units. * **X-axis:** Label: "# Parameters". Categories: "2.7B", "3.7B", "5.5B", "6.7B". ### Detailed Analysis **Left Chart: VRAM (GB)** * **Trend Verification:** For the "Ours" (blue) and "Ours + GPTQ" (teal) series, VRAM usage increases with the number of parameters. The "Original" (black) and "Original + GPTQ" (gray) series are only present for the 6.7B parameter model. * **Data Points (Approximate Values):** * **2.7B Parameters:** * Ours (Blue): ~5.2 GB * Ours + GPTQ (Teal): ~2.6 GB * **3.7B Parameters:** * Ours (Blue): ~7.1 GB * Ours + GPTQ (Teal): ~3.1 GB * **5.5B Parameters:** * Ours (Blue): ~10.5 GB * Ours + GPTQ (Teal): ~4.1 GB * **6.7B Parameters:** * Original (Black): ~12.4 GB * Original + GPTQ (Gray): ~4.8 GB **Right Chart: Ave Acc (%)** * **Trend Verification:** For all series, average accuracy generally increases with the number of parameters. The "Ours" and "Ours + GPTQ" bars are very close in height for each parameter size, indicating minimal accuracy difference. * **Data Points (Approximate Values):** * **2.7B Parameters:** * Ours (Blue): ~55.0% * Ours + GPTQ (Teal): ~54.6% * **3.7B Parameters:** * Ours (Blue): ~57.2% * Ours + GPTQ (Teal): ~57.2% * **5.5B Parameters:** * Ours (Blue): ~61.5% * Ours + GPTQ (Teal): ~60.7% * **6.7B Parameters:** * Original (Black): ~66.0% * Original + GPTQ (Gray): ~63.5% ### Key Observations 1. **VRAM Reduction with GPTQ:** Applying GPTQ quantization ("Ours + GPTQ" vs. "Ours", and "Original + GPTQ" vs. "Original") results in a dramatic reduction in VRAM usage across all model sizes. The reduction is most pronounced for the 6.7B model, where VRAM drops from ~12.4 GB to ~4.8 GB. 2. **Accuracy Impact of GPTQ:** The impact of GPTQ on average accuracy is minimal for the "Ours" models (a difference of less than ~1% for 2.7B and 5.5B, and no difference for 3.7B). For the 6.7B "Original" model, GPTQ causes a more noticeable accuracy drop of approximately 2.5 percentage points. 3. **Model Comparison at 6.7B:** At the largest parameter size (6.7B), the "Original" model achieves the highest accuracy (~66.0%) but also requires the most VRAM (~12.4 GB). The "Original + GPTQ" variant offers a significant memory saving (~4.8 GB) with a moderate accuracy trade-off (~63.5%). 4. **Scaling Trend:** Both VRAM usage and average accuracy scale positively with the number of parameters for all model configurations. ### Interpretation This data demonstrates the trade-offs between model size, memory efficiency, and performance. The primary finding is the effectiveness of GPTQ quantization in drastically reducing VRAM requirements (by more than 50% in most cases) with a relatively small cost to accuracy, especially for the "Ours" model architecture. This suggests that the "Ours" method may be more robust to quantization than the "Original" method at the 6.7B scale. The charts are likely from a research paper or technical report aiming to showcase a new model architecture ("Ours") and its compatibility with post-training quantization techniques like GPTQ. The key message is that one can deploy larger, more accurate models (like the 6.7B variant) within practical memory constraints by applying GPTQ, making advanced AI models more accessible for deployment on consumer or edge hardware. The absence of "Original" data for smaller models implies the study's focus is on comparing the two architectures at the largest scale or that "Ours" is a new method being proposed for these model sizes.

## Bar Charts: VRAM Usage and Accuracy Comparison Across Model Sizes ### Overview The image contains two side-by-side bar charts comparing model performance metrics (VRAM usage and accuracy) across four parameter sizes (2.7B, 3.7B, 5.5B, 6.7B). Each chart uses four color-coded categories: "Ours" (blue), "Ours + GPTQ" (teal), "Original" (black), and "Original + GPTQ" (gray). The charts demonstrate trade-offs between resource efficiency and performance. ### Components/Axes **Left Chart (VRAM Usage in GB):** - **X-axis**: Model parameter sizes (2.7B, 3.7B, 5.5B, 6.7B) - **Y-axis**: VRAM usage (GB), ranging from 2 to 12 - **Legend**: - Blue = "Ours" - Teal = "Ours + GPTQ" - Black = "Original" - Gray = "Original + GPTQ" - **Legend Position**: Top of chart **Right Chart (Average Accuracy %):** - **X-axis**: Same parameter sizes as left chart - **Y-axis**: Accuracy (%), ranging from 52.5 to 65 - **Legend**: Same color coding as left chart - **Legend Position**: Top of chart ### Detailed Analysis **VRAM Usage (Left Chart):** - **2.7B Parameters**: - Blue ("Ours"): ~5.2GB - Teal ("Ours + GPTQ"): ~2.5GB - Black ("Original"): ~10.5GB - Gray ("Original + GPTQ"): ~4.7GB - **3.7B Parameters**: - Blue: ~7.2GB - Teal: ~3.1GB - Black: ~11.8GB - Gray: ~4.9GB - **5.5B Parameters**: - Blue: ~10.6GB - Teal: ~4.1GB - Black: ~12.4GB - Gray: ~5.1GB - **6.7B Parameters**: - Blue: ~12.4GB - Teal: ~4.8GB - Black: ~12.4GB - Gray: ~5.1GB **Accuracy (Right Chart):** - **2.7B Parameters**: - Blue: ~55.1% - Teal: ~54.7% - Black: ~57.8% - Gray: ~56.9% - **3.7B Parameters**: - Blue: ~57.3% - Teal: ~57.2% - Black: ~59.5% - Gray: ~57.4% - **5.5B Parameters**: - Blue: ~63.2% - Teal: ~61.3% - Black: ~66.1% - Gray: ~63.5% - **6.7B Parameters**: - Blue: ~63.2% - Teal: ~61.3% - Black: ~66.1% - Gray: ~63.5% ### Key Observations 1. **VRAM Trends**: - "Ours" (blue) shows linear VRAM growth with parameter size (5.2GB → 12.4GB). - "Original" (black) maintains high VRAM (~10.5GB–12.4GB) across all sizes. - "Ours + GPTQ" (teal) consistently uses the least VRAM (2.5GB–4.8GB). - "Original + GPTQ" (gray) reduces VRAM by ~50% compared to "Original" but remains higher than "Ours + GPTQ". 2. **Accuracy Trends**: - "Ours" (blue) improves accuracy from 55.1% to 63.2% as parameters increase. - "Original" (black) achieves the highest accuracy (66.1% at 5.5B/6.7B). - "Ours + GPTQ" (teal) shows minimal accuracy gains (54.7% → 61.3%). - "Original + GPTQ" (gray) maintains ~63.5% accuracy at larger sizes. ### Interpretation The data reveals a trade-off between resource efficiency and performance: - **"Ours" models** prioritize accuracy growth with parameter size but require significant VRAM. - **GPTQ quantization** reduces VRAM usage by ~50% for both "Ours" and "Original" models but sacrifices ~2–3% accuracy. - The "Original" model achieves the highest accuracy but at the cost of high VRAM consumption. - At 6.7B parameters, "Ours" matches "Original" in VRAM (12.4GB) but lags in accuracy (63.2% vs. 66.1%). This suggests that "Ours" offers a scalable architecture for accuracy-focused applications, while GPTQ provides a lightweight alternative for resource-constrained environments. The "Original" model remains optimal for accuracy-critical tasks despite its resource demands.