## Line Chart: Accuracy Comparison of Four AI Models Across Chinese Math Problem Categories ### Overview This image is a line chart comparing the performance (accuracy) of four different large language models (LLMs) on a wide variety of Chinese-language mathematics problem categories. The chart displays the accuracy percentage for each model across approximately 50 distinct problem types, revealing significant variability in performance both between models and across different mathematical domains. ### Components/Axes * **Chart Type:** Multi-series line chart with markers. * **Y-Axis:** * **Label:** "Accuracy" (written vertically on the left side). * **Scale:** Linear scale from 0 to 100. * **Major Ticks/Gridlines:** At 0, 20, 40, 60, 80, 100. Horizontal dashed gridlines extend from these ticks across the chart. * **X-Axis:** * **Label:** None explicitly stated. The axis represents discrete categories of math problems. * **Tick Labels:** A long series of Chinese text labels, each representing a specific math problem category. They are rotated approximately 45 degrees for readability. * **Legend:** * **Position:** Centered at the top of the chart, inside the plot area. * **Content:** Four entries, each with a colored line segment and marker: * **Blue line with circle marker:** `Baichuan2-13B` * **Orange line with circle marker:** `LLaMA2-13B` * **Green line with circle marker:** `Qwen-14B` * **Red line with circle marker:** `InternLM2-Math-20B` ### Detailed Analysis **X-Axis Categories (Translated from Chinese):** The categories, from left to right, are: 1. 三角形 (Triangle) 2. 圆 (Circle) 3. 平行四边形 (Parallelogram) 4. 梯形 (Trapezoid) 5. 平面图形综合 (Plane Figure Synthesis) 6. 长方体 (Cuboid) 7. 圆柱 (Cylinder) 8. 圆锥 (Cone) 9. 立体图形综合 (Solid Figure Synthesis) 10. 和差问题 (Sum and Difference Problem) 11. 提问问题 (Question Problem - *likely a specific problem type*) 12. 归一问题 (Unitary Method Problem) 13. 和倍问题 (Sum and Multiple Problem) 14. 差倍问题 (Difference and Multiple Problem) 15. 对称问题 (Symmetry Problem) 16. 工程问题 (Work Problem) 17. 年龄问题 (Age Problem) 18. 扩倍问题 (Expansion and Multiple Problem) 19. 积木问题 (Block Problem - *likely spatial reasoning*) 20. 交通问题 (Traffic Problem) 21. 鸡兔同笼 (Chicken and Rabbit in the Same Cage) 22. 相遇问题 (Meeting Problem) 23. 行程问题 (Travel Problem) 24. 人民币问题 (RMB/Currency Problem) 25. 计数问题 (Counting Problem) 26. 浓度问题 (Concentration Problem) 27. 盈亏问题 (Surplus and Deficit Problem) 28. 面积问题 (Area Problem) 29. 统计图表 (Statistical Charts) 30. 指数律 (Exponent Laws) 31. 分数与小数 (Fractions and Decimals) 32. 分数应用题 (Fraction Word Problems) 33. 公因数与公倍数 (Common Factors and Multiples) 34. 因数与倍数综合 (Factors and Multiples Synthesis) 35. 比和比例综合 (Ratio and Proportion Synthesis) 36. 案例问题 (Case Problem) 37. 定义新运算 (Define New Operations) 38. 方程与方程组 (Equations and Systems of Equations) 39. 除法与减法 (Division and Subtraction) 40. 倍数问题 (Multiple Problem) 41. 移动问题 (Movement Problem) 42. 百分率问题 (Percentage Problem) **Model Performance Trends (Visual Verification):** * **InternLM2-Math-20B (Red Line):** This line is frequently the highest on the chart, showing a generally upward trend with high volatility. It peaks at or near 100% accuracy for "Percentage Problem" (far right) and shows very high accuracy (>90%) for categories like "Sum and Difference Problem", "Unitary Method Problem", and "Define New Operations". Its lowest points are around 20-40% for categories like "Statistical Charts" and "Concentration Problem". * **Baichuan2-13B (Blue Line):** This line is highly volatile, often competing with the red line for the top position but also dropping significantly. It shows strong performance (>80%) in "Triangle", "Circle", "Sum and Difference Problem", and "Define New Operations". It has notable dips below 40% in areas like "立体图形综合 (Solid Figure Synthesis)" and "统计图表 (Statistical Charts)". * **Qwen-14B (Green Line):** This line generally occupies the middle-to-lower range of accuracy. It has a significant peak above 90% for "鸡兔同笼 (Chicken and Rabbit in the Same Cage)" but otherwise mostly stays between 20% and 60%. It shows a notable dip to 0% for "盈亏问题 (Surplus and Deficit Problem)". * **LLaMA2-13B (Orange Line):** This line is consistently the lowest-performing model across almost all categories. Its accuracy rarely exceeds 40%, with many points at or near 0%. Its highest points are around 60-65% for "分数与小数 (Fractions and Decimals)" and "百分率问题 (Percentage Problem)". ### Key Observations 1. **Performance Hierarchy:** There is a clear, though not absolute, hierarchy: InternLM2-Math-20B ≥ Baichuan2-13B > Qwen-14B > LLaMA2-13B. 2. **Domain Specificity:** All models show extreme variability. No model is uniformly good or bad. Performance is highly dependent on the specific math domain. For example, Qwen-14B excels at "鸡兔同笼" but fails at "盈亏问题". 3. **Common Struggles:** The category "统计图表 (Statistical Charts)" appears to be challenging for all models, with accuracies clustered between ~20% and ~50%. 4. **Model Strengths:** * **InternLM2-Math-20B:** Shows particular strength in algebraic and arithmetic word problems (e.g., Sum/Difference, Unitary Method, Define New Operations). * **Baichuan2-13B:** Shows strength in geometry (Triangle, Circle) and some word problems. * **Qwen-14B:** Has a standout performance on the classic "鸡兔同笼" problem. * **LLaMA2-13B:** Shows relative strength in foundational arithmetic (Fractions/Decimals, Percentages) compared to its own performance on other topics. 5. **Volatility:** The blue (Baichuan) and red (InternLM) lines are the most volatile, indicating their performance is the most sensitive to the problem type. ### Interpretation This chart provides a granular benchmark of LLM capabilities in mathematical reasoning within the Chinese language context. The data suggests that: 1. **Specialization Over Generalization:** The models, especially the top performers, are not general-purpose math solvers. Their capabilities are highly specialized. The InternLM2-Math-20B model, likely fine-tuned for mathematics, demonstrates the benefit of domain-specific training, but even it has clear weaknesses. 2. **The "Chinese Math Problem" Spectrum:** The x-axis represents a comprehensive curriculum of Chinese elementary and middle school math. The chart effectively maps which parts of this curriculum are more or less accessible to current LLMs. Foundational arithmetic and classic puzzle types (鸡兔同笼) are more accessible than applied topics like statistics or complex concentration problems. 3. **Model Architecture and Training Data Implications:** The stark difference between LLaMA2-13B (a general English-centric model) and the others (likely with more Chinese and/or math-specific data) highlights the critical role of pre-training data composition and potential fine-tuning for achieving proficiency in specific domains and languages. 4. **A Diagnostic Tool:** For a researcher, this chart is a diagnostic map. It doesn't just say "Model X is better." It shows *where* and *by how much* it is better, and more importantly, *where it fails*. This is crucial for guiding future model development, indicating which mathematical reasoning skills (e.g., handling statistical data, understanding concentration) require more focused training or architectural innovation. **In summary, the image is a dense, information-rich performance matrix. It moves beyond aggregate scores to reveal the nuanced, domain-specific landscape of AI mathematical reasoning in Chinese, highlighting both significant progress and persistent challenges.**