## Line Graph: Accuracy Comparison of Math Models on Various Problems ### Overview The image is a multi-line graph comparing the accuracy performance of four large language models (LLMs) across 25 math problem categories. The models compared are Baichuan2-13B (blue), LLaMA2-13B (orange), Qwen-14B (green), and InternLM2-Math-20B (red). Accuracy percentages range from 0-100% on the y-axis, with math topics listed sequentially on the x-axis. ### Components/Axes - **Legend**: Top-left corner, mapping colors to models: - Blue: Baichuan2-13B - Orange: LLaMA2-13B - Green: Qwen-14B - Red: InternLM2-Math-20B - **X-axis**: "Math Problems" with 25 labeled categories (e.g., Angles, Area, Circles, Classifying & sorting, Coin names & value, Coordinate plane, Cubes, Decimals, Estimation & rounding, Fractions, Light & heavy, Mixed operations, Multiple operations, Numerical expressions, Patterns, Perimeter, Place value, Powers, Rational number, Spheres, Subtraction, Time, Triangles, Variable expressions, Volume of 3d shapes, Add, Compare, Count, Division, Equations, Length, Statistics, Percent, Polygons, Probability, Proportional, Quadrilaterals, Ratio, Temperature, Volume). - **Y-axis**: "Accuracy (%)" with ticks at 0, 20, 40, 60, 80, 100. - **Lines**: Four colored lines representing model performance across topics. ### Detailed Analysis 1. **InternLM2-Math-20B (Red Line)**: - Consistently highest performer, peaking above 90% in multiple categories (e.g., Coordinate plane, Fractions, Time). - Shows minor dips but maintains >70% accuracy in all categories. - Notable peaks: 95%+ in Coordinate plane, Fractions, and Time. 2. **Baichuan2-13B (Blue Line)**: - Second-highest performer overall, with peaks near 90% (e.g., Coordinate plane, Fractions). - More volatile than InternLM2, with sharper drops (e.g., 60% in Decimals, 50% in Estimation & rounding). - Strong in geometry topics (Angles, Area, Perimeter). 3. **LLaMA2-13B (Orange Line)**: - Most variable performance, with extreme lows (e.g., 0% in Coordinate plane, 5% in Subtraction). - Strong in algebraic topics (Equations, Variables) with peaks near 80%. - Weaknesses in spatial reasoning (Coordinate plane, Geometry). 4. **Qwen-14B (Green Line)**: - Moderate performance, averaging 60-70%. - Peaks in algebraic topics (Equations, Variables) at ~75%. - Notable dip to 15% in Place value, recovery in Statistics (~60%). ### Key Observations - **Outliers**: - LLaMA2-13B: 0% accuracy in Coordinate plane (potential data error or model weakness). - Qwen-14B: 15% in Place value (significant drop). - **Trends**: - InternLM2-Math-20B dominates in geometry and arithmetic (Angles, Fractions, Time). - All models struggle with spatial reasoning (Coordinate plane, Geometry). - Algebraic topics (Equations, Variables) show higher performance across models. ### Interpretation The data suggests **InternLM2-Math-20B** is the most robust model for math problem-solving, likely due to specialized training on mathematical reasoning. Its consistent performance across diverse topics indicates strong generalization. **LLaMA2-13B** exhibits the most variability, with critical failures in spatial reasoning (Coordinate plane) but strengths in algebraic manipulation. **Qwen-14B** and **Baichuan2-13B** show mid-tier performance, with Qwen excelling in algebraic topics and Baichuan2 performing well in geometry. The anomalies (e.g., LLaMA2's 0% in Coordinate plane) highlight potential gaps in model training data or architectural limitations for specific problem types. This comparison underscores the importance of model specialization for domain-specific tasks like mathematics.