## Line Graphs: Model Performance vs. Token Count ### Overview The image contains three line graphs comparing the performance of different AI models across tasks as token count increases. Each graph tracks "Score" on the y-axis against "Token (B)" on the x-axis, with distinct data series represented by colored lines and legends. --- ### Components/Axes 1. **Graph 1: "average"** - **X-axis**: Token (B) (0–2000) - **Y-axis**: Score (0–40) - **Legend**: - Red: Ling-coder-1B - Blue dashed: Qwen2.5-Coder-1.5B-Base - Green dashed: Qwen3-1.7B-Base 2. **Graph 2: "average"** - **X-axis**: Token (B) (2047–3547) - **Y-axis**: Score (30–47.5) - **Legend**: - Red: Ling-math-1B - Blue dashed: Qwen2.5-Math-1.5B - Green dashed: Qwen3-1.7B 3. **Graph 3: "average"** - **X-axis**: Token (B) (0–300) - **Y-axis**: Score (2–18) - **Legend**: - Blue: Finemath-3plus - Orange: Nemotron-3plus - Green: Inf-mm-webmath - Red: Megamath-web - Purple: Ling-math-web --- ### Detailed Analysis #### Graph 1: Ling-coder-1B vs. Qwen Models - **Ling-coder-1B (Red)**: Starts at ~5, rises sharply to ~40 by 2000 tokens, with fluctuations. - **Qwen2.5-Coder-1.5B-Base (Blue dashed)**: Flat line at ~35. - **Qwen3-1.7B-Base (Green dashed)**: Flat line at ~35. #### Graph 2: Ling-math-1B vs. Qwen Math Models - **Ling-math-1B (Red)**: Starts at ~30, rises to ~47.5 by 3547 tokens, with volatility. - **Qwen2.5-Math-1.5B (Blue dashed)**: Flat line at ~42.5. - **Qwen3-1.7B (Green dashed)**: Flat line at ~42.5. #### Graph 3: Math/Web Math Models - **Finemath-3plus (Blue)**: Starts at ~4, peaks at ~11 by 300 tokens. - **Nemotron-3plus (Orange)**: Starts at ~3, peaks at ~9 by 300 tokens. - **Inf-mm-webmath (Green)**: Starts at ~2, peaks at ~8 by 300 tokens. - **Megamath-web (Red)**: Starts at ~2, peaks at ~5 by 300 tokens. - **Ling-math-web (Purple)**: Starts at ~5, peaks at ~17 by 300 tokens. --- ### Key Observations 1. **Ling Models Dominance**: - Ling-coder-1B and Ling-math-1B consistently outperform Qwen models in their respective graphs, suggesting superior scalability or optimization. - Ling-math-web (purple) in Graph 3 shows the steepest growth, surpassing all other models. 2. **Qwen Model Stagnation**: - Qwen2.5 and Qwen3 models in Graphs 1–2 remain flat, indicating no improvement with increased tokens (possibly base models without fine-tuning). 3. **Task-Specific Performance**: - Graph 3 reveals divergent trends: Finemath-3plus and Nemotron-3plus perform better in math tasks, while Ling-math-web excels in web math. --- ### Interpretation - **Scalability Insights**: Ling models demonstrate strong token efficiency, likely due to specialized training (e.g., code/math tasks). Qwen models’ flat lines suggest they are static base models. - **Task Optimization**: Ling-math-web’s rapid improvement in Graph 3 highlights its effectiveness in web math, outperforming generalist models like Finemath-3plus. - **Anomalies**: The abrupt rise in Ling-coder-1B (Graph 1) and Ling-math-1B (Graph 2) may indicate task-specific breakthroughs or dataset biases. This analysis underscores the importance of model architecture and training focus in achieving token-efficient performance across domains.