## Chart Type: Multiple Line Graphs ### Overview The image presents three line graphs comparing the performance of different sized language models across various metrics as they are trained on increasing numbers of tokens. The graphs are titled "(a) Brain Alignment", "(b) Formal Competence", and "(c) Functional Competence". Each graph plots the metric on the y-axis against the number of tokens on the x-axis. The models are differentiated by line color and style, with a legend provided in each subplot. A vertical line indicates a point representing 94.4% of training time. ### Components/Axes **General Components:** * **Titles:** (a) Brain Alignment, (b) Formal Competence, (c) Functional Competence * **Legends:** Located in the top-left corner of each subplot, indicating model size (410M, 1B, 1.4B, 2.8B, 6.9B) with corresponding line styles and colors. * **Vertical Line:** A vertical black line is present in each graph, labeled "94.4% of training time" in the Brain Alignment graph. * **R-squared values:** R^2 = 0.65 is present to the left of the Brain Alignment graph, and R^2 = 0.36 is present to the right. **Graph (a) Brain Alignment:** * **Y-axis:** "Brain Alignment", scale from 0.2 to 0.6, with ticks at 0.2, 0.3, 0.4, 0.5, and 0.6. * **X-axis:** "Number of Tokens", with values ranging from 0 to 286B (billions). Specific values marked are 0, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 512M, 1B, 2B, 4B, 8B, 16B, 20B, 32B, 40B, 60B, 80B, 100B, 120B, 140B, 160B, 180B, 200B, 220B, 240B, 260B, 280B, 286B. * **Data Series:** * **410M (light green):** Starts at approximately 0.25, increases to around 0.5 at 16B tokens, then fluctuates between 0.5 and 0.6. * **1B (green):** Starts at approximately 0.25, increases to around 0.5 at 16B tokens, then fluctuates between 0.45 and 0.55. * **1.4B (green-grey):** Starts at approximately 0.25, increases to around 0.45 at 16B tokens, then fluctuates between 0.4 and 0.5. * **2.8B (dark green):** Starts at approximately 0.25, increases to around 0.45 at 16B tokens, then fluctuates between 0.4 and 0.5. * **6.9B (darkest green):** Starts at approximately 0.35, increases to around 0.45 at 16B tokens, then fluctuates between 0.4 and 0.5. **Graph (b) Formal Competence:** * **Y-axis:** "Formal Competence", scale from 0.1 to 0.7, with ticks at 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, and 0.7. * **X-axis:** "Number of Tokens", with values ranging from 0 to 286B (billions). Specific values marked are 0, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 512MB, 1B, 2B, 4B, 8B, 16B, 20B, 32B, 40B, 60B, 80B, 100B, 120B, 140B, 160B, 180B, 200B, 220B, 240B, 260B, 280B, 286B. * **Data Series:** * **410M (dark blue):** Starts at approximately 0.15, increases sharply to around 0.65 at 16B tokens, then plateaus around 0.7. * **1B (blue-grey):** Starts at approximately 0.15, increases sharply to around 0.65 at 16B tokens, then plateaus around 0.7. * **1.4B (green-grey):** Starts at approximately 0.15, increases sharply to around 0.65 at 16B tokens, then plateaus around 0.7. * **2.8B (light green):** Starts at approximately 0.15, increases sharply to around 0.65 at 16B tokens, then plateaus around 0.7. * **6.9B (yellow-green):** Starts at approximately 0.2, increases sharply to around 0.65 at 16B tokens, then plateaus around 0.7. **Graph (c) Functional Competence:** * **Y-axis:** "Functional Competence", scale from 0.00 to 0.30, with ticks at 0.00, 0.05, 0.10, 0.15, 0.20, 0.25, and 0.30. * **X-axis:** "Number of Tokens", with values ranging from 0 to 286B (billions). Specific values marked are 0, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 5120, 1B, 2B, 4B, 8B, 16B, 20B, 32B, 40B, 60B, 80B, 100B, 120B, 140B, 160B, 180B, 200B, 220B, 240B, 260B, 280B, 286B. * **Data Series:** * **410M (lightest blue):** Starts at approximately 0.01, increases to around 0.15 at 16B tokens, then plateaus around 0.17. * **1B (light blue):** Starts at approximately 0.01, increases to around 0.20 at 16B tokens, then plateaus around 0.22. * **1.4B (blue):** Starts at approximately 0.01, increases to around 0.23 at 16B tokens, then plateaus around 0.25. * **2.8B (dark blue):** Starts at approximately 0.01, increases to around 0.27 at 16B tokens, then plateaus around 0.28. * **6.9B (darkest blue):** Starts at approximately 0.01, increases to around 0.30 at 16B tokens, then plateaus around 0.31. ### Key Observations * **Brain Alignment:** The brain alignment metric shows an initial increase for all model sizes, followed by fluctuations. The R^2 value of 0.65 suggests a moderate correlation. * **Formal Competence:** All models exhibit a sharp increase in formal competence around 16B tokens, after which they plateau. The R^2 value is not provided for this graph. * **Functional Competence:** Functional competence also increases sharply around 16B tokens, with larger models achieving higher levels of competence. The R^2 value of 0.36 suggests a weak correlation. * **Model Size Impact:** Larger models generally achieve higher levels of formal and functional competence. The impact of model size on brain alignment is less clear. * **Training Time Threshold:** The vertical line at 94.4% of training time appears to coincide with the point where the models begin to plateau in formal and functional competence. ### Interpretation The data suggests that increasing the number of training tokens significantly improves the formal and functional competence of language models, particularly up to a certain point (around 16B tokens or 94.4% of training time). After this point, the gains diminish, and the models plateau. Larger models tend to perform better in terms of formal and functional competence, indicating that model size is also a crucial factor. The brain alignment metric shows a less clear relationship with model size and training tokens, suggesting that it may be influenced by other factors or require a different analysis approach. The R^2 values indicate the strength of the relationship between the number of tokens and the metrics, with higher values indicating a stronger correlation.

## Charts: Model Performance Metrics ### Overview The image presents three scatter plots (a, b, and c) illustrating the performance of different language models (410M, 1B, 1.4B, 2.8B, and 6.9B parameters) across three metrics: Brain Alignment, Formal Competence, and Functional Competence. The x-axis of each plot represents the number of tokens processed during training, and the y-axis represents the corresponding metric value. Each plot includes an R-squared value and a vertical dashed line indicating 94.4% of the training time. ### Components/Axes * **X-axis (all plots):** Number of Tokens. Scale: 2M to 300B, with markers at 2M, 10M, 50M, 100M, 200M, 500M, 1B, 2B, 5B, 10B, 20B, 50B, 100B, 200B, 300B. * **Y-axis (a):** Brain Alignment. Scale: 0.2 to 0.6, with markers at 0.2, 0.3, 0.4, 0.5, 0.6. * **Y-axis (b):** Formal Competence. Scale: 0.1 to 0.7, with markers at 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7. * **Y-axis (c):** Functional Competence. Scale: 0.0 to 0.3, with markers at 0.0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3. * **Legend (all plots):** * 410M (Light Green) * 1B (Dark Green) * 1.4B (Teal) * 2.8B (Blue) * 6.9B (Purple) * **R-squared values:** R² = 0.65 (left side of plot a), R² = 0.36 (right side of plot a). * **Training Time Marker:** Vertical dashed line at approximately 20B tokens, labeled "94.4% of training time". ### Detailed Analysis or Content Details **Plot (a): Brain Alignment** * **410M:** Starts at approximately 0.22, remains relatively flat around 0.25 until approximately 50B tokens, then increases slightly to around 0.28. * **1B:** Starts at approximately 0.23, increases rapidly to around 0.55 by 50B tokens, then plateaus around 0.58. * **1.4B:** Starts at approximately 0.24, increases rapidly to around 0.60 by 50B tokens, then plateaus around 0.62. * **2.8B:** Starts at approximately 0.25, increases rapidly to around 0.58 by 50B tokens, then plateaus around 0.60. * **6.9B:** Starts at approximately 0.26, increases rapidly to around 0.55 by 50B tokens, then plateaus around 0.58. * The R-squared value of 0.65 applies to the initial portion of the curves (before the plateau), while 0.36 applies to the plateau region. **Plot (b): Formal Competence** * **410M:** Remains consistently low around 0.10 across all token counts. * **1B:** Remains consistently low around 0.10 across all token counts. * **1.4B:** Remains consistently low around 0.10 across all token counts. * **2.8B:** Increases rapidly from approximately 0.10 to around 0.70 by 50B tokens, then plateaus. * **6.9B:** Increases rapidly from approximately 0.10 to around 0.70 by 50B tokens, then plateaus. **Plot (c): Functional Competence** * **410M:** Remains consistently low around 0.01 across all token counts. * **1B:** Remains consistently low around 0.01 across all token counts. * **1.4B:** Remains consistently low around 0.01 across all token counts. * **2.8B:** Increases gradually from approximately 0.01 to around 0.15 by 200B tokens, then increases more rapidly to around 0.25 by 300B tokens. * **6.9B:** Increases gradually from approximately 0.01 to around 0.10 by 200B tokens, then increases more rapidly to around 0.30 by 300B tokens. ### Key Observations * Brain Alignment plateaus for all models after approximately 50B tokens. * Formal Competence shows a clear divergence between smaller models (410M, 1B, 1.4B) and larger models (2.8B, 6.9B). * Functional Competence exhibits a more gradual increase compared to Brain Alignment and Formal Competence, and continues to improve even after 200B tokens. * The R-squared values suggest a stronger correlation between the number of tokens and Brain Alignment in the initial training phase. * The 94.4% training time marker highlights the point at which the models have processed the majority of their training data. ### Interpretation The data suggests that increasing model size significantly improves Brain Alignment and Formal Competence, but the gains diminish after a certain point (around 50B tokens). Functional Competence, however, continues to improve with more training data, even for the larger models. The plateau in Brain Alignment could indicate that the models are reaching a limit in their ability to align with human brain activity, or that the metric itself is not sensitive enough to capture further improvements. The divergence in Formal Competence highlights the importance of model size for achieving strong performance on formal tasks. The continued improvement in Functional Competence suggests that more training data is beneficial for developing practical skills. The R-squared values provide a quantitative measure of the strength of the relationship between training data and performance, and the difference between the two values in plot (a) suggests a change in the underlying dynamics of the training process. The vertical line at 94.4% of training time is a useful reference point for understanding the performance of the models at different stages of training.

## [Multi-Panel Chart]: Scaling Laws for Model Competence and Brain Alignment ### Overview The image contains three line charts arranged in a triangular layout, labeled (a), (b), and (c). They collectively illustrate how different performance metrics of language models evolve as a function of training data (Number of Tokens) and model size. The charts share a common x-axis and a consistent vertical reference line at 16B tokens. The overall theme is the relationship between scale (model parameters and training data) and various forms of model capability. ### Components/Axes **Common Elements:** * **X-axis (All Charts):** "Number of Tokens". The scale is logarithmic, with major tick marks at: 0, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 512M, 1B, 2B, 4B, 8B, 16B, 20B, 32B, 40B, 60B, 80B, 100B, 120B, 140B, 160B, 180B, 200B, 220B, 240B, 260B, 280B, 286B. * **Vertical Reference Line:** A solid black vertical line is drawn at the 16B token mark in all three charts. * **Model Size Legend:** Each chart has a legend titled "Model Size" with five entries, each associated with a specific color and line/marker style. The model sizes are: 410M, 1B, 1.4B, 2.8B, 6.9B. **Chart-Specific Elements:** * **(a) Brain Alignment (Top Center):** * **Y-axis:** "Brain Alignment". Linear scale from 0.2 to 0.6. * **Legend Colors:** Shades of green. 410M (lightest green, circle marker), 1B (light green, circle), 1.4B (medium green, circle), 2.8B (dark green, circle), 6.9B (darkest green, circle). * **Annotations:** * Left side: "R² = 0.65" with an arrow pointing to the left portion of the chart (before 16B). * Right side: "R² = 0.36" with an arrow pointing to the right portion of the chart (after 16B). * Above the right portion: A bracket labeled "94.4% of training time". * **(b) Formal Competence (Bottom Left):** * **Y-axis:** "Formal Competence". Linear scale from 0.1 to 0.7. * **Legend Colors:** A gradient from dark purple/blue to yellow-green. 410M (dark purple, circle), 1B (blue, circle), 1.4B (teal, circle), 2.8B (green, circle), 6.9B (yellow-green, circle). * **(c) Functional Competence (Bottom Right):** * **Y-axis:** "Functional Competence". Linear scale from 0.00 to 0.30. * **Legend Colors:** Shades of blue. 410M (lightest blue, circle), 1B (light blue, circle), 1.4B (medium blue, circle), 2.8B (dark blue, circle), 6.9B (darkest blue, circle). ### Detailed Analysis **Chart (a) Brain Alignment:** * **Trend Verification:** All five model size series show a similar pattern: a relatively flat or slightly noisy phase from 0 to ~512M tokens, followed by a steep, roughly linear increase (on this log-scale x-axis) from ~512M to 16B tokens. After the 16B token vertical line, the growth rate slows dramatically, and the lines plateau with significant noise/fluctuation. * **Data Points (Approximate):** * **Pre-16B (Steep Growth Phase):** At 512M tokens, values range from ~0.25 (6.9B model) to ~0.35 (410M model). At 16B tokens, values converge to a range of approximately 0.50 to 0.58. * **Post-16B (Plateau Phase):** Values fluctuate between ~0.45 and ~0.62. The 410M model (lightest green) often shows the highest values in this region, while the 6.9B model (darkest green) is often among the lowest. * **R² Annotation:** The coefficient of determination (R²) is 0.65 for the relationship between model size and brain alignment during the steep growth phase (left of 16B). It drops to 0.36 for the plateau phase (right of 16B), indicating model size is a weaker predictor of brain alignment after the 16B token mark. **Chart (b) Formal Competence:** * **Trend Verification:** All series show a very low, flat baseline (<0.2) from 0 to ~512M tokens. There is an extremely sharp, near-vertical increase between ~512M and 4B tokens. After ~4B tokens, all series reach a high plateau (between 0.65 and 0.75) and remain essentially flat with minimal growth up to 286B tokens. * **Data Points (Approximate):** * **Baseline (0-512M):** Values cluster between 0.10 and 0.20. * **Sharp Rise (512M-4B):** Values jump from ~0.2 to over 0.6. * **Plateau (4B-286B):** All model sizes converge into a tight band between approximately 0.68 and 0.75. There is no clear ordering by model size in the plateau; the lines are intertwined. **Chart (c) Functional Competence:** * **Trend Verification:** All series start near zero. There is a gradual, accelerating increase beginning around 512M tokens. The growth continues steadily past the 16B token line, showing no clear plateau within the plotted range. Larger models consistently achieve higher functional competence at any given token count after the initial rise. * **Data Points (Approximate):** * **Initial Rise (512M-16B):** At 16B tokens, values range from ~0.08 (410M) to ~0.18 (6.9B). * **Continued Growth (16B-286B):** At the final point (286B tokens), values range from ~0.15 (410M) to ~0.30 (6.9B). The separation between model sizes is clear and maintained. ### Key Observations 1. **Phase Transition at 16B Tokens:** The vertical line at 16B tokens marks a critical point. Brain Alignment growth saturates here, while Functional Competence continues to grow. Formal Competence saturates much earlier (~4B tokens). 2. **Divergence of Metrics:** The three metrics behave fundamentally differently with scale. Brain Alignment and Formal Competence show saturation, while Functional Competence does not saturate within the observed data range. 3. **Model Size Effect:** The benefit of increased model size is most pronounced and consistent for Functional Competence. For Brain Alignment, larger models are not necessarily better after the 16B token point. For Formal Competence, model size makes little difference once the sharp rise is complete. 4. **Noise in Brain Alignment:** The post-16B region of the Brain Alignment chart shows high variance and noise compared to the smooth curves of the other two metrics. ### Interpretation This data suggests a nuanced view of scaling language models. The findings can be interpreted through a Peircean lens of signs: * **Formal Competence (Chart b)** appears to be a **symbol**—a learned, conventional capability (like grammatical correctness) that is acquired rapidly once a sufficient data threshold (~512M tokens) is crossed and then mastered, showing little further improvement with massive scale. * **Brain Alignment (Chart a)** may function as an **index**—a sign that points to a causal relationship between model internal representations and human brain activity. The strong initial correlation (R²=0.65) suggests training data causally drives this alignment. The saturation and noise after 16B tokens imply this causal link weakens or becomes obscured by other factors at extreme scale, making model size a poor predictor (R²=0.36). * **Functional Competence (Chart c)** behaves as an **icon**—it resembles or continuously maps onto real-world utility or problem-solving ability. Its steady, unsaturated growth with both data and model size suggests it is an open-ended capability that benefits from continued scaling, making it the most promising metric for predicting future performance gains. **Notable Anomaly:** The 410M model often achieves the highest *Brain Alignment* scores in the plateau phase, which is counterintuitive. This could indicate that smaller models, perhaps due to simpler internal representations, develop patterns that coincidentally align better with certain measured brain signals after extensive training, even if they are less functionally competent. This highlights a potential decoupling between brain-alignment metrics and practical utility at the extremes of scale.

## Multi-Metric Model Performance Analysis ### Overview The image presents three comparative graphs analyzing model performance across different sizes (410M to 6.9B parameters) against three metrics: Brain Alignment, Formal Competence, and Functional Competence. Each graph tracks performance against token count (0-286B) with distinct color-coded model size indicators. ### Components/Axes 1. **Brain Alignment Graph** - X-axis: Number of Tokens (0-286B) - Y-axis: Brain Alignment (0-0.6 scale) - Legend: Model sizes (410M, 1B, 1.4B, 2.8B, 6.9B) with color gradients - R² values: 0.65 (left) and 0.36 (right) - Notable annotation: "94.4% of training time" at 100B tokens 2. **Formal Competence Graph** - X-axis: Number of Tokens (0-286B) - Y-axis: Formal Competence (0-0.7 scale) - Legend: Model sizes with line style variations - Key threshold: Vertical line at 100B tokens 3. **Functional Competence Graph** - X-axis: Number of Tokens (0-286B) - Y-axis: Functional Competence (0-0.3 scale) - Legend: Model sizes with blue gradient - Vertical threshold line at 100B tokens ### Detailed Analysis **Brain Alignment Trends** - 410M (light green): Gradual increase with plateau at ~0.45 - 1B (green): Steeper rise to ~0.55 - 1.4B (dark green): Peaks at 0.58 before declining - 2.8B (dotted green): Stable ~0.52 - 6.9B (solid green): Sharp peak at 100B tokens (0.58), then decline **Formal Competence Trends** - All models show sigmoidal growth - 410M: Reaches 0.65 at 100B tokens - 6.9B: Fastest ascent, plateauing at 0.72 - Post-100B tokens: Minimal improvement across models **Functional Competence Trends** - Linear progression for all models - 410M: Max 0.28 at 286B tokens - 6.9B: Reaches 0.30 at 200B tokens - Post-100B tokens: Accelerated gains for larger models ### Key Observations 1. **Model Size Correlation**: Larger models consistently outperform smaller ones across all metrics 2. **Token Threshold Effect**: Significant performance shifts occur around 100B tokens 3. **Brain Alignment Paradox**: 6.9B model shows peak at 100B tokens followed by decline 4. **R² Discrepancy**: Brain Alignment has higher correlation (0.65) than Functional Competence (0.36) 5. **Divergent Scaling**: Formal Competence plateaus faster than Functional Competence ### Interpretation The data suggests diminishing returns in Brain Alignment for the largest model (6.9B) beyond 100B tokens, contrasting with sustained gains in Functional Competence. This implies architectural limitations in neural alignment mechanisms at extreme scale. The 94.4% training time marker at 100B tokens indicates a critical inflection point where models achieve ~95% of their potential performance. The R² values reveal that Brain Alignment metrics better capture model capabilities than Functional Competence measures, possibly due to more direct neural correlation metrics. The consistent performance across all metrics for models above 1.4B suggests diminishing returns in parameter scaling beyond this threshold.