## Line Charts: Brain Alignment vs. Number of Tokens for Different Model Sizes ### Overview The image presents three line charts comparing brain alignment (Pearson's r) against the number of tokens processed by different language models. Each chart corresponds to a model size (14M, 70M, and 160M). The charts display the brain alignment for two regions: the Language Network and V1, as the number of tokens increases. Shaded regions around each line represent the uncertainty or variability in the data. ### Components/Axes * **Title:** Each chart is titled with the model size: "14M", "70M", and "160M". * **Y-axis:** "Brain Alignment (Pearson's r)". The scale ranges from -0.025 to 0.150 with increments of 0.025. * **X-axis:** "Number of Tokens". The scale is non-linear and includes values like 0, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 512M, 1B, 2B, 4B, 8B, 16B, 20B, 40B, 60B, 80B, 100B, 120B, 140B, 160B, 180B, 200B, 220B, 240B, 260B, 280B, 286B. * **Legend:** Located at the bottom of the image. * **Language Network:** Represented by a green line with circular markers and a light green shaded area. * **V1:** Represented by a purple line with 'x' markers and a light purple shaded area. ### Detailed Analysis #### 14M Model * **Language Network (Green):** The brain alignment starts at approximately 0.06 (±0.01) and remains relatively stable until around 64M tokens. After 64M, the alignment increases sharply, reaching approximately 0.12 (±0.01) around 128M tokens, and then plateaus around 0.12-0.13 (±0.01) for the rest of the token range. * **V1 (Purple):** The brain alignment starts near 0.01 (±0.01) and fluctuates between 0.01 and 0.03 (±0.01) across the entire range of tokens, with no clear increasing or decreasing trend. #### 70M Model * **Language Network (Green):** Similar to the 14M model, the brain alignment starts around 0.05 (±0.01) and remains stable until approximately 64M tokens. It then increases sharply, reaching approximately 0.12 (±0.01) around 128M tokens, and plateaus around 0.12-0.13 (±0.01) for the rest of the token range. * **V1 (Purple):** The brain alignment starts near 0.00 (±0.01) and fluctuates between 0.00 and 0.03 (±0.01) across the entire range of tokens, with no clear increasing or decreasing trend. #### 160M Model * **Language Network (Green):** The brain alignment starts around 0.06 (±0.01) and remains relatively stable until approximately 64M tokens. It then increases sharply, reaching approximately 0.11 (±0.01) around 128M tokens, and plateaus around 0.11-0.12 (±0.01) for the rest of the token range. * **V1 (Purple):** The brain alignment starts near 0.02 (±0.01) and fluctuates between 0.00 and 0.03 (±0.01) across the entire range of tokens, with no clear increasing or decreasing trend. ### Key Observations * **Language Network Improvement:** The Language Network shows a significant increase in brain alignment after a certain number of tokens (around 64M), regardless of the model size. * **V1 Stability:** The V1 region shows relatively stable and low brain alignment across all model sizes and token ranges. * **Model Size Impact:** The 14M and 70M models show a slightly higher plateau in brain alignment for the Language Network compared to the 160M model. ### Interpretation The data suggests that increasing the number of tokens processed by a language model leads to improved brain alignment in the Language Network region, but only after a certain threshold (around 64M tokens). The V1 region does not show a similar improvement, indicating that the Language Network is more sensitive to the amount of training data. The slight difference in plateau levels between the models suggests that there might be an optimal model size or that other factors beyond the number of tokens influence brain alignment. The shaded regions indicate the variability in the data, which could be due to individual differences or other experimental factors.