## Chart: Explained Effect vs. Number of Edges Kept for Different Models and Tasks ### Overview The image presents four line charts comparing the "Explained Effect" against the "Number of Edges Kept (log scale)" for different models (GPT-2 and OLMo-7B) and tasks (Greater Than, IOI, Docstring, and IOI Long). Each chart compares a dense model with its sparse counterpart. The x-axis is logarithmic, and the y-axis represents the explained effect, ranging from 0.0 to 1.0. The charts also indicate the multiplicative factor by which the number of edges kept needs to be increased in the sparse model to achieve the same explained effect as the dense model. ### Components/Axes * **X-axis:** Number of Edges Kept (log scale). The scale ranges from 10^0 to 10^4 or 10^5 depending on the chart. * **Y-axis:** Explained Effect. The scale ranges from 0.0 to 1.0. * **Titles:** * Top-left: Greater Than * Top-middle-left: IOI * Top-middle-right: Docstring * Top-right: IOI Long * **Legends:** * Greater Than and IOI charts: * Blue line: GPT-2 * Orange line: Sparse GPT-2 * Docstring and IOI Long charts: * Green line: OLMo-7B * Pink line: Sparse OLMo-7B * **Multiplicative Factors:** * Greater Than: 41.9x * IOI: 14.9x * Docstring: 5.5x * IOI Long: 3.1x ### Detailed Analysis **1. Greater Than** * **GPT-2 (Blue):** The explained effect increases sharply between 10^1 and 10^2 edges, reaching near 1.0 by 10^3 edges. * At 10^0 edges, the explained effect is approximately 0.1. * At 10^1 edges, the explained effect is approximately 0.2. * At 10^2 edges, the explained effect is approximately 0.9. * At 10^3 edges, the explained effect is approximately 1.0. * **Sparse GPT-2 (Orange):** The explained effect increases gradually between 10^0 and 10^3 edges, reaching near 1.0 by 10^4 edges. * At 10^0 edges, the explained effect is approximately 0.0. * At 10^1 edges, the explained effect is approximately 0.2. * At 10^2 edges, the explained effect is approximately 0.7. * At 10^3 edges, the explained effect is approximately 0.95. * At 10^4 edges, the explained effect is approximately 1.0. * **Multiplicative Factor:** 41.9x. This indicates that the sparse model needs 41.9 times more edges to achieve the same explained effect as the dense model. **2. IOI** * **GPT-2 (Blue):** The explained effect increases sharply between 10^1 and 10^2 edges, reaching near 1.0 by 10^3 edges. * At 10^0 edges, the explained effect is approximately 0.05. * At 10^1 edges, the explained effect is approximately 0.2. * At 10^2 edges, the explained effect is approximately 0.8. * At 10^3 edges, the explained effect is approximately 0.95. * **Sparse GPT-2 (Orange):** The explained effect increases gradually between 10^0 and 10^3 edges, reaching near 1.0 by 10^4 edges. * At 10^0 edges, the explained effect is approximately 0.15. * At 10^1 edges, the explained effect is approximately 0.3. * At 10^2 edges, the explained effect is approximately 0.85. * At 10^3 edges, the explained effect is approximately 0.95. * At 10^4 edges, the explained effect is approximately 1.0. * **Multiplicative Factor:** 14.9x. **3. Docstring** * **OLMo-7B (Green):** The explained effect increases sharply between 10^2 and 10^4 edges, reaching near 1.0 by 10^5 edges. * At 10^1 edges, the explained effect is approximately 0.0. * At 10^2 edges, the explained effect is approximately 0.1. * At 10^3 edges, the explained effect is approximately 0.5. * At 10^4 edges, the explained effect is approximately 0.9. * At 10^5 edges, the explained effect is approximately 1.0. * **Sparse OLMo-7B (Pink):** The explained effect increases sharply between 10^2 and 10^4 edges, reaching near 1.0 by 10^5 edges. * At 10^1 edges, the explained effect is approximately 0.0. * At 10^2 edges, the explained effect is approximately 0.05. * At 10^3 edges, the explained effect is approximately 0.3. * At 10^4 edges, the explained effect is approximately 0.8. * At 10^5 edges, the explained effect is approximately 1.0. * **Multiplicative Factor:** 5.5x. **4. IOI Long** * **OLMo-7B (Green):** The explained effect increases sharply between 10^2 and 10^4 edges, reaching near 1.0 by 10^5 edges. * At 10^1 edges, the explained effect is approximately 0.0. * At 10^2 edges, the explained effect is approximately 0.1. * At 10^3 edges, the explained effect is approximately 0.4. * At 10^4 edges, the explained effect is approximately 0.8. * At 10^5 edges, the explained effect is approximately 1.0. * **Sparse OLMo-7B (Pink):** The explained effect increases sharply between 10^2 and 10^4 edges, reaching near 1.0 by 10^5 edges. * At 10^1 edges, the explained effect is approximately 0.0. * At 10^2 edges, the explained effect is approximately 0.05. * At 10^3 edges, the explained effect is approximately 0.3. * At 10^4 edges, the explained effect is approximately 0.7. * At 10^5 edges, the explained effect is approximately 1.0. * **Multiplicative Factor:** 3.1x. ### Key Observations * The sparse models consistently require more edges than their dense counterparts to achieve the same level of explained effect. * The "Greater Than" task shows the largest difference between the dense and sparse models (41.9x), while "IOI Long" shows the smallest difference (3.1x). * The x-axis is logarithmic, indicating that the number of edges has a significant impact on the explained effect, especially in the lower range of edge counts. * The explained effect generally plateaus as the number of edges increases, approaching 1.0 for all models and tasks. ### Interpretation The charts demonstrate the trade-off between model sparsity and performance. Sparse models, by definition, have fewer connections (edges) than dense models. The data suggests that while sparse models can achieve comparable performance to dense models, they often require a significantly larger number of edges to do so. The multiplicative factors (41.9x, 14.9x, 5.5x, and 3.1x) quantify this difference, indicating how much more "capacity" (in terms of edges) the sparse model needs to match the dense model's explained effect. The variation in multiplicative factors across different tasks ("Greater Than," "IOI," "Docstring," and "IOI Long") suggests that the impact of sparsity depends on the specific task being performed. For example, the "Greater Than" task appears to be more sensitive to sparsity than the "IOI Long" task. This could be due to differences in the complexity of the tasks or the types of relationships that need to be captured by the model. The logarithmic scale of the x-axis highlights the importance of the initial edges in the model. Adding edges in the lower range of the scale has a much more significant impact on the explained effect than adding edges in the higher range. This suggests that the initial connections in the model are crucial for capturing the essential relationships in the data.

## Line Charts: Explained Effect vs. Number of Edges Kept Across Models and Tasks ### Overview The image consists of four horizontally aligned line charts comparing the performance of standard language models against their "sparse" counterparts across four different tasks: "Greater Than", "IOI", "Docstring", and "IOI Long". The charts demonstrate how many "edges" (presumably components or connections in a computational graph or neural network) must be kept to achieve a certain level of "Explained Effect" (performance or fidelity). In all four charts, the sparse models achieve high performance using significantly fewer edges than the standard models. ### Components/Axes **Global Axes (Consistent across all four charts):** * **Y-axis:** Labeled "Explained Effect". Linear scale with major tick marks at `0.0`, `0.5`, and `1.0`. Light gray horizontal grid lines correspond to these ticks. * **X-axis:** Labeled "Number of Edges Kept (log scale)". Logarithmic scale. * Charts 1 & 2 (Left): Major ticks at `10^0`, `10^1`, `10^2`, `10^3`, `10^4`. * Charts 3 & 4 (Right): Major ticks at `10^0`, `10^1`, `10^2`, `10^3`, `10^4`, `10^5`. Light gray vertical grid lines correspond to these major ticks. **Legends & Color Mapping:** * **Legend 1 (Located in the bottom-right of the first chart):** Applies to the first two charts. * Blue Line: `GPT-2` * Orange Line: `Sparse GPT-2` * **Legend 2 (Located in the bottom-right of the third chart):** Applies to the last two charts. * Green Line: `OLMo-7B` * Purple/Pink Line: `Sparse OLMo-7B` **Annotations:** * Each chart features a horizontal dashed black line connecting the two curves near the top of the Y-axis (approximately at Y = 0.9). * Above each dashed line is a text label indicating a multiplier (e.g., `41.9x`), representing the ratio of edges required by the standard model versus the sparse model to achieve that specific Explained Effect. --- ### Detailed Analysis #### Region 1: "Greater Than" Chart (Far Left) * **Trend Verification:** Both the orange line (Sparse GPT-2) and blue line (GPT-2) follow an upward-sloping S-curve (sigmoid) trajectory. The orange line rises steeply at a much lower X-value than the blue line. Faint shaded bands (indicating variance or confidence intervals) are visible around both lines, slightly more pronounced on the orange line at the lower end. * **Data Points (Approximate):** * **Sparse GPT-2 (Orange):** Starts at Y ≈ 0.05 at X = $10^0$. Crosses Y = 0.5 at X ≈ $10^1$. Reaches Y = 1.0 at X ≈ $2 \times 10^2$. * **GPT-2 (Blue):** Starts at Y ≈ 0.0 at X = $10^0$. Crosses Y = 0.5 at X ≈ $10^3$. Reaches Y = 1.0 at X ≈ $10^4$. * **Annotation:** A dashed line at Y ≈ 0.9 connects the orange curve (at X ≈ 100) to the blue curve (at X ≈ 4000). The label reads **`41.9x`**. #### Region 2: "IOI" Chart (Second from Left) * **Trend Verification:** Both lines slope upward. The orange line (Sparse GPT-2) again precedes the blue line (GPT-2). The shaded variance band is notably visible on the orange line between X = $10^0$ and $10^1$. * **Data Points (Approximate):** * **Sparse GPT-2 (Orange):** Starts at Y ≈ 0.15 at X = $10^0$. Crosses Y = 0.5 at X ≈ $10^1$. Reaches Y = 1.0 at X ≈ $10^3$. * **GPT-2 (Blue):** Starts at Y ≈ 0.05 at X = $10^0$. Crosses Y = 0.5 at X ≈ $10^2$. Reaches Y = 1.0 at X ≈ $5 \times 10^3$. * **Annotation:** A dashed line at Y ≈ 0.9 connects the orange curve to the blue curve. The label reads **`14.9x`**. #### Region 3: "Docstring" Chart (Third from Left) * **Trend Verification:** Both the purple line (Sparse OLMo-7B) and green line (OLMo-7B) slope upward in an S-curve. The purple line rises earlier than the green line. The curves here are smoother and more gradual than in the GPT-2 charts. * **Data Points (Approximate):** * **Sparse OLMo-7B (Purple):** Starts at Y ≈ 0.0 at X = $10^0$. Crosses Y = 0.5 at X ≈ $2 \times 10^3$. Reaches Y = 1.0 at X ≈ $5 \times 10^4$. * **OLMo-7B (Green):** Starts at Y ≈ 0.0 at X = $10^0$. Crosses Y = 0.5 at X ≈ $10^4$. Reaches Y = 1.0 at X ≈ $2 \times 10^5$. * **Annotation:** A dashed line at Y ≈ 0.9 connects the purple curve to the green curve. The label reads **`5.5x`**. #### Region 4: "IOI Long" Chart (Far Right) * **Trend Verification:** Both lines slope upward. The purple line (Sparse OLMo-7B) rises earlier than the green line (OLMo-7B). * **Data Points (Approximate):** * **Sparse OLMo-7B (Purple):** Starts at Y ≈ 0.0 at X = $10^0$. Crosses Y = 0.5 at X ≈ $10^3$. Reaches Y = 1.0 at X ≈ $5 \times 10^4$. * **OLMo-7B (Green):** Starts at Y ≈ 0.0 at X = $10^0$. Crosses Y = 0.5 at X ≈ $5 \times 10^3$. Reaches Y = 1.0 at X ≈ $10^5$. * **Annotation:** A dashed line at Y ≈ 0.9 connects the purple curve to the green curve. The label reads **`3.1x`**. --- ### Key Observations 1. **Consistent Superiority of Sparse Models in Efficiency:** Across all four tasks and both base models (GPT-2 and OLMo-7B), the "Sparse" variant consistently achieves a high "Explained Effect" using exponentially fewer edges than the standard model. This is visually represented by the sparse curves being shifted significantly to the left. 2. **Varying Degrees of Compression:** The efficiency gain (represented by the multiplier annotations at Y ≈ 0.9) varies drastically depending on the task and model. * The highest efficiency gain is seen in GPT-2 on the "Greater Than" task (41.9x fewer edges needed). * The lowest efficiency gain shown is OLMo-7B on the "IOI Long" task (3.1x fewer edges needed). 3. **Scale Differences:** The OLMo-7B tasks (Docstring, IOI Long) generally require more edges overall to achieve a high explained effect compared to the GPT-2 tasks, evidenced by the X-axis extending to $10^5$ rather than $10^4$. ### Interpretation These charts are highly likely from a research paper focused on model interpretability, circuit discovery, or network pruning. * **"Edges Kept"** likely refers to the number of connections, attention heads, or computational sub-components retained when attempting to isolate the specific "circuit" within the neural network responsible for a specific task (like identifying "Greater Than" relationships or resolving Indirect Object Identification - "IOI"). * **"Explained Effect"** serves as a metric of fidelity—how well the pruned/isolated sub-network performs the task compared to the full, unpruned model (where 1.0 equals 100% of the original performance). * **The Core Finding:** The data demonstrates that applying "Sparse" techniques (likely sparse autoencoders or sparse probing methods) allows researchers to isolate the functional circuits of a language model much more effectively. Because the sparse models reach 90% explained effect with 3.1x to 41.9x fewer edges, it proves that standard models have highly entangled representations, whereas sparse models disentangle these representations, allowing for much tighter, more interpretable, and highly compressed computational graphs without losing task-specific performance.

\n ## Charts: Effect of Edge Pruning on Model Performance ### Overview The image presents four separate charts, each depicting the relationship between the number of edges kept (on a logarithmic scale) and the explained effect for different model configurations. Each chart focuses on a different pruning target: "Greater Than", "IOI", "Docstring", and "IOI Long". Each chart compares the performance of a dense model (GPT-2 or OLMo-7B) against a sparse version of the same model. The explained effect is a measure of how well the model performs, ranging from 0 to 1. ### Components/Axes Each chart shares the following components: * **X-axis:** "Number of Edges Kept (log scale)". The scale ranges from 10¹ to 10⁵. * **Y-axis:** "Explained Effect". The scale ranges from 0.0 to 1.0. * **Title:** Each chart has a title indicating the pruning target ("Greater Than", "IOI", "Docstring", "IOI Long"). * **Legend:** Each chart has a legend identifying the two data series: a dense model and a sparse model. * **Data Series:** Each chart contains two lines representing the explained effect for the dense and sparse models. The specific models used in each chart are: * **"Greater Than" Chart:** GPT-2 (blue line), Sparse GPT-2 (orange line). * **"IOI" Chart:** GPT-2 (blue line), Sparse GPT-2 (orange line). * **"Docstring" Chart:** OLMo-7B (pink line), Sparse OLMo-7B (teal line). * **"IOI Long" Chart:** OLMo-7B (pink line), Sparse OLMo-7B (teal line). Each chart also displays a text label indicating the compression ratio achieved by the sparse model, e.g., "41.9x". ### Detailed Analysis **"Greater Than" Chart:** * The blue line (GPT-2) starts at approximately 0.15 at 10¹ edges and quickly rises to approximately 0.95 at 10³ edges, then plateaus. * The orange line (Sparse GPT-2) starts at approximately 0.05 at 10¹ edges, rises more gradually than the blue line, reaching approximately 0.85 at 10³ edges, and then plateaus. * Compression ratio: 41.9x **"IOI" Chart:** * The blue line (GPT-2) starts at approximately 0.1 at 10¹ edges and rises rapidly to approximately 0.95 at 10³ edges, then plateaus. * The orange line (Sparse GPT-2) starts at approximately 0.05 at 10¹ edges, rises more gradually, reaching approximately 0.8 at 10³ edges, and then plateaus. * Compression ratio: 14.9x **"Docstring" Chart:** * The pink line (OLMo-7B) starts at approximately 0.1 at 10¹ edges and rises rapidly to approximately 0.95 at 10⁴ edges, then plateaus. * The teal line (Sparse OLMo-7B) starts at approximately 0.05 at 10¹ edges, rises more gradually, reaching approximately 0.85 at 10⁴ edges, and then plateaus. * Compression ratio: 5.5x **"IOI Long" Chart:** * The pink line (OLMo-7B) starts at approximately 0.05 at 10¹ edges and rises gradually to approximately 0.9 at 10⁵ edges. * The teal line (Sparse OLMo-7B) starts at approximately 0.02 at 10¹ edges and rises more gradually, reaching approximately 0.75 at 10⁵ edges. * Compression ratio: 3.1x ### Key Observations * In all charts, the sparse models consistently underperform the dense models, but achieve significant compression. * The compression ratio varies depending on the pruning target. "Greater Than" achieves the highest compression (41.9x), while "IOI Long" achieves the lowest (3.1x). * The "IOI Long" chart shows the slowest rise in explained effect for both models, indicating that this pruning target is the most challenging to maintain performance on. * The sparse models exhibit a more gradual increase in explained effect as the number of edges kept increases, suggesting that they are more sensitive to edge pruning. ### Interpretation These charts demonstrate the trade-off between model performance and compression achieved through edge pruning. Pruning edges reduces the model size (and thus computational cost) but also reduces the model's ability to explain the data. The compression ratio indicates how much smaller the sparse model is compared to the dense model. The varying compression ratios across different pruning targets suggest that some targets are more amenable to pruning than others. "Greater Than" appears to be the most effective pruning target, as it allows for the highest compression with a relatively small performance loss. "IOI Long" is the least effective, requiring a significantly larger number of edges to achieve comparable performance. The slower rise in explained effect for the "IOI Long" target suggests that this target captures more complex relationships in the data, which are more difficult to approximate with a sparse model. This implies that the "IOI Long" target may be more important for maintaining model accuracy, even at the cost of compression. The consistent underperformance of the sparse models highlights the importance of carefully evaluating the trade-off between performance and compression when applying edge pruning techniques. The optimal pruning strategy will depend on the specific application and the desired balance between these two factors.

## Line Charts: Model Sparsity Efficiency Comparison ### Overview The image displays four separate line charts arranged horizontally, comparing the performance of standard versus sparse versions of two language models (GPT-2 and OLMo-7B) across four different tasks or benchmarks. Each chart plots "Explained Effect" against the "Number of Edges Kept" on a logarithmic scale, demonstrating how model performance (explained effect) improves as more edges (likely representing model parameters or connections) are retained. The sparse model variants consistently achieve higher explained effect with fewer edges. ### Components/Axes * **Titles (Top of each chart, left to right):** "Greater Than", "IOI", "Docstring", "IOI Long". * **Y-Axis (All charts):** Label: "Explained Effect". Scale: Linear, from 0.0 to 1.0, with major ticks at 0.0, 0.5, and 1.0. * **X-Axis (All charts):** Label: "Number of Edges Kept (log scale)". Scale: Logarithmic. * Charts 1 & 2 ("Greater Than", "IOI"): Range from 10⁰ to 10⁴. * Charts 3 & 4 ("Docstring", "IOI Long"): Range from 10⁰ to 10⁵. * **Legends:** * Charts 1 & 2: Located in the bottom-right corner. Contains two entries: a blue line labeled "GPT-2" and an orange line labeled "Sparse GPT-2". * Charts 3 & 4: Located in the bottom-right corner. Contains two entries: a green line labeled "OLMo-7B" and a pink line labeled "Sparse OLMo-7B". * **Annotations:** Each chart contains a horizontal dashed black line near the top, connecting the plateau points of the two curves. Above this line is a text annotation indicating a multiplier (e.g., "41.9x"). ### Detailed Analysis **Chart 1: Greater Than** * **Trend Verification:** The blue line (GPT-2) shows a gradual, sigmoidal increase from near 0.0 at 10⁰ edges to 1.0 at approximately 10⁴ edges. The orange line (Sparse GPT-2) rises much more steeply, reaching 1.0 at just above 10² edges. * **Data Points & Annotation:** The dashed line and annotation "41.9x" indicate that the Sparse GPT-2 model achieves the same maximum explained effect (1.0) using approximately 41.9 times fewer edges than the standard GPT-2 model. **Chart 2: IOI** * **Trend Verification:** Similar pattern to Chart 1. The blue line (GPT-2) increases gradually. The orange line (Sparse GPT-2) increases more rapidly. * **Data Points & Annotation:** The annotation "14.9x" signifies that Sparse GPT-2 reaches peak performance with about 14.9 times fewer edges than GPT-2 on the IOI task. **Chart 3: Docstring** * **Trend Verification:** The green line (OLMo-7B) shows a steady increase. The pink line (Sparse OLMo-7B) has a steeper slope, indicating faster performance gain per edge added. * **Data Points & Annotation:** The annotation "5.5x" shows Sparse OLMo-7B is 5.5 times more edge-efficient than standard OLMo-7B for the Docstring task. **Chart 4: IOI Long** * **Trend Verification:** The green line (OLMo-7B) and pink line (Sparse OLMo-7B) follow similar trajectories to Chart 3, with the sparse variant maintaining a lead. * **Data Points & Annotation:** The annotation "3.1x" indicates a 3.1x edge efficiency advantage for Sparse OLMo-7B over OLMo-7B on the IOI Long task. ### Key Observations 1. **Consistent Superiority of Sparse Models:** In all four tasks, the sparse model variant (orange or pink line) achieves a higher "Explained Effect" at any given number of edges compared to its standard counterpart (blue or green line). 2. **Varying Efficiency Gains:** The magnitude of the efficiency gain (the multiplier) varies significantly by task and model. It is highest for GPT-2 on the "Greater Than" task (41.9x) and lowest for OLMo-7B on the "IOI Long" task (3.1x). 3. **Task/Model Dependency:** The efficiency gap appears larger for the GPT-2 model pair (Charts 1 & 2) than for the OLMo-7B pair (Charts 3 & 4) across the presented tasks. 4. **Performance Ceiling:** All models eventually reach an Explained Effect of 1.0, but the sparse models reach this ceiling at a much lower edge count. ### Interpretation These charts provide a Peircean investigation into the efficiency of model sparsification techniques. The "Explained Effect" likely measures how well a simplified (sparse) model can replicate the behavior or performance of a full model. The data demonstrates a clear and significant finding: **pruning a model to keep only a subset of its edges (sparsification) does not merely maintain performance but does so with dramatically greater parameter efficiency.** The relationship shown is a classic efficiency frontier: sparse models dominate the standard models on a plot of performance vs. resource (edges) usage. The varying multipliers (41.9x, 14.9x, etc.) suggest that the benefit of sparsification is not uniform. It is highly dependent on both the base model architecture (GPT-2 vs. OLMo-7B) and the specific cognitive task being evaluated (e.g., "Greater Than" vs. "IOI Long"). This implies that some tasks or model structures are more amenable to compression than others. The anomaly to note is the significant drop in the efficiency multiplier for the OLMo-7B model on the "IOI Long" task compared to its performance on "Docstring," suggesting the long-context IOI task may rely on distributed knowledge that is harder to sparsify effectively. Overall, the visual evidence strongly advocates for the use of sparse models as a means to achieve high performance with a fraction of the computational footprint.

## Line Graphs: Model Performance vs. Edge Retention ### Overview The image contains four comparative line graphs analyzing the performance of dense vs. sparse neural network models across different tasks. Each graph plots "Explained Effect" (y-axis) against "Number of Edges Kept" (x-axis, log scale). The graphs demonstrate how model efficiency improves with edge retention while maintaining performance. ### Components/Axes 1. **X-Axis**: "Number of Edges Kept (log scale)" ranging from 10⁰ to 10⁵ 2. **Y-Axis**: "Explained Effect" ranging from 0.0 to 1.0 3. **Legends**: Positioned bottom-right in each graph, showing: - GPT-2 (blue) vs. Sparse GPT-2 (orange) - OLMo-7B (teal) vs. Sparse OLMo-7B (pink) 4. **Task Titles**: Top of each graph: - Greater Than - IOI - Docstring - IOI Long ### Detailed Analysis 1. **Greater Than** - Blue (GPT-2): Gradual curve reaching ~0.8 at 10³ edges - Orange (Sparse GPT-2): Steeper ascent, plateaus at 1.0 with 41.9x efficiency gain - Key point: Sparse model achieves full effect at 10² edges vs. 10³ for dense 2. **IOI** - Blue (GPT-2): Reaches ~0.9 at 10³ edges - Orange (Sparse GPT-2): Peaks at 1.0 with 14.9x efficiency - Crossover: Sparse model surpasses dense at ~10¹ edges 3. **Docstring** - Teal (OLMo-7B): Slow rise to ~0.8 at 10⁴ edges - Pink (Sparse OLMo-7B): Rapid ascent to 1.0 with 5.5x gain - Efficiency: Sparse model achieves 90% effect at 10² edges 4. **IOI Long** - Teal (OLMo-7B): Gradual increase to ~0.9 at 10⁵ edges - Pink (Sparse OLMo-7B): Reaches 1.0 with 3.1x efficiency - Longest retention needed for sparse model to match dense ### Key Observations 1. All sparse models achieve full effect (1.0) with significantly fewer edges 2. Efficiency multipliers range from 3.1x to 41.9x across tasks 3. "Greater Than" shows highest efficiency gain (41.9x) 4. "IOI Long" demonstrates lowest efficiency multiplier (3.1x) 5. All graphs show crossover points where sparse models outperform dense ### Interpretation The data reveals that sparse neural network architectures maintain comparable performance to dense models while using fewer computational resources. The efficiency gains (multipliers) suggest sparse models could reduce hardware requirements by 3-40x depending on task complexity. The "IOI Long" task shows diminishing returns for sparsity, possibly due to longer sequence dependencies requiring more edge retention. These findings support the hypothesis that model sparsity preserves functionality while enabling more efficient deployment, particularly for tasks with localized dependencies like "Greater Than" and "Docstring".