## Line Graphs: Mistral-7B Model Performance Comparison ### Overview The image presents two line graphs comparing the performance of Mistral-7B models (v0.1 and v0.3) across different layers and question-answering datasets. The graphs depict the change in performance (ΔP) as a function of the layer number, with separate lines for question-anchored (Q-Anchored) and answer-anchored (A-Anchored) approaches on various datasets. ### Components/Axes * **Titles:** * Left Graph: "Mistral-7B-v0.1" * Right Graph: "Mistral-7B-v0.3" * **Y-Axis:** * Label: "ΔP" (Change in Performance) * Scale: -60 to 0, with tick marks at -40, -20, and 0. * **X-Axis:** * Label: "Layer" * Scale: 0 to 30, with tick marks every 10 units. * **Legend:** Located at the bottom of the image, spanning both graphs. * **Q-Anchored (PopQA):** Solid Blue Line * **A-Anchored (PopQA):** Dashed Brown Line * **Q-Anchored (TriviaQA):** Dotted Green Line * **A-Anchored (TriviaQA):** Dotted-Dashed Pink Line * **Q-Anchored (HotpotQA):** Dash-Dot Blue Line * **A-Anchored (HotpotQA):** Solid Green Line * **Q-Anchored (NQ):** Dotted-Dashed Pink Line * **A-Anchored (NQ):** Dotted Black Line ### Detailed Analysis **Left Graph (Mistral-7B-v0.1):** * **Q-Anchored (PopQA):** (Solid Blue Line) Starts at 0, decreases sharply to approximately -40 by layer 10, fluctuates between -30 and -50 until layer 30, and ends around -60. * **A-Anchored (PopQA):** (Dashed Brown Line) Starts at 0, decreases to approximately -10 by layer 10, and then remains relatively stable between -10 and -5 until layer 30. * **Q-Anchored (TriviaQA):** (Dotted Green Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -50 until layer 30, and ends around -60. * **A-Anchored (TriviaQA):** (Dotted-Dashed Pink Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **Q-Anchored (HotpotQA):** (Dash-Dot Blue Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **A-Anchored (HotpotQA):** (Solid Green Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **Q-Anchored (NQ):** (Dotted-Dashed Pink Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **A-Anchored (NQ):** (Dotted Black Line) Starts at 0, increases to approximately 10 by layer 10, and then remains relatively stable between 10 and 5 until layer 30. **Right Graph (Mistral-7B-v0.3):** * **Q-Anchored (PopQA):** (Solid Blue Line) Starts at 0, decreases sharply to approximately -40 by layer 10, fluctuates between -30 and -50 until layer 30, and ends around -60. * **A-Anchored (PopQA):** (Dashed Brown Line) Starts at 0, decreases to approximately -10 by layer 10, and then remains relatively stable between -10 and -5 until layer 30. * **Q-Anchored (TriviaQA):** (Dotted Green Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -50 until layer 30, and ends around -60. * **A-Anchored (TriviaQA):** (Dotted-Dashed Pink Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **Q-Anchored (HotpotQA):** (Dash-Dot Blue Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **A-Anchored (HotpotQA):** (Solid Green Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **Q-Anchored (NQ):** (Dotted-Dashed Pink Line) Starts at 0, decreases sharply to approximately -30 by layer 10, fluctuates between -30 and -40 until layer 30, and ends around -50. * **A-Anchored (NQ):** (Dotted Black Line) Starts at 0, increases to approximately 10 by layer 10, and then remains relatively stable between 10 and 5 until layer 30. ### Key Observations * The Q-Anchored approaches for PopQA, TriviaQA, HotpotQA, and NQ datasets generally show a decrease in performance (negative ΔP) as the layer number increases. * The A-Anchored approach for PopQA shows a slight decrease in performance, while A-Anchored for NQ shows a slight increase in performance. * The performance trends are similar between Mistral-7B-v0.1 and Mistral-7B-v0.3. * There is a noticeable drop in performance for Q-Anchored approaches in the initial layers (0-10). ### Interpretation The graphs suggest that the Q-Anchored approaches are more sensitive to the layer number, with performance decreasing as the model processes deeper layers. This could indicate that the question encoding becomes less relevant or effective in later layers. The A-Anchored approaches, on the other hand, show more stable performance across layers, suggesting that the answer encoding remains relevant throughout the model. The similarity in trends between v0.1 and v0.3 indicates that the performance characteristics are consistent across these versions of the Mistral-7B model. The initial drop in performance for Q-Anchored approaches may be due to the model initially focusing on question encoding but then shifting its attention to other aspects of the task in later layers.

\n ## Line Chart: Delta P (ΔP) vs. Layer for Mistral-7B Models ### Overview The image presents two line charts, side-by-side, displaying the change in probability (ΔP) as a function of layer number for two versions of the Mistral-7B language model: v0.1 and v0.3. Each chart shows multiple lines representing different question-answering datasets and anchoring methods. The x-axis represents the layer number, ranging from 0 to approximately 32. The y-axis represents ΔP, ranging from approximately -65 to 5. ### Components/Axes * **X-axis:** Layer (ranging from 0 to 32, with gridlines at integer values). * **Y-axis:** ΔP (Delta P, change in probability, ranging from approximately -65 to 5, with gridlines at intervals of 10). * **Left Chart Title:** Mistral-7B-v0.1 * **Right Chart Title:** Mistral-7B-v0.3 * **Legend (Bottom-Center):** Contains labels for each line, indicating the anchoring method and dataset. * Q-Anchored (PopQA) - Blue solid line * A-Anchored (PopQA) - Orange dashed line * Q-Anchored (TriviaQA) - Purple solid line * A-Anchored (TriviaQA) - Brown dashed line * Q-Anchored (HotpotQA) - Green dashed-dotted line * A-Anchored (HotpotQA) - Light Green dotted line * Q-Anchored (NQ) - Teal solid line * A-Anchored (NQ) - Grey dashed line ### Detailed Analysis or Content Details **Mistral-7B-v0.1 (Left Chart):** * **Q-Anchored (PopQA):** Starts at approximately 2, decreases steadily to approximately -45 at layer 30. * **A-Anchored (PopQA):** Starts at approximately 0, fluctuates around 0 until layer 20, then decreases to approximately -20 at layer 30. * **Q-Anchored (TriviaQA):** Starts at approximately 0, decreases to approximately -40 at layer 20, then continues to decrease to approximately -60 at layer 30. * **A-Anchored (TriviaQA):** Starts at approximately 0, fluctuates around 0 until layer 10, then decreases to approximately -30 at layer 30. * **Q-Anchored (HotpotQA):** Starts at approximately 0, decreases to approximately -30 at layer 10, then decreases more rapidly to approximately -60 at layer 30. * **A-Anchored (HotpotQA):** Starts at approximately 0, fluctuates around 0 until layer 15, then decreases to approximately -25 at layer 30. * **Q-Anchored (NQ):** Starts at approximately 0, decreases to approximately -35 at layer 10, then decreases more rapidly to approximately -60 at layer 30. * **A-Anchored (NQ):** Starts at approximately 0, fluctuates around 0 until layer 15, then decreases to approximately -20 at layer 30. **Mistral-7B-v0.3 (Right Chart):** * **Q-Anchored (PopQA):** Starts at approximately 2, decreases steadily to approximately -40 at layer 30. * **A-Anchored (PopQA):** Starts at approximately 0, fluctuates around 0 until layer 20, then decreases to approximately -15 at layer 30. * **Q-Anchored (TriviaQA):** Starts at approximately 0, decreases to approximately -35 at layer 20, then continues to decrease to approximately -55 at layer 30. * **A-Anchored (TriviaQA):** Starts at approximately 0, fluctuates around 0 until layer 10, then decreases to approximately -25 at layer 30. * **Q-Anchored (HotpotQA):** Starts at approximately 0, decreases to approximately -25 at layer 10, then decreases more rapidly to approximately -55 at layer 30. * **A-Anchored (HotpotQA):** Starts at approximately 0, fluctuates around 0 until layer 15, then decreases to approximately -20 at layer 30. * **Q-Anchored (NQ):** Starts at approximately 0, decreases to approximately -30 at layer 10, then decreases more rapidly to approximately -55 at layer 30. * **A-Anchored (NQ):** Starts at approximately 0, fluctuates around 0 until layer 15, then decreases to approximately -15 at layer 30. ### Key Observations * In both charts, the "Q-Anchored" lines generally exhibit a steeper decline in ΔP compared to the "A-Anchored" lines. * The "HotpotQA" and "NQ" datasets consistently show the most significant decreases in ΔP, particularly for the "Q-Anchored" method. * The v0.3 model generally shows less negative ΔP values across all datasets and anchoring methods compared to the v0.1 model, suggesting an improvement in performance. * The "A-Anchored" lines tend to remain closer to 0 for a longer period before decreasing, indicating a more stable initial probability change. ### Interpretation The charts illustrate how the change in probability (ΔP) varies across different layers of the Mistral-7B language model for different question-answering datasets and anchoring methods. The negative ΔP values suggest a decrease in the model's confidence or probability assignment as information propagates through the layers. The steeper decline in ΔP for "Q-Anchored" lines suggests that anchoring based on the question itself leads to a more pronounced shift in probability distribution as the model processes the information. The datasets "HotpotQA" and "NQ" appear to be more challenging for the model, as they exhibit the largest decreases in ΔP. The improvement observed in the v0.3 model, with less negative ΔP values, indicates that the model updates have likely enhanced its ability to maintain probability assignments across layers, potentially leading to more stable and accurate predictions. The "A-Anchored" lines' initial stability suggests that anchoring based on the answer might provide a more consistent starting point for probability calculations. The differences between datasets highlight the varying difficulty levels and characteristics of each dataset, influencing how the model processes and assigns probabilities. This analysis provides insights into the model's internal workings and potential areas for improvement.

## Comparative Line Charts: Model Performance Across Layers ### Overview The image displays two side-by-side line charts comparing the performance change (ΔP) across 30 layers of two language model versions: **Mistral-7B-v0.1** (left chart) and **Mistral-7B-v0.3** (right chart). Each chart plots the performance delta for four different question-answering (QA) datasets, using two distinct anchoring methods: "Q-Anchored" (solid lines) and "A-Anchored" (dashed lines). ### Components/Axes * **Chart Titles:** * Left Chart: `Mistral-7B-v0.1` * Right Chart: `Mistral-7B-v0.3` * **X-Axis (Both Charts):** * Label: `Layer` * Scale: Linear, from 0 to 30, with major ticks at 0, 10, 20, 30. * **Y-Axis (Both Charts):** * Label: `ΔP` (Delta P, likely representing a change in performance or probability). * Scale: Linear, from -60 to 0, with major ticks at -60, -40, -20, 0. * **Legend (Located below both charts):** * The legend contains 8 entries, mapping line color and style to a specific dataset and anchoring method. * **Q-Anchored (Solid Lines):** * Blue: `Q-Anchored (PopQA)` * Green: `Q-Anchored (TriviaQA)` * Purple: `Q-Anchored (HotpotQA)` * Pink: `Q-Anchored (NQ)` * **A-Anchored (Dashed Lines):** * Orange: `A-Anchored (PopQA)` * Red: `A-Anchored (TriviaQA)` * Brown: `A-Anchored (HotpotQA)` * Gray: `A-Anchored (NQ)` ### Detailed Analysis **Chart 1: Mistral-7B-v0.1** * **A-Anchored Lines (Dashed):** All four dashed lines (Orange, Red, Brown, Gray) remain relatively high and stable, fluctuating mostly between ΔP = -20 and 0 across all layers. They show minor dips but no severe downward trend. * **Q-Anchored Lines (Solid):** All four solid lines (Blue, Green, Purple, Pink) show a pronounced downward trend as layer number increases. * They start near ΔP = 0 at Layer 0. * They begin a steep decline around Layer 5-10. * They reach their lowest points (most negative ΔP) between Layers 25-30. * **Approximate Trough Values (Layer ~30):** * Blue (PopQA): ~ -60 * Green (TriviaQA): ~ -55 * Purple (HotpotQA): ~ -50 * Pink (NQ): ~ -45 * The lines are tightly clustered, with Blue (PopQA) generally being the lowest. **Chart 2: Mistral-7B-v0.3** * **A-Anchored Lines (Dashed):** Similar to v0.1, the dashed lines remain in the upper region (ΔP between -20 and 0). The Orange (PopQA) line appears slightly more volatile, with a notable dip around Layer 15. * **Q-Anchored Lines (Solid):** The downward trend is even more severe and consistent compared to v0.1. * The decline starts earlier, around Layer 3-5. * The lines are more tightly grouped during the descent. * They reach lower troughs overall. * **Approximate Trough Values (Layer ~30):** * Blue (PopQA): ~ -65 * Green (TriviaQA): ~ -60 * Purple (HotpotQA): ~ -55 * Pink (NQ): ~ -50 * The final drop from Layer 25 to 30 is particularly sharp for all Q-Anchored series. ### Key Observations 1. **Anchoring Method Dominance:** Across both model versions and all datasets, the **A-Anchored (dashed) method consistently results in significantly higher ΔP values** (closer to zero) than the Q-Anchored (solid) method. This is the most striking pattern. 2. **Layer-Dependent Degradation:** Performance change (ΔP) for the Q-Anchored method degrades dramatically with increasing layer depth. The effect is non-linear, with the steepest decline occurring in the middle to later layers (10-30). 3. **Model Version Comparison:** The degradation trend for Q-Anchored methods is **more severe in Mistral-7B-v0.3** than in v0.1. The lines descend faster and reach lower minima in the v0.3 chart. 4. **Dataset Variation:** Within the Q-Anchored group, the **PopQA dataset (Blue line) consistently shows the largest negative ΔP**, followed by TriviaQA, HotpotQA, and NQ. This hierarchy is consistent across both model versions. 5. **Stability of A-Anchored:** The A-Anchored lines, while showing some noise, do not exhibit the systematic layer-dependent collapse seen in the Q-Anchored lines. ### Interpretation This data suggests a fundamental difference in how information is processed or retained across the layers of the Mistral-7B model depending on the anchoring strategy. * **A-Anchored vs. Q-Anchored:** The "A-Anchored" method (likely anchoring on the *Answer*) appears to create a more stable representation that is robust to the transformations occurring across the model's depth. In contrast, the "Q-Anchored" method (anchoring on the *Question*) leads to representations that progressively diverge or degrade as they pass through subsequent layers, resulting in a large negative ΔP. This could indicate that answer-centric representations are more invariant within the model's processing pipeline. * **Layer-wise Function:** The charts imply that the model's middle and later layers (10-30) are where the most significant transformation or "drift" occurs for question-anchored representations. The early layers (0-5) show minimal change. * **Model Evolution:** The increased degradation in v0.3 suggests that the updates between model versions may have altered the internal processing dynamics, making the question-anchored pathway even more susceptible to layer-wise transformation. This could be a side effect of other training improvements. * **Dataset Difficulty:** The consistent ordering of datasets (PopQA > TriviaQA > HotpotQA > NQ in terms of negative ΔP) might reflect inherent properties of the datasets, such as the complexity or specificity of the questions, which affects how stable their anchored representations are through the network. **In summary, the visualization provides strong evidence that the choice of anchoring point (Question vs. Answer) is a critical factor influencing the stability of internal representations across the layers of a large language model, with answer anchoring providing far greater robustness.**

## Line Graph: ΔP vs. Layer in Mistral-7B Models (v0.1 and v0.3) ### Overview The image contains two side-by-side line graphs comparing the performance of Q-Anchored and A-Anchored methods across different datasets (PopQA, TriviaQA, HotpotQA, NQ) in Mistral-7B models (v0.1 and v0.3). The y-axis represents ΔP (change in performance), and the x-axis represents model layers (0–30). Each line corresponds to a specific anchoring method and dataset, with distinct colors and styles. --- ### Components/Axes - **Y-Axis**: ΔP (Performance Change), ranging from -60 to 0. - **X-Axis**: Layer (0–30), representing model depth. - **Legends**: - **Left Graph (v0.1)**: - Solid blue: Q-Anchored (PopQA) - Dashed orange: A-Anchored (PopQA) - Solid green: Q-Anchored (TriviaQA) - Dashed gray: A-Anchored (TriviaQA) - Solid purple: Q-Anchored (HotpotQA) - Dashed pink: A-Anchored (HotpotQA) - Solid red: Q-Anchored (NQ) - Dashed brown: A-Anchored (NQ) - **Right Graph (v0.3)**: - Same legend as v0.1 but applied to updated model version. --- ### Detailed Analysis #### Mistral-7B-v0.1 1. **Q-Anchored (PopQA)** (solid blue): - Starts at ΔP ≈ 0 (layer 0). - Sharp decline to ΔP ≈ -50 (layer 10). - Fluctuates between -30 and -50 until layer 30. 2. **A-Anchored (PopQA)** (dashed orange): - Starts at ΔP ≈ 0. - Gradual decline to ΔP ≈ -30 (layer 10). - Stabilizes around -25–-30. 3. **Q-Anchored (TriviaQA)** (solid green): - Sharp drop to ΔP ≈ -40 (layer 5). - Oscillates between -20 and -40. 4. **A-Anchored (TriviaQA)** (dashed gray): - Smoother decline to ΔP ≈ -25 (layer 10). - Stabilizes around -20–-25. 5. **Q-Anchored (HotpotQA)** (solid purple): - Moderate decline to ΔP ≈ -35 (layer 15). - Fluctuates between -25 and -35. 6. **A-Anchored (HotpotQA)** (dashed pink): - Gradual decline to ΔP ≈ -20 (layer 20). - Stabilizes around -15–-20. 7. **Q-Anchored (NQ)** (solid red): - Sharp drop to ΔP ≈ -55 (layer 10). - Recovers to ΔP ≈ -40 (layer 30). 8. **A-Anchored (NQ)** (dashed brown): - Steady decline to ΔP ≈ -30 (layer 20). - Stabilizes around -25–-30. #### Mistral-7B-v0.3 1. **Q-Anchored (PopQA)** (solid blue): - Starts at ΔP ≈ 0. - Gradual decline to ΔP ≈ -25 (layer 20). - Stabilizes around -20–-25. 2. **A-Anchored (PopQA)** (dashed orange): - Smooth decline to ΔP ≈ -20 (layer 20). - Stabilizes around -15–-20. 3. **Q-Anchored (TriviaQA)** (solid green): - Moderate decline to ΔP ≈ -30 (layer 15). - Fluctuates between -20 and -30. 4. **A-Anchored (TriviaQA)** (dashed gray): - Gradual decline to ΔP ≈ -22 (layer 25). - Stabilizes around -18–-22. 5. **Q-Anchored (HotpotQA)** (solid purple): - Slight decline to ΔP ≈ -15 (layer 10). - Stabilizes around -10–-15. 6. **A-Anchored (HotpotQA)** (dashed pink): - Minimal decline to ΔP ≈ -10 (layer 20). - Stabilizes around -5–-10. 7. **Q-Anchored (NQ)** (solid red): - Sharp drop to ΔP ≈ -45 (layer 10). - Recovers to ΔP ≈ -30 (layer 30). 8. **A-Anchored (NQ)** (dashed brown): - Steady decline to ΔP ≈ -25 (layer 25). - Stabilizes around -20–-25. --- ### Key Observations 1. **General Trend**: Both models show a decline in ΔP across layers, but v0.3 exhibits smoother and more stable trends. 2. **Q-Anchored vs. A-Anchored**: - Q-Anchored methods (solid lines) exhibit sharper initial declines and greater volatility, especially in v0.1. - A-Anchored methods (dashed lines) show more gradual and stable performance. 3. **Dataset Impact**: - **PopQA/TriviaQA**: Higher volatility in Q-Anchored methods. - **HotpotQA/NQ**: Smoother trends, with NQ showing the most extreme initial drops. 4. **Version Comparison**: - v0.3 demonstrates improved stability across all methods, with reduced fluctuations compared to v0.1. --- ### Interpretation The data suggests that anchoring methods significantly influence model performance stability. Q-Anchored methods are more sensitive to layer changes, leading to larger ΔP variations, while A-Anchored methods maintain steadier performance. The datasets' complexity correlates with volatility: simpler datasets (e.g., PopQA) show sharper declines, while complex ones (e.g., HotpotQA) exhibit smoother trends. The transition from v0.1 to v0.3 indicates architectural improvements, reducing performance instability. Notably, Q-Anchored (NQ) in v0.1 experiences the most drastic drop (-55), suggesting potential overfitting or dataset-specific challenges. These findings highlight the importance of anchoring strategy selection based on dataset characteristics and model version.