## Line Chart: Llama-3-8B and Llama-3-70B Performance ### Overview The image presents two line charts comparing the performance of Llama-3-8B and Llama-3-70B models across different layers. The y-axis represents ΔP (change in performance), and the x-axis represents the layer number. Each chart displays six data series, representing Q-Anchored and A-Anchored performance on PopQA, TriviaQA, HotpotQA, and NQ datasets. ### Components/Axes * **Titles:** * Left Chart: Llama-3-8B * Right Chart: Llama-3-70B * **Y-axis:** * Label: ΔP * Scale: -80 to 0, with increments of 20 (-60, -40, -20, 0) * **X-axis:** * Label: Layer * Left Chart Scale: 0 to 30, with increments of 10 (10, 20, 30) * Right Chart Scale: 0 to 80, with increments of 20 (20, 40, 60, 80) * **Legend:** Located at the bottom of the image. * Q-Anchored (PopQA): Solid Blue Line * A-Anchored (PopQA): Dashed Brown Line * Q-Anchored (TriviaQA): Dotted Green Line * A-Anchored (TriviaQA): Dotted Brown Line * Q-Anchored (HotpotQA): Dashed Purple Line * A-Anchored (HotpotQA): Dotted Green Line * Q-Anchored (NQ): Dotted Purple Line * A-Anchored (NQ): Dotted Gray Line ### Detailed Analysis **Llama-3-8B (Left Chart):** * **Q-Anchored (PopQA):** (Solid Blue) Starts at approximately 0 and decreases to around -60 by layer 30. * **A-Anchored (PopQA):** (Dashed Brown) Remains relatively stable around 0, with minor fluctuations. * **Q-Anchored (TriviaQA):** (Dotted Green) Starts near 0 and decreases to approximately -60 by layer 30. * **A-Anchored (TriviaQA):** (Dotted Brown) Remains relatively stable around 0, with minor fluctuations. * **Q-Anchored (HotpotQA):** (Dashed Purple) Starts near 0 and decreases to approximately -40 by layer 30. * **A-Anchored (HotpotQA):** (Dotted Green) Remains relatively stable around 0, with minor fluctuations. * **Q-Anchored (NQ):** (Dotted Purple) Remains relatively stable around 0, with minor fluctuations. * **A-Anchored (NQ):** (Dotted Gray) Remains relatively stable around 0, with minor fluctuations. **Llama-3-70B (Right Chart):** * **Q-Anchored (PopQA):** (Solid Blue) Starts at approximately 0 and decreases to around -80 by layer 80. * **A-Anchored (PopQA):** (Dashed Brown) Remains relatively stable around 0, with minor fluctuations. * **Q-Anchored (TriviaQA):** (Dotted Green) Starts near 0 and decreases to approximately -40 by layer 80. * **A-Anchored (TriviaQA):** (Dotted Brown) Remains relatively stable around 0, with minor fluctuations. * **Q-Anchored (HotpotQA):** (Dashed Purple) Starts near 0 and decreases to approximately -40 by layer 80. * **A-Anchored (HotpotQA):** (Dotted Green) Remains relatively stable around 0, with minor fluctuations. * **Q-Anchored (NQ):** (Dotted Purple) Remains relatively stable around 0, with minor fluctuations. * **A-Anchored (NQ):** (Dotted Gray) Remains relatively stable around 0, with minor fluctuations. ### Key Observations * For both models, the Q-Anchored performance on PopQA, TriviaQA, and HotpotQA datasets decreases significantly as the layer number increases. * The A-Anchored performance on all datasets remains relatively stable around 0 for both models. * The Llama-3-70B model has a longer x-axis (layer count) than the Llama-3-8B model. * The Q-Anchored (PopQA) line for Llama-3-70B shows the most significant decrease in performance, reaching approximately -80. ### Interpretation The data suggests that Q-Anchoring negatively impacts the performance of Llama models on PopQA, TriviaQA, and HotpotQA datasets as the model goes deeper into its layers. In contrast, A-Anchoring seems to have a negligible impact on performance. The Llama-3-70B model, with its increased number of layers, exhibits a more pronounced decrease in Q-Anchored performance, particularly for the PopQA dataset. This could indicate that the effect of Q-Anchoring becomes more detrimental with increased model depth. The stable performance of A-Anchored data suggests that anchoring the answer has little to no impact on the model's performance across different layers.

## Line Chart: Delta P (ΔP) vs. Layer for Llama Models ### Overview The image presents two line charts, side-by-side, displaying the change in probability (ΔP) as a function of layer number for two different Llama models: Llama-3-8B and Llama-3-70B. Each chart shows multiple lines representing different question-answering datasets and anchoring methods. The charts aim to visualize how the probability change varies across layers for each model and dataset combination. ### Components/Axes * **X-axis:** Layer (ranging from 0 to approximately 30 for Llama-3-8B and 0 to approximately 80 for Llama-3-70B). * **Y-axis:** ΔP (Delta P), representing the change in probability. The scale ranges from approximately -80 to 0. * **Models:** Llama-3-8B (left chart), Llama-3-70B (right chart). * **Datasets/Anchoring Methods (Legend):** * Q-Anchored (PopQA) - Blue solid line * A-Anchored (PopQA) - Orange dashed line * Q-Anchored (TriviaQA) - Pink solid line * A-Anchored (TriviaQA) - Brown solid line * Q-Anchored (HotpotQA) - Green solid line * A-Anchored (HotpotQA) - Teal dashed line * Q-Anchored (NQ) - Purple solid line * A-Anchored (NQ) - Grey solid line * **Legend Position:** Bottom-center of the image. ### Detailed Analysis or Content Details **Llama-3-8B (Left Chart):** * **Q-Anchored (PopQA):** The line starts at approximately 0 ΔP at layer 0, rapidly decreases to approximately -60 ΔP by layer 10, and continues to decrease to approximately -70 ΔP by layer 30. * **A-Anchored (PopQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 5, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. * **Q-Anchored (TriviaQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 10, and continues to decrease to approximately -60 ΔP by layer 30. * **A-Anchored (TriviaQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 5, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. * **Q-Anchored (HotpotQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 10, and continues to decrease to approximately -60 ΔP by layer 30. * **A-Anchored (HotpotQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 5, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. * **Q-Anchored (NQ):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 10, and continues to decrease to approximately -60 ΔP by layer 30. * **A-Anchored (NQ):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 5, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. **Llama-3-70B (Right Chart):** * **Q-Anchored (PopQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 20, and continues to decrease to approximately -60 ΔP by layer 60, then fluctuates. * **A-Anchored (PopQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 10, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. * **Q-Anchored (TriviaQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 20, and continues to decrease to approximately -60 ΔP by layer 60, then fluctuates. * **A-Anchored (TriviaQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 10, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. * **Q-Anchored (HotpotQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 20, and continues to decrease to approximately -60 ΔP by layer 60, then fluctuates. * **A-Anchored (HotpotQA):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 10, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. * **Q-Anchored (NQ):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -40 ΔP by layer 20, and continues to decrease to approximately -60 ΔP by layer 60, then fluctuates. * **A-Anchored (NQ):** The line starts at approximately 0 ΔP at layer 0, decreases to approximately -20 ΔP by layer 10, and fluctuates between approximately -20 and -40 ΔP for the remainder of the layers. ### Key Observations * For both models, the Q-Anchored lines consistently show a more significant decrease in ΔP compared to the A-Anchored lines. * The A-Anchored lines tend to plateau after a certain layer, while the Q-Anchored lines continue to decrease, albeit with some fluctuations. * The Llama-3-70B model exhibits a slower initial decrease in ΔP compared to the Llama-3-8B model, but the overall trend is similar. * The datasets (PopQA, TriviaQA, HotpotQA, NQ) do not appear to significantly alter the overall trend for either anchoring method within each model. ### Interpretation The charts suggest that question anchoring (Q-Anchored) leads to a more substantial reduction in probability as the layer number increases, compared to answer anchoring (A-Anchored). This could indicate that the model's confidence in its answers decreases more rapidly as it processes deeper layers when the question is used as the anchor. The plateauing of the A-Anchored lines might suggest that the model's initial answer representation stabilizes relatively quickly. The larger model (Llama-3-70B) shows a more gradual decrease in ΔP, potentially due to its increased capacity to maintain information across layers. The consistency of the trends across different datasets suggests that the observed behavior is not specific to any particular question-answering task. The negative ΔP values indicate a decrease in probability, which could be interpreted as a reduction in the model's certainty or confidence in its predictions as it processes information through deeper layers. The differences between the anchoring methods and model sizes provide insights into how these factors influence the model's internal representations and decision-making processes.

## Line Charts: Llama-3 Model Layer-wise ΔP Analysis ### Overview The image displays two side-by-side line charts comparing the layer-wise change in probability (ΔP) for two different-sized language models: Llama-3-8B (left) and Llama-3-70B (right). The charts analyze the performance of two anchoring methods (Q-Anchored and A-Anchored) across four different question-answering datasets (PopQA, TriviaQA, HotpotQA, NQ). ### Components/Axes * **Chart Titles:** "Llama-3-8B" (left chart), "Llama-3-70B" (right chart). * **Y-Axis:** Labeled "ΔP" (Delta P, likely representing a change in probability or performance metric). The scale ranges from -80 to 0, with major gridlines at intervals of 20. * **X-Axis:** Labeled "Layer". The scale for Llama-3-8B ranges from 0 to 30. The scale for Llama-3-70B ranges from 0 to 80. * **Legend:** Positioned at the bottom, spanning both charts. It defines eight data series using a combination of line color and style (solid vs. dashed): * **Q-Anchored (Solid Lines):** * Blue: PopQA * Green: TriviaQA * Purple: HotpotQA * Pink: NQ * **A-Anchored (Dashed Lines):** * Orange: PopQA * Red: TriviaQA * Brown: HotpotQA * Gray: NQ ### Detailed Analysis **Llama-3-8B Chart (Left):** * **Q-Anchored Series (Solid Lines):** All four solid lines show a pronounced downward trend, indicating a significant negative ΔP as the layer number increases. * The **Blue (PopQA)** and **Green (TriviaQA)** lines exhibit the steepest decline, dropping from near 0 at layer 0 to approximately -60 by layer 30. Their lowest points are around layers 15-20. * The **Purple (HotpotQA)** and **Pink (NQ)** lines follow a similar but slightly less severe downward trajectory, ending near -50 by layer 30. * **A-Anchored Series (Dashed Lines):** All four dashed lines remain relatively stable and close to the zero line throughout all layers, fluctuating mostly between -10 and +5. They show no strong downward or upward trend. **Llama-3-70B Chart (Right):** * **Q-Anchored Series (Solid Lines):** The pattern is more volatile but follows the same core trend as the 8B model. The lines show a general decline with significant fluctuations. * The **Blue (PopQA)**, **Green (TriviaQA)**, and **Purple (HotpotQA)** lines all descend sharply, reaching values between -60 and -80 by layer 80. The **Pink (NQ)** line also declines but ends slightly higher, around -50. * The decline appears to accelerate after approximately layer 40. * **A-Anchored Series (Dashed Lines):** Similar to the 8B model, the dashed lines for A-Anchored methods remain clustered near the zero line across all layers, showing minor fluctuations but no significant drift. ### Key Observations 1. **Method Dichotomy:** There is a stark and consistent contrast between the two anchoring methods. Q-Anchored methods (solid lines) result in a large, layer-dependent negative ΔP, while A-Anchored methods (dashed lines) maintain a ΔP near zero. 2. **Model Size Scaling:** The trend observed in the 8B model is amplified and extended in the larger 70B model. The negative ΔP for Q-Anchored methods reaches similar or greater magnitudes but is distributed across many more layers (80 vs. 30). 3. **Dataset Variation:** Within the Q-Anchored group, the PopQA (blue) and TriviaQA (green) datasets consistently show the most negative ΔP across both model sizes. The NQ (pink) dataset often shows the least negative ΔP among the Q-Anchored series. 4. **Volatility:** The Llama-3-70B chart exhibits greater high-frequency volatility (more jagged lines) in the Q-Anchored series compared to the Llama-3-8B chart. ### Interpretation This data suggests a fundamental difference in how the "Q-Anchored" and "A-Anchored" techniques influence the internal processing of the Llama-3 models across their layers. * **Q-Anchored Impact:** The strong negative ΔP trend for Q-Anchored methods indicates that this technique causes a progressive and significant reduction in the measured probability metric as information flows through the network's layers. This could imply that Q-Anchoring suppresses or alters the model's confidence or the probability assigned to certain outputs in a layer-wise manner. The effect is more pronounced for datasets like PopQA and TriviaQA. * **A-Anchored Stability:** In contrast, A-Anchored methods appear to be largely neutral, preserving the ΔP near its initial value throughout the network. This suggests this technique does not induce the same layer-wise drift in the model's internal representations for this metric. * **Scaling Effect:** The pattern holds across model scales (8B to 70B parameters), but the larger model's deeper architecture allows the effect to manifest over a longer sequence of layers, with increased volatility possibly reflecting more complex internal dynamics. **In essence, the charts demonstrate that the choice of anchoring method (Q vs. A) is a critical determinant of layer-wise behavior in Llama-3 models, with Q-Anchoring introducing a strong, dataset-sensitive, and layer-dependent suppression effect that scales with model depth.**

## Line Graph: Performance Comparison of Q-Anchored and A-Anchored Models Across Layers for Llama-3-8B and Llama-3-70B ### Overview The image contains two side-by-side line graphs comparing the performance (ΔP) of Q-Anchored and A-Anchored models across layers for two Llama versions: 3-8B (left) and 3-70B (right). The graphs show six distinct data series, each representing a combination of anchoring method (Q or A) and dataset/method (PopQA, TriviaQA, HotpotQA, NQ). Performance is measured as ΔP (change in performance) across layers, with values ranging from -80 to +20. ### Components/Axes - **X-Axis (Horizontal)**: Layer (0 to 30 for Llama-3-8B, 0 to 80 for Llama-3-70B) - **Y-Axis (Vertical)**: ΔP (Performance Change, range: -80 to +20) - **Legend**: Located at the bottom, with six entries: 1. **Q-Anchored (PopQA)**: Solid blue line 2. **Q-Anchored (TriviaQA)**: Dotted green line 3. **Q-Anchored (HotpotQA)**: Dashed purple line 4. **Q-Anchored (NQ)**: Dotted pink line 5. **A-Anchored (PopQA)**: Solid orange line 6. **A-Anchored (TriviaQA)**: Dotted orange line 7. **A-Anchored (HotpotQA)**: Dashed orange line 8. **A-Anchored (NQ)**: Dotted gray line ### Detailed Analysis #### Llama-3-8B (Left Chart) - **Q-Anchored (PopQA)**: Starts near 0, decreases sharply to ~-60 by layer 30. - **Q-Anchored (TriviaQA)**: Begins at ~-10, fluctuates between -20 and 0, ending near -40. - **Q-Anchored (HotpotQA)**: Starts at ~-5, peaks at ~+5 around layer 15, then drops to ~-30. - **Q-Anchored (NQ)**: Starts at ~-15, stabilizes near -10 by layer 30. - **A-Anchored (PopQA)**: Starts at ~-5, fluctuates between -10 and 0, ending near -15. - **A-Anchored (TriviaQA)**: Begins at ~-20, rises to ~-5 by layer 10, then drops to ~-35. - **A-Anchored (HotpotQA)**: Starts at ~-10, peaks at ~+10 around layer 15, then declines to ~-25. - **A-Anchored (NQ)**: Starts at ~-25, stabilizes near -20 by layer 30. #### Llama-3-70B (Right Chart) - **Q-Anchored (PopQA)**: Starts near 0, decreases to ~-70 by layer 80. - **Q-Anchored (TriviaQA)**: Begins at ~-10, fluctuates between -30 and -10, ending near -50. - **Q-Anchored (HotpotQA)**: Starts at ~-5, peaks at ~+10 around layer 40, then drops to ~-60. - **Q-Anchored (NQ)**: Starts at ~-20, stabilizes near -30 by layer 80. - **A-Anchored (PopQA)**: Starts at ~-5, fluctuates between -15 and 0, ending near -20. - **A-Anchored (TriviaQA)**: Begins at ~-20, rises to ~-5 by layer 20, then drops to ~-40. - **A-Anchored (HotpotQA)**: Starts at ~-10, peaks at ~+15 around layer 40, then declines to ~-35. - **A-Anchored (NQ)**: Starts at ~-25, stabilizes near -35 by layer 80. ### Key Observations 1. **General Trend**: Most lines show a downward trend in ΔP as layers increase, indicating performance degradation. 2. **Model Size Impact**: Llama-3-70B exhibits more pronounced fluctuations and steeper declines compared to Llama-3-8B. 3. **Anchoring Method**: Q-Anchored models generally perform worse (lower ΔP) than A-Anchored models across most datasets. 4. **Dataset Variability**: HotpotQA and NQ datasets show higher volatility in performance compared to PopQA and TriviaQA. 5. **Layer-Specific Peaks**: Some lines (e.g., Q-Anchored HotpotQA in Llama-3-70B) show mid-layer performance peaks before declining. ### Interpretation The data suggests that anchoring method (Q vs. A) and dataset choice significantly influence model performance. A-Anchored models consistently outperform Q-Anchored counterparts, particularly in larger models (70B). The HotpotQA and NQ datasets appear more challenging, causing sharper performance drops. The mid-layer peaks observed in some lines (e.g., HotpotQA) may indicate temporary stabilization or optimization points. The larger model (70B) shows greater sensitivity to anchoring choices, with more extreme performance variations. These trends highlight the importance of anchoring strategy and dataset selection in fine-tuning large language models.