## Bar Chart: Relative Attention Share by Model Layer ### Overview The image is a bar chart showing the relative attention share of "System Prompt" and "Maps" across different model layers (L0 to L27). The y-axis represents the relative attention share, ranging from 0.0 to 1.0. The x-axis represents the model layer. Each bar is divided into two colored sections: gray for "System Prompt" and blue for "Maps," indicating the proportion of attention each receives at that layer. ### Components/Axes * **Y-axis:** "Relative Attention Share," ranging from 0.0 to 1.0 in increments of 0.2. * **X-axis:** "Model Layer," with labels L0, L1, L2, L3, L14, L15, L16, L18, L20, L21, L24, L25, L26, L27. * **Legend:** Located in the bottom-right corner, indicating gray for "System Prompt" and blue for "Maps." ### Detailed Analysis Here's a breakdown of the attention share for each layer: * **L0:** System Prompt ~0.3, Maps ~0.7 * **L1:** System Prompt ~0.56, Maps ~0.44 * **L2:** System Prompt ~0.95, Maps ~0.05 * **L3:** System Prompt ~0.92, Maps ~0.08 * **L14:** System Prompt ~0.87, Maps ~0.13 * **L15:** System Prompt ~0.68, Maps ~0.32 * **L16:** System Prompt ~0.7, Maps ~0.3 * **L18:** System Prompt ~0.52, Maps ~0.48 * **L20:** System Prompt ~0.74, Maps ~0.26 * **L21:** System Prompt ~0.81, Maps ~0.19 * **L24:** System Prompt ~0.92, Maps ~0.08 * **L25:** System Prompt ~0.92, Maps ~0.08 * **L26:** System Prompt ~0.9, Maps ~0.1 * **L27:** System Prompt ~0.74, Maps ~0.26 **Trends:** * The "System Prompt" attention share generally increases from L0 to L2, then remains high with some fluctuations. * The "Maps" attention share is highest at L0 and L1, then decreases significantly and remains low for most of the later layers. ### Key Observations * Layers L2, L3, L14, L24, and L25 show the highest attention share for the "System Prompt." * Layer L0 shows the highest attention share for "Maps." * There is a significant shift in attention from "Maps" to "System Prompt" between layers L1 and L2. ### Interpretation The chart suggests that the model initially relies heavily on "Maps" (as seen in L0 and L1), but as the processing goes deeper into the network (from L2 onwards), the "System Prompt" becomes the dominant focus. This could indicate that the initial layers are involved in processing and understanding the map data, while later layers focus more on the instructions or context provided by the system prompt to generate the final output. The fluctuations in later layers might reflect specific interactions or dependencies between the prompt and the map data at different stages of processing.

## Stacked Bar Chart: Relative Attention Share by Model Layer ### Overview This is a stacked bar chart visualizing the relative attention share between "System Prompt" and "Maps" across different model layers (L0 to L27). The y-axis represents the "Relative Attention Share", ranging from 0.0 to 1.0, while the x-axis represents the "Model Layer". Each bar is divided into two sections, representing the attention share of each category. ### Components/Axes * **X-axis:** "Model Layer" with markers L0, L1, L2, L3, L14, L15, L16, L18, L20, L21, L24, L25, L26, L27. * **Y-axis:** "Relative Attention Share" ranging from 0.0 to 1.0. * **Legend:** Located in the top-right corner. * "System Prompt" - Light Gray * "Maps" - Blue ### Detailed Analysis The chart consists of 14 bars, each representing a specific model layer. The attention share is represented as a stacked bar, with the "System Prompt" at the bottom (light gray) and "Maps" on top (blue). Here's a breakdown of the approximate attention shares for each layer: * **L0:** System Prompt ~ 0.25, Maps ~ 0.75 * **L1:** System Prompt ~ 0.20, Maps ~ 0.80 * **L2:** System Prompt ~ 0.25, Maps ~ 0.75 * **L3:** System Prompt ~ 0.30, Maps ~ 0.70 * **L14:** System Prompt ~ 0.40, Maps ~ 0.60 * **L15:** System Prompt ~ 0.45, Maps ~ 0.55 * **L16:** System Prompt ~ 0.40, Maps ~ 0.60 * **L18:** System Prompt ~ 0.35, Maps ~ 0.65 * **L20:** System Prompt ~ 0.30, Maps ~ 0.70 * **L21:** System Prompt ~ 0.20, Maps ~ 0.80 * **L24:** System Prompt ~ 0.25, Maps ~ 0.75 * **L25:** System Prompt ~ 0.25, Maps ~ 0.75 * **L26:** System Prompt ~ 0.30, Maps ~ 0.70 * **L27:** System Prompt ~ 0.40, Maps ~ 0.60 **Trends:** * Initially, from L0 to L1, the attention share of "Maps" increases while that of "System Prompt" decreases. * From L1 to L3, the attention share of "Maps" decreases slightly while that of "System Prompt" increases. * From L3 to L14, the attention share of "System Prompt" increases significantly, while that of "Maps" decreases. * From L14 to L16, the attention share of "System Prompt" decreases slightly, while that of "Maps" increases. * From L16 to L27, the attention share of "System Prompt" fluctuates, while that of "Maps" remains relatively stable. ### Key Observations * The "Maps" category consistently receives a higher attention share than the "System Prompt" category across all layers. * The most significant shift in attention share occurs between layers L3 and L14, where the "System Prompt" attention share increases substantially. * The attention share of "Maps" is highest at L1 and L21, reaching approximately 0.80. * The attention share of "System Prompt" is highest at L14, reaching approximately 0.40. ### Interpretation The chart demonstrates how the model's attention is distributed between the "System Prompt" and "Maps" data sources across different layers. The initial high attention to "Maps" (layers L0-L3) suggests that the model initially relies heavily on the map data. The subsequent increase in attention to the "System Prompt" (around L14) indicates that the model begins to integrate the system instructions more effectively as it processes information through deeper layers. The relatively stable attention shares in the later layers (L16-L27) suggest a balance between utilizing map data and following system instructions. The fluctuations in attention share could be due to the complexity of the task or the specific information being processed at each layer. The chart provides insights into the model's internal reasoning process and how it leverages different data sources to perform its task.

\n ## Stacked Bar Chart: Relative Attention Share by Model Layer ### Overview This image is a stacked bar chart visualizing the distribution of "Relative Attention Share" between two categories, "System Prompt" and "Maps," across a series of discrete model layers. The chart aims to show how the allocation of attention changes at different depths within a model architecture. ### Components/Axes * **Chart Type:** Stacked Bar Chart. * **X-Axis (Horizontal):** Labeled "Model Layer". It displays categorical labels for specific layers: L0, L1, L2, L3, L14, L15, L16, L18, L20, L21, L24, L25, L26, L27. The layers are not presented in strict numerical sequence. * **Y-Axis (Vertical):** Labeled "Relative Attention Share". It is a linear scale ranging from 0.0 to 1.0, with major tick marks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **Legend:** Located in the bottom-right corner of the chart area. It defines the two data series: * A gray square labeled "System Prompt". * A blue square labeled "Maps". * **Data Series:** Each bar represents a model layer and is divided into two colored segments whose heights sum to 1.0 (the total relative attention share for that layer). ### Detailed Analysis The following table reconstructs the approximate data from the chart. Values are estimated based on visual alignment with the y-axis grid. The "System Prompt" (gray) segment is always the bottom portion of the bar. | Model Layer | System Prompt (Gray) Share (Approx.) | Maps (Blue) Share (Approx.) | Visual Trend Description | | :--- | :--- | :--- | :--- | | **L0** | 0.30 | 0.70 | Maps dominates; the blue segment is more than twice the height of the gray. | | **L1** | 0.56 | 0.44 | System Prompt becomes the majority share. | | **L2** | 0.95 | 0.05 | System Prompt overwhelmingly dominates; the Maps segment is a very thin sliver. | | **L3** | 0.92 | 0.08 | System Prompt remains dominant. | | **L14** | 0.86 | 0.14 | System Prompt share decreases slightly from L3 but remains high. | | **L15** | 0.68 | 0.32 | Notable increase in Maps share compared to previous layers. | | **L16** | 0.70 | 0.30 | Similar distribution to L15. | | **L18** | 0.52 | 0.48 | Nearly equal split; this is the only layer besides L0 where Maps approaches or exceeds half the share. | | **L20** | 0.74 | 0.26 | System Prompt share increases again. | | **L21** | 0.81 | 0.19 | System Prompt share continues to increase. | | **L24** | 0.92 | 0.08 | System Prompt share returns to a very high level, similar to L3. | | **L25** | 0.92 | 0.08 | Identical or near-identical distribution to L24. | | **L26** | 0.89 | 0.11 | Slight decrease in System Prompt share from L25. | | **L27** | 0.74 | 0.26 | System Prompt share decreases to a level similar to L20. | ### Key Observations 1. **Dominance of System Prompt:** In 12 out of the 14 layers shown, the "System Prompt" category holds the majority of the relative attention share (value > 0.5). 2. **Two Exceptional Layers:** Layer **L0** is the only layer where "Maps" has a clear majority (~0.70). Layer **L18** shows the most balanced distribution, with shares nearly equal (~0.52 vs. ~0.48). 3. **High-Attention Layers:** Layers **L2, L3, L24, and L25** show the highest concentration of attention on the System Prompt, with shares exceeding 0.90. 4. **Non-Sequential Layers:** The x-axis does not list layers in order (e.g., jumps from L3 to L14, includes L18 between L16 and L20). This suggests the chart may be highlighting specific layers of interest rather than a continuous sequence. 5. **Pattern of Fluctuation:** There is no simple linear trend. The System Prompt share spikes early (L2), dips in the middle layers (L15, L16, L18), rises again (L20-L25), and dips slightly at the end (L27). ### Interpretation This chart provides a diagnostic view into the internal attention mechanisms of a model, likely a large language model (LLM) given the "System Prompt" label. The data suggests that the model's attention is not uniformly distributed. * **Functional Specialization:** The stark contrast between L0 (Maps-heavy) and L2 (System Prompt-heavy) implies rapid specialization in the earliest layers. L0 may be processing raw input or spatial/metadata information ("Maps"), while subsequent layers quickly shift focus to the core instructions ("System Prompt"). * **Mid-Layer Re-engagement:** The increased share for "Maps" in layers L15, L16, and especially L18 indicates a potential phase where the model re-evaluates or integrates map-based information after having processed the system prompt extensively. This could be crucial for tasks requiring grounding in external data or spatial reasoning. * **Late-Layer Consensus:** The return to high System Prompt attention in layers L24-L25 suggests a final consolidation or application of the core instructions before output generation. * **Architectural Insight:** The non-sequential layer selection implies these specific layers are considered critical checkpoints for understanding how the model balances its foundational instructions against other input types. The visualization effectively argues that attention allocation is dynamic and layer-dependent, challenging a view of uniform processing depth. **Language Note:** All text in the image is in English.

## Bar Chart: Relative Attention Share by Model Layer ### Overview The chart visualizes the distribution of attention between two components ("System Prompt" and "Maps") across 28 model layers (L0 to L27). Each bar represents a layer, with stacked segments showing the relative contribution of each component to total attention. ### Components/Axes - **X-axis**: Model Layers (L0 to L27, labeled sequentially) - **Y-axis**: Relative Attention Share (0.0 to 1.0) - **Legend**: - Gray: System Prompt - Blue: Maps - **Bar Structure**: Stacked vertical bars (total height = 1.0 per layer) ### Detailed Analysis | Layer | System Prompt (Gray) | Maps (Blue) | Notes | |-------|----------------------|-------------|-------| | L0 | ~0.3 | ~0.7 | Maps dominate early layers | | L1 | ~0.4 | ~0.6 | Gradual shift toward System Prompt | | L2 | ~0.95 | ~0.05 | System Prompt peaks | | L3 | ~0.9 | ~0.1 | Sustained System Prompt focus | | L14 | ~0.85 | ~0.15 | Stable System Prompt dominance | | L15 | ~0.7 | ~0.3 | Maps regain partial attention | | L16 | ~0.7 | ~0.3 | Similar to L15 | | L18 | ~0.5 | ~0.5 | Balanced attention split | | L20 | ~0.75 | ~0.25 | System Prompt resurgence | | L21 | ~0.8 | ~0.2 | Continued System Prompt focus | | L24 | ~0.9 | ~0.1 | Near-peak System Prompt | | L25 | ~0.9 | ~0.1 | Matches L24 | | L26 | ~0.85 | ~0.15 | Slight Maps rebound | | L27 | ~0.75 | ~0.25 | Maps regain significant share | ### Key Observations 1. **Early Layers (L0-L3)**: Maps dominate attention (60-95% share), suggesting initial processing prioritizes mapping tasks. 2. **Mid-Layers (L14-L16)**: System Prompt maintains 70-85% share, indicating sustained focus on prompt-guided processing. 3. **Late Layers (L18-L27)**: Attention becomes more variable: - L18 shows equal split (50-50) - L24-L25 peak at 90% System Prompt - L27 returns to 75% System Prompt / 25% Maps 4. **Non-linear Pattern**: No consistent monotonic trend; attention shifts cyclically between components. ### Interpretation The data suggests a dynamic interplay between System Prompt and Maps attention across layers: - **Early Processing (L0-L3)**: Maps likely handle spatial/geometric computations, while System Prompt begins integrating contextual instructions. - **Mid-Processing (L14-L16)**: System Prompt dominates, possibly reflecting hierarchical decision-making or rule application. - **Late Processing (L18-L27)**: The cyclical pattern may indicate iterative refinement, where Maps re-emerge for spatial adjustments after System Prompt establishes logical frameworks. Notably, the 50-50 split at L18 could represent a critical decision point where both components equally influence outcomes. The final layers (L24-L27) show System Prompt maintaining >80% share despite Maps' resurgence, implying prompt-guided logic remains foundational even as spatial considerations re-emerge.