## Bar Chart: Indexical 'tomorrow' Performance
### Overview
The image is a faceted bar chart comparing the performance of four Large Language Models (LLMs) on the task of interpreting the indexical term "tomorrow." The chart is organized into a 2x2 grid, with each quadrant representing a specific model: Claude 3.5 Sonnet, Deepseek V3, Gemini 1.5 pro, and GPT-4o. The data is categorized by "Sentence Type" (Non-quoted vs. Quoted) on the x-axis, with a numerical performance metric (likely accuracy or success rate) on the y-axis.
### Components/Axes
* **Title:** "Indexical 'tomorrow'" (Top-left of the entire figure).
* **Y-Axis:** A continuous scale ranging from 0.00 to 1.00, representing the performance metric.
* **X-Axis:** Categorical labels "Non-quoted" and "Quoted" at the bottom of the chart.
* **Facets (Models):**
* **Top-Left:** Claude 3.5 Sonnet
* **Top-Right:** Deepseek V3
* **Bottom-Left:** Gemini 1.5 pro
* **Bottom-Right:** GPT-4o
* **Data Series:** Each category contains two bars:
* **Light Blue:** Left bar in each pair.
* **Dark Blue:** Right bar in each pair.
* **Annotations:** Numerical values are placed directly above each bar to indicate the exact data point. Small black error bars are visible atop each bar.
### Detailed Analysis
The data shows a consistent pattern across all models: high performance on "Non-quoted" sentences and a sharp decline in performance on "Quoted" sentences.
| Model | Category | Light Blue Bar | Dark Blue Bar |
| :--- | :--- | :--- | :--- |
| **Claude 3.5 Sonnet** | Non-quoted | 1 | 1 |
| **Claude 3.5 Sonnet** | Quoted | 0 | 0 |
| **Deepseek V3** | Non-quoted | 1 | 0.99 |
| **Deepseek V3** | Quoted | 0.16 | 0.19 |
| **Gemini 1.5 pro** | Non-quoted | 0.99 | 1 |
| **Gemini 1.5 pro** | Quoted | 0 | 0 |
| **GPT-4o** | Non-quoted | 1 | 1 |
| **GPT-4o** | Quoted | 0.01 | 0.11 |
**Trend Verification:**
* **Non-quoted:** All models show a flat, high-performance trend, with values consistently at or near 1.00.
* **Quoted:** All models show a dramatic downward trend, dropping to near-zero performance.
### Key Observations
* **Near-Perfect Baseline:** All four models demonstrate near-perfect accuracy (0.99–1.00) when processing "Non-quoted" sentences.
* **The "Quoted" Failure:** There is a catastrophic drop in performance for all models when the sentence type is "Quoted."
* **Deepseek V3 Outlier:** Deepseek V3 is the only model that maintains a non-zero performance level in the "Quoted" category (0.16 and 0.19). While still low, it is significantly higher than the other models, which effectively drop to 0 (or near 0 in the case of GPT-4o).
* **Consistency:** The two bars (Light Blue and Dark Blue) are nearly identical in value for every model, suggesting high consistency between the two test runs or conditions represented by the colors.
### Interpretation
This data demonstrates a specific failure mode in LLM reasoning regarding indexicality (words like "tomorrow" that depend on context).
* **Contextual Sensitivity:** The models are highly capable of resolving "tomorrow" when it is used in a standard, direct context ("Non-quoted").
* **Quotation Blindness:** When "tomorrow" is placed inside quotes, the models struggle significantly. This suggests the models may be over-relying on the literal token "tomorrow" to trigger a date-resolution process, or they are failing to distinguish between the model's own temporal context and the temporal context embedded within a quoted string.
* **Deepseek V3's Resilience:** The fact that Deepseek V3 retains some performance (0.16–0.19) on "Quoted" sentences suggests it may have a slightly different architectural approach or training bias that allows it to occasionally correctly identify that the indexical inside the quote should not be resolved relative to the model's current date, or perhaps it is simply guessing more effectively than the others. The other models (Claude, Gemini, GPT-4o) appear to fail completely in this specific linguistic scenario.
