## Diagram: LLM Dependency Graph Analysis with Self-Correction Attempts
### Overview
The diagram compares true and predicted dependency graphs in a game context (likely Minecraft), illustrating an LLM's ability to identify and correct errors in dependency relationships. It includes four panels: (a) True Dependency Graph, (b) LLM-predicted Graph, (c) LLM self-correction for dependencies, and (d) LLM self-correction for actions. Speech bubbles show iterative reasoning, while a legend on the left defines symbolic elements.
---
### Components/Axes
#### Legend (Left Panel)
- **Correct dependency**: Solid black arrow
- **Missed dependency**: Dashed orange arrow
- **Redundant dependency**: Dashed red arrow
- **Hallucinated item**: Red bug icon
- **Wrong knowledge**: Red rectangle
- **Ground-truth**: Blue rectangle
#### Panels
1. **(a) True Dependency Graph**
- Central block (gray) with arrows to:
- Missed dependency (wooden crate, dashed orange)
- Redundant dependency (stone block, dashed red)
- Correct dependencies (diamond ore, pickaxe, etc., solid black)
- Hallucinated item (red bug) connected to stone block.
2. **(b) LLM-predicted Graph**
- Similar structure to (a) but with:
- Incorrect redundant dependency (stone block, dashed red)
- Missed correct dependency (diamond ore, dashed orange)
- Hallucinated item (red bug) connected to stone block.
3. **(c) LLM Self-Correction for Dependencies**
- **Prior attempt**: "requires" speech bubble with stone block and wooden crate.
- **Correction prompt**: "You failed to get [stone block] many times. You had [stone block] at those times."
- **Next attempt**: "I still think [stone block] requires [wooden crate]."
- Outcome: "Fail!" in red.
4. **(d) LLM Self-Correction for Actions**
- **Prior attempt**: "I will do 'mine' [stone block]."
- **Correction prompt**: "You failed to 'mine' [stone block] many times. You had [stone block] at those times."
- **Next attempt**: "I will 'mine' [stone block] again. I failed since I had no [stone block] and [diamond ore]."
- Outcome: "Fail!" in red.
---
### Detailed Analysis
#### Panel (a) vs. (b): True vs. Predicted Graphs
- **True Graph**: Accurately maps dependencies (e.g., stone block requires pickaxe).
- **Predicted Graph**: Introduces errors:
- Missed correct dependency (diamond ore → stone block).
- Added redundant dependency (stone block → wooden crate).
- Hallucinated red bug connected to stone block.
#### Panels (c) and (d): Self-Correction Process
- **Dependencies (c)**:
- LLM identifies missed dependency but fails to resolve it, persisting in incorrect belief.
- Speech bubbles show iterative reasoning but no resolution.
- **Actions (d)**:
- LLM attempts to "mine" stone block but fails due to missing prerequisites (diamond ore).
- Correction prompt highlights missing items, but next attempt still fails due to unresolved dependencies.
---
### Key Observations
1. **Persistent Errors**: The LLM struggles to resolve missed dependencies (e.g., diamond ore requirement).
2. **Hallucinations**: Red bug icon appears in both true and predicted graphs, suggesting a recurring error.
3. **Redundant Dependencies**: LLM incorrectly adds stone block → wooden crate in (b), which is absent in (a).
4. **Action Failures**: Self-correction for actions (d) fails due to incomplete prerequisite resolution.
---
### Interpretation
The diagram demonstrates the challenges of LLM-based error correction in dependency resolution. While the model identifies some errors (e.g., missed dependencies), it fails to fully resolve them, leading to persistent hallucinations and incorrect actions. The iterative self-correction process highlights the need for improved grounding in prerequisite relationships. The "Fail!" outcomes underscore limitations in current LLM architectures for complex, interdependent systems like game mechanics.
