## Diagram: LLM-Based Question Answering System with Evidence-Gap Tracking
### Overview
The image is a technical flowchart illustrating a system architecture for answering user queries using a Large Language Model (LLM). The system incorporates a memory retrieval mechanism, an evidence-gap tracker to identify missing information, and an iterative generation process with a router to manage resources. The specific example query concerns calculating the time between two pet adoption events.
### Components/Axes
The diagram is organized into three primary vertical sections, connected by directional arrows indicating data flow.
**1. Input Data (Left Section)**
* **User Query q:** A text box containing the question: "How many months passed between Andrew adopting Toby and Buddy?"
* **Memory M:** A container with two subsections:
* **Chunk-based:** Represented by a stack of horizontal bars.
* **Graph-based:** Represented by a network diagram of nodes and edges.
**2. LLM Generation (Central Section)**
* **Evidence-gap Tracker:** A box at the top containing:
* **Evidence:** "Andrew adopted Toby on July 11, 2023, and Buddy was named on October 19, 2023."
* **Gap:** "It lacks the specific adoption date for Buddy."
* **Generated Actions:** A list of three possible actions, each with an icon:
* **Retrieve:** new query Δq (Magnifying glass icon)
* **Reflect:** reasoning r (Brain icon)
* **Answer:** draft answer w (Pencil icon)
* **Router:** A box at the bottom managing the process, containing:
* Iteration budget
* Reflect-streak capacity
* Retrieval opportunity check
**3. Final Answer (Right Section)**
* A green box containing the final output: "Answer: the total number of full months that passed between Andrew adopting Toby and Buddy is **3 months**."
**Flow and Control Elements:**
* **Start/End:** Green rounded rectangles labeled "Start" and "End".
* **Process Nodes:** Dark blue rounded rectangles labeled "Retrieve", "Reflect", "Answer".
* **Arrows:** Solid and dashed lines showing the flow of data and control between components. A key dashed line shows the updated query `q^ret = q ⊕ Δq` feeding back into the "Retrieve" step.
### Detailed Analysis
The diagram depicts a cyclical, iterative process:
1. The system begins with a **User Query** and accesses **Memory** (both chunk-based and graph-based).
2. It enters the **LLM Generation** module, where the **Evidence-gap Tracker** analyzes retrieved evidence against the query. In this case, it finds a gap: the adoption date for "Buddy" is missing (only a "named on" date is provided).
3. Based on this gap, the system generates potential **Actions** (Retrieve, Reflect, Answer).
4. The **Router** evaluates the state (iteration budget, reflect streak, retrieval opportunity) and directs the flow to one of the process nodes ("Retrieve", "Reflect", or "Answer").
5. The process loops back to the "Retrieve" step with an updated query (`q^ret`) that incorporates the need to find the missing evidence (`Δq`).
6. Once sufficient evidence is gathered and processed, the flow proceeds to the "Answer" node and then to the **Final Answer**.
### Key Observations
* **Evidence Gap Identification:** The system explicitly identifies missing information ("specific adoption date for Buddy") rather than proceeding with incomplete data.
* **Iterative Refinement:** The architecture is designed for multiple passes, using a router to decide whether to gather more evidence, reason further, or attempt an answer.
* **Resource Management:** The Router component includes explicit constraints ("Iteration budget", "Reflect-streak capacity") to control computational resources and prevent infinite loops.
* **Query Augmentation:** The feedback loop shows the original query being augmented (`q ⊕ Δq`) to specifically target the identified information gap.
### Interpretation
This diagram illustrates a sophisticated approach to question-answering that moves beyond simple retrieval. It models a system that:
1. **Self-Monitors Comprehension:** The Evidence-gap Tracker acts as a metacognitive component, assessing its own knowledge state relative to the query.
2. **Makes Strategic Decisions:** The Router functions as a controller, choosing actions based on both the information gap and operational constraints (budget, streaks).
3. **Learns from Failure:** The process is inherently iterative; an initial inability to answer (due to a gap) triggers a targeted retrieval action, making the system more robust.
4. **Prioritizes Accuracy over Speed:** By incorporating reflection and verification steps, the architecture suggests a design goal of producing reliable, evidence-backed answers, even if it requires more processing steps.
The specific example demonstrates the system's logic: it finds partial evidence (Toby's adoption date, Buddy's naming date) but correctly flags the missing Buddy adoption date as a critical gap before concluding it cannot provide a precise answer. The final output of "3 months" implies that in a successful run, the system would have retrieved the missing date (likely close to the naming date) to perform the calculation.
