This image is a comparative analysis table illustrating the performance of two different AI agent architectures—"Agent-as-tool-Base" and "Agent-as-tool-Instruct"—in answering a specific biographical query. The image uses a structured log format (resembling XML tags) to show the reasoning process of each agent, supplemented by orange text boxes that provide meta-commentary on the agents' decision-making strategies. ## Table: AI Agent Performance Comparison ### Overview The image presents a side-by-side comparison of two AI agents attempting to answer the question: *"Where did Edward Dunn (Bishop)'s father graduate from?"* The expected answer is "Corpus Christi College, Cambridge." The left column shows the "Base" agent failing to retrieve the correct information, while the right column shows the "Instruct" agent successfully navigating the query through sequential reasoning. ### Components/Axes * **Header:** Contains the target question and the ground-truth answer. * **Left Column (Agent-as-tool-Base):** Documents the failure path of the base model. * **Right Column (Agent-as-tool-Instruct):** Documents the success path of the instructed model. * **Annotations:** Orange text boxes overlaid on the content provide qualitative assessments of the agents' actions (e.g., "Planner try to ask 2 hops simultaneously"). * **Syntax:** The content uses pseudo-code tags: `<think>` (internal reasoning), `<tool_calling>` (external search queries), `<obs>` (search results/observations), and `<answer>` (final output). ### Detailed Analysis #### Header * **Question:** Where did Edward Dunn (Bishop)'s father graduate from? * **Expected Answer:** Corpus Christi College, Cambridge #### Left Column: Result from Agent-as-tool-Base 1. **Initial Step:** The agent attempts to find the father's education. * *Annotation:* "Planner try to ask 2 hops simultaneously" 2. **Observation:** The agent fails to find specific info on the father, finding info on Edward Dunn himself instead. * *Annotation:* "The agent was unable to answer" 3. **Second Step:** The agent tries a different search query. * *Annotation:* "Planner asked 2 hops in the meantime in another manner" 4. **Observation:** The agent retrieves a list of findings, including a mention of "Marlborough and Pembroke College, Cambridge" for Andrew Hunter Dunn. * *Annotation:* "The agent cannot get exact information" 5. **Final Answer:** The agent concludes with "Marlborough and Pembroke College, Cambridge (Wrong Answer)." #### Right Column: Result from Agent-as-tool-Instruct 1. **Initial Step:** The agent identifies the need to find information about the father. * *Annotation:* "Learned to Ask the prerequisite question before the next hop" 2. **Observation:** The agent identifies the father as Andrew Hunter Dunn, Bishop of Quebec. * *Annotation:* "The agent could answer this question easily" 3. **Second Step:** The agent recognizes the missing information (where the father graduated) and formulates a specific follow-up query. * *Annotation:* "Asked the next-hop question" 4. **Observation:** The agent successfully retrieves the correct data: "Andrew Hunter Dunn graduated from **Corpus Christi College, Cambridge**, where he earned his BA as the 29th Wrangler in 1863." * *Annotation:* "The agent was able to answer the question easily" 5. **Final Answer:** The agent concludes with "Corpus Christi College, Cambridge (Correct Answer)." ### Key Observations * **Strategy Divergence:** The "Base" agent attempts to solve the problem in a non-linear or overly complex manner ("2 hops simultaneously"), leading to confusion and incorrect data retrieval. * **Sequential Logic:** The "Instruct" agent demonstrates superior performance by breaking the task into a prerequisite step (identifying the father) followed by a specific follow-up step (identifying the graduation college). * **Hallucination/Error:** The "Base" agent provides an incorrect answer ("Marlborough and Pembroke College"), suggesting that when an agent lacks a structured reasoning path, it is more prone to retrieving or synthesizing incorrect information. ### Interpretation This image serves as a technical demonstration of **Chain-of-Thought (CoT) prompting** or **Instruction Tuning**. It highlights that the architecture of an AI agent—specifically its ability to decompose complex queries into sequential, manageable sub-tasks—is the primary determinant of accuracy. The "Base" agent fails because it lacks the "instruction" to prioritize logical sequencing, causing it to conflate information or fail to isolate the specific variable required. The "Instruct" agent succeeds because it mimics a human research process: verify the subject first, then verify the specific attribute. This visual comparison is likely used to advocate for training methodologies that enforce step-by-step reasoning in AI agents.