## Dialogue Comparison: Agent-as-tool-Base vs. Agent-as-tool-Instruct
### Overview
The image presents a comparison of two agents, "Agent-as-tool-Base" and "Agent-as-tool-Instruct", attempting to answer the question: "Where did Edward Dunn (Bishop)'s father graduate from?". The expected answer is "Corpus Christi College, Cambridge". The image shows the step-by-step reasoning and actions of each agent, including their thoughts, tool calls, observations, and final answers.
### Components/Axes
The image is divided into two main columns, one for each agent. Each column contains a sequence of dialogue turns, marked by tags such as `<think>`, `<tool_calling>`, `<obs>`, and `<answer>`. Orange boxes highlight key actions or conclusions of each agent.
### Detailed Analysis or ### Content Details
**Agent-as-tool-Base (Left Column):**
1. **Initial Thought:** The agent starts by stating its need to find Edward Dunn's father's educational background.
2. **Tool Call:** The agent calls a tool to provide information on Edward Dunn's father's education.
3. **Observation:** The agent reports that it couldn't find specific information about Edward Dunn's father's education but found information related to Edward Dunn himself. It asks if the user needs information on a different topic. An orange box states "Planner try to ask 2 hops simultaneously".
4. **Thought:** The agent concludes that the observation did not provide the specific information it was looking for and that it needs to find a more direct way to get the information. An orange box states "The agent was unable to answer".
5. **Tool Call:** The agent calls a tool to provide information on the educational background of Edward Dunn's father.
6. **Observation:** The agent finds some relevant information regarding Edward Dunn's family background, but specific details about his father's educational background are limited. It provides some notable findings:
* Edward T. Dunn studied theology and earned a master's degree in history and a PhD, but this relates to him rather than his father.
* Bishop Edward Dunn's father, Andrew Hunter Dunn, was an education graduate from Marlborough and Pembroke College, Cambridge.
* Other sources mainly discuss different individuals named Edward Dunn or their obituaries.
7. **Thought:** The agent concludes that it cannot get exact information. An orange box states "The agent cannot get exact information".
8. **Thought:** The agent states that the observation provided the specific information it needed: Edward Dunn's father, Andrew Hunter Dunn, graduated from Marlborough and Pembroke College, Cambridge.
9. **Answer:** The agent provides the answer "Marlborough and Pembroke College, Cambridge (Wrong Answer)".
**Agent-as-tool-Instruct (Right Column):**
1. **Initial Thought:** The agent starts by stating its need to find information about Edward Dunn's father.
2. **Tool Call:** The agent calls a tool to provide information about Edward Dunn (Bishop)'s father.
3. **Observation:** The agent states that Edward Dunn, the bishop, was born into an ecclesiastical family. His father was Andrew Hunter Dunn, who at one time served as the Bishop of Quebec. It provides a link to Edward Dunn's Wikipedia page. An orange box states "Learned to Ask the prerequisite question before the next hop".
4. **Thought:** The agent concludes that it could answer this question easily. An orange box states "The agent could answer this question easily". From the observation, it sees that Edward Dunn's father was Andrew Hunter Dunn, who was the Bishop of Quebec. However, the observation does not specify where Andrew Hunter Dunn graduated from. It needs to find this information.
5. **Tool Call:** The agent calls a tool to provide information about where Andrew Hunter Dunn graduated from.
6. **Observation:** The agent states that Andrew Hunter Dunn graduated from Corpus Christi College, Cambridge, where he earned his BA as the 29th Wrangler in 1863. It provides a link to Andrew Hunter Dunn's Wikipedia page. An orange box states "Asked the next-hop question".
7. **Thought:** The agent concludes that it was able to answer the question easily. An orange box states "The agent was able to answer the question easily". Now it has the information that Andrew Hunter Dunn, the father of Edward Dunn (Bishop), graduated from Corpus Christi College, Cambridge.
8. **Answer:** The agent provides the answer "Corpus Christi College, Cambridge (Correct Answer)".
### Key Observations
* Agent-as-tool-Base initially struggles to find the correct information and provides an incorrect answer. It attempts to ask 2 hops simultaneously.
* Agent-as-tool-Instruct successfully answers the question by breaking it down into smaller steps and asking the prerequisite question before the next hop.
### Interpretation
The comparison highlights the difference in performance between the two agents. Agent-as-tool-Instruct demonstrates a more effective reasoning process by breaking down the question into smaller, more manageable steps. This allows it to gather the necessary information and provide the correct answer. Agent-as-tool-Base, on the other hand, struggles to find the correct information and ultimately provides an incorrect answer, suggesting a less effective reasoning strategy. The orange boxes highlight the key differences in the agents' approaches, with Agent-as-tool-Instruct learning to ask the prerequisite question and Agent-as-tool-Base attempting to ask 2 hops simultaneously.
