## Diagram: Comparison of Search Strategies for Question Answering ### Overview The image presents a comparison between two search strategies, "Search-R1" and "Agent-as-a-tool (Ours)", for answering the question: "Where was the mother of Ernst I, Prince Of Hohenlohe-Langenburg born?". It illustrates the step-by-step reasoning and search processes of each strategy, highlighting their differences and outcomes. ### Components/Axes The diagram is divided into two main columns: * **Left Column:** "Result from Search-R1" * **Right Column:** "Result from Agent-as-a-tool (Ours)" Each column contains a sequence of steps enclosed in XML-like tags, including: * `<think>`: Represents a thought process or reasoning step. * `<search>`: Represents a search query. * `<information>`: Represents the information retrieved from a search. * `<tool_calling>`: Represents a call to an external tool. * `<obs>`: Represents an observation or piece of information. * `<answer>`: Represents the final answer provided by the strategy. Additionally, there are annotations with orange boxes and arrows indicating: * "Fuzzier result to be processed" (pointing to the information block in Search-R1) * "Needs to find the answer from scratch (And get the wrong one)" (pointing to the second search block in Search-R1) * "Clearer result to be processed" (pointing to the observation block in Agent-as-a-tool) * "Reasoning with answer given by another agent" (pointing to the second tool_calling block in Agent-as-a-tool) ### Detailed Analysis or Content Details **Left Column: Result from Search-R1** 1. **`<think>`:** "I need to find out where the mother of Ernst I, Prince of Hohenlohe-Langenburg, was born. I'll search for it." 2. **`<search>`:** "mother of Ernst I, Prince of Hohenlohe-Langenburg" 3. **`<information>`:** * `result_id`: 1 * `title`: "Ernst I, Prince of Hohenlohe-Langenburg" * `link`: "https://en.wikipedia.org/wiki/Ernst\_I\_Prince\_of\_Hohenlohe-Langenburg" * `snippet`: "He married Princess Feodora of Leiningen, the only daughter of Emich Carl, 2nd Prince of Leiningen, and Princess Victoria of Saxe-Coburg-Saalfeld on 18\xa0},... (save space for 9 more results)" 4. **`<think>`:** "I found out that the mother of Ernst I, Prince of Hohenlohe-Langenburg, was Princess Feodora of Leiningen. Now I need to find out where she was born." 5. **`<search>`:** "where was Princess Feodora of Leiningen born" 6. **`<information>`:** * `result_id`: 1 * `title`: "Princess Feodora of Leiningen" * `link`: "https://en.wikipedia.org/wiki/Princess\_Feodora\_of\_Leiningen" * `snippet`: "Life. Feodora was born in Amorbach, Bavaria, on 7 December 1807 to Princess Victoria of Saxe-Coburg-Saalfeld and her first husband, Emich Carl, Prince of\xa0..." * `result_id`: 2 * `title`: "Princess Feodora of Leiningen: Queen Victoria's Half Sister" * `link`: "https://owlcation.com/humanities/Feodora-of-Leiningen-Queen-Victorias-Half-Sister" * `snippet`: "Feodora was born Princess Anna Feodora Auguste Charlotte Wilhemina of Leiningen on the 7th of December 1807 in Amorbach, then in Leiningen and\xa0..." 7. **`<think>`:** "I found out that Princess Feodora of Leiningen was born in Amorbach, Bavaria, Germany. Now I can provide the answer." 8. **`<answer>`:** "Amorbach (Wrong Answer)" **Right Column: Result from Agent-as-a-tool (Ours)** 1. **`<think>`:** "I need to find out who the mother of Ernst I, Prince of Hohenlohe-Langenburg is first." 2. **`<tool_calling>`:** "Who is the mother of Ernst I, Prince of Hohenlohe-Langenburg?" 3. **`<obs>`:** "The mother of Ernst I, Prince of Hohenlohe-Langenburg, is Countess Amalie Henriette of Solms-Baruth. For more information, you can visit the [Wikipedia page on Ernst I, Prince of Hohenlohe-Langenburg](https://en.wikipedia.org/wiki/Ernst\_I,\_Prince\_of\_Hohenlohe-Langenburg)." 4. **`<think>`:** "Now that I know the mother's name, I need to find out where she was born." 5. **`<tool_calling>`:** "Where was Countess Amalie Henriette of Solms-Baruth born?" 6. **`<obs>`:** "Countess Amalie Henriette of Solms-Baruth was born in Kliczków on January 30, 1768." 7. **`<think>`:** "The information provided states that Countess Amalie Henriette of Solms-Baruth, the mother of Ernst I, Prince of Hohenlohe-Langenburg, was born in Kliczków." 8. **`<answer>`:** "Kliczków (Correct Answer)" ### Key Observations * Search-R1 directly searches for the mother of Ernst I's birthplace, retrieves information about Ernst I, then incorrectly focuses on Princess Feodora of Leiningen, leading to the wrong answer (Amorbach). * Agent-as-a-tool first identifies the mother's name (Countess Amalie Henriette of Solms-Baruth) and then searches for her birthplace, leading to the correct answer (Kliczków). * The annotations highlight that Search-R1's initial search result was "fuzzier" and required it to "find the answer from scratch," ultimately leading to an incorrect path. Agent-as-a-tool, on the other hand, obtained a "clearer result" and used "reasoning with answer given by another agent" to arrive at the correct answer. ### Interpretation The comparison demonstrates the effectiveness of a multi-step reasoning approach (Agent-as-a-tool) compared to a direct search approach (Search-R1) for complex question answering. Agent-as-a-tool's ability to break down the question into sub-questions (identifying the mother first, then her birthplace) allows it to leverage information more effectively and avoid getting sidetracked by irrelevant details. Search-R1's approach, while simpler, is more prone to errors due to its reliance on the initial search results and its inability to correct its course when it goes down the wrong path. The annotations emphasize the importance of clear initial information and the benefits of reasoning and iterative refinement in question answering systems.