## Diagram: Comparison of LLM Tool Calling Workflows ### Overview The image presents a comparative diagram illustrating three distinct methodologies for how Large Language Models (LLMs) utilize external tools to answer questions. It uses a specific example question—"Invincible is based on the story of which Philadelphia Eagles player?"—to demonstrate the evolution from simple, linear tool calling to complex, iterative, agent-based interactions. ### Components/Axes The diagram is divided into three horizontal sections, labeled (a), (b), and (c), each representing a different architectural approach. **Color-Coded Legend (Inferred from diagram elements):** * **Light Orange:** `<tool_calling>` (Action of invoking a tool) * **Light Yellow:** `<raw_obs>` (Unprocessed data returned from tools) * **Light Blue:** `<think>` (LLM reasoning/processing step) * **Light Red/Pink:** `<answer>` (Final output generation) * **Light Green:** `<processed_obs>` (Refined/summarized data) * **Light Grey:** "Agent-Toolcaller" and "Tools" (Intermediate processing components) ### Detailed Analysis #### Top Header * **Question:** "Invincible is based on the story of which Philadelphia Eagles player?" * *Note:* This serves as the shared input for all three workflows. #### (a) Vanila Tool Calling+LLM Reasoning * **Flow:** `<tool_calling>` $\rightarrow$ `<raw_obs>` $\rightarrow$ `<think>` $\rightarrow$ `<answer>` * **Description:** This is a linear, single-pass process. The LLM invokes a tool, receives raw observations, performs reasoning, and generates an answer. * **Text:** "The tool was called to process the original question (<tool_calling>), and then the unprocessed observations (<raw_obs>) were obtained, then LLM think with the given observations (<think>) to give the answer (<answer>)" #### (b) Multi-step Tool Calling with unprocessed results * **Flow:** `<think>` $\rightarrow$ `<tool_calling>` $\rightarrow$ `<raw_obs>` $\rightarrow$ `<answer>` * **Loop:** An arrow returns from `<answer>` back to the start of the `<think>` block, labeled "N-times". * **Description:** This is an iterative process. The LLM thinks about the subquery, calls a tool, receives raw observations, and attempts an answer. This cycle repeats N times. * **Text:** "LLM think about where to start and how to answer the question (<think>), then calls tools to process the subquery (<tool_calling>), after obtaining the unprocessed observations (<raw_obs>), the LLM then think again. After multiple iterations, the final answer was reached (<answer>). The process could be finetuned by reinforcement learning." #### (c) Agent-as-tool (ours) * **Flow:** `<think>` $\rightarrow$ `<tool_calling>` $\rightarrow$ [**Agent-Toolcaller** $\rightarrow$ Interact with $\rightarrow$ **Tools**] $\rightarrow$ `<processed_obs>` $\rightarrow$ `<answer>` * **Loop:** An arrow returns from `<processed_obs>` back to the start of the `<think>` block, labeled "N-times". * **Description:** This approach introduces an intermediate "Agent-Toolcaller" layer. The LLM calls this agent, which interacts with tools to produce *processed* observations, which are then fed back to the LLM. * **Text:** "LLM think about where to start and how to answer the question (<think>), then calls the agent (Toolcaller) to process the subquery (<tool_calling>), the agent use tools (Tools) to process the subqueries for one or more times and then generate the processed results based on the interaction with tools (<processed_obs>). After multiple iterations, the final answer was reached (<answer>)." ### Key Observations * **Evolution of Complexity:** The diagram shows a clear progression from simple linear execution (a) to iterative execution (b), and finally to agent-mediated execution (c). * **Data Quality:** A critical distinction is the transition from `<raw_obs>` in (a) and (b) to `<processed_obs>` in (c). This implies that the "Agent-as-tool" method is designed to handle noisy or unstructured data by refining it before the LLM performs its reasoning. * **Iterative Loops:** Both (b) and (c) utilize an "N-times" loop, indicating that complex queries require multiple turns of reasoning and tool interaction, rather than a single "one-shot" attempt. ### Interpretation The data demonstrates the architectural shift in modern AI agent design. * **The Problem:** Methods (a) and (b) rely on the LLM to interpret "raw" data directly from tools. This can be problematic if the tool output is verbose, unstructured, or contains irrelevant information, which increases the cognitive load on the LLM and increases the likelihood of hallucination or error. * **The Solution:** Method (c), labeled "ours," introduces an abstraction layer (the "Agent-Toolcaller"). By having a specialized agent interact with tools and return "processed" observations, the system effectively filters and summarizes information before the main LLM processes it. * **Strategic Advantage:** This architecture likely improves performance on complex, multi-step reasoning tasks by ensuring the LLM receives high-quality, relevant context rather than raw, potentially overwhelming data. The inclusion of reinforcement learning in (b) and the agentic structure in (c) suggests a focus on optimizing the *process* of reasoning, not just the final output.