## Bar Chart: Tool Call Ratio vs. Accuracy After Fine-tuning ### Overview This bar chart compares the Tool Call Ratio and Accuracy of four different search methods (Base Generator, Google Search, Web Search, and Wikipedia Search) before and after fine-tuning. The chart uses bar graphs to represent the Tool Call Ratio for each method at "Step 0" (before fine-tuning) and "Step 32" (after fine-tuning). Accuracy is displayed as text boxes above the bar graphs, with the percentage increase after fine-tuning indicated in parentheses. A gray arrow indicates the transition from Step 0 to Step 32. ### Components/Axes * **X-axis:** Step (0 and 32) * **Y-axis:** Tool Call Ratio (%) - Scale ranges from 0 to 60. * **Legend:** Located at the top of the chart, horizontally aligned. * Red: Base Generator * Green: Google Search * Blue: Web Search * Purple: Wikipedia Search * **Accuracy Labels:** Two text boxes displaying accuracy percentages. * "Acc: 19.2%" above Step 0 bars. * "Acc: 25.2% (+6.21%)" above Step 32 bars. * **Title:** "After Fine-tuning" positioned centrally between the two sets of bars. * **Arrow:** A gray arrow pointing from "Step 0" to "Step 32", indicating the progression of fine-tuning. ### Detailed Analysis **Step 0 (Before Fine-tuning):** * **Base Generator (Red):** Tool Call Ratio is approximately 3.1%. * **Google Search (Green):** Tool Call Ratio is approximately 38.7%. This is the highest ratio at Step 0. * **Web Search (Blue):** Tool Call Ratio is approximately 18.4%. * **Wikipedia Search (Purple):** Tool Call Ratio is approximately 38.5%. **Step 32 (After Fine-tuning):** * **Base Generator (Red):** Tool Call Ratio is approximately 0.9%. A decrease of -2.2% from Step 0. * **Google Search (Green):** Tool Call Ratio is approximately 13.6%. A decrease of -1.5% from Step 0. * **Web Search (Blue):** Tool Call Ratio is approximately 13.6%. An increase of +5.2% from Step 0. * **Wikipedia Search (Purple):** Tool Call Ratio is approximately 13.6%. A decrease of -4.7% from Step 0. **Accuracy:** * Overall Accuracy at Step 0: 19.2% * Overall Accuracy at Step 32: 25.2% * Accuracy Increase: +6.21% ### Key Observations * Google Search and Wikipedia Search have the highest Tool Call Ratios at Step 0, both around 38-39%. * After fine-tuning (Step 32), the Tool Call Ratio for all methods decreased except for Web Search, which increased significantly. * The Base Generator consistently has the lowest Tool Call Ratio in both steps. * The overall accuracy increased by 6.21% after fine-tuning. * The largest change in Tool Call Ratio is observed for the Base Generator (-2.2%) and Web Search (+5.2%). ### Interpretation The data suggests that fine-tuning improves the overall accuracy of the system. However, the impact on the Tool Call Ratio varies significantly depending on the search method. The decrease in Tool Call Ratio for Google Search, Wikipedia Search, and Base Generator could indicate that the fine-tuning process has refined the system to rely less on these tools, potentially because they were being overused or providing less relevant information. The increase in Tool Call Ratio for Web Search suggests that fine-tuning has identified Web Search as a more valuable tool for the task at hand. The fact that the Base Generator consistently has the lowest Tool Call Ratio suggests it may be less effective or less frequently needed compared to the other search methods. The substantial accuracy increase (+6.21%) despite the varying Tool Call Ratio changes indicates that the fine-tuning process has optimized the system's overall performance, potentially by improving the quality of results from the selected tools. The negative changes in Tool Call Ratio for Google Search and Wikipedia Search could be a result of the fine-tuning process identifying that these tools were sometimes leading to incorrect or irrelevant information, and therefore reducing their usage.