## Bar Chart: MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines ### Overview The image is a bar chart comparing the Hit@1 scores of different models on the MetaQA dataset. The x-axis represents the number of hops (1-hop, 2-hop, 3-hop), and the y-axis represents the Hit@1 score. The chart compares "Our method" with three baseline models: LLM+KG, LLM+QD, and LLM. Error bars indicate the standard deviation. ### Components/Axes * **Title:** MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines * **X-axis:** MetaQA Dataset, with categories 1-hop, 2-hop, and 3-hop. * **Y-axis:** Hit@1 Score, ranging from 0.0 to 1.0 in increments of 0.2. * **Legend (Top-Right):** * Red: Our method * Orange: LLM+KG * Magenta: LLM+QD * Purple: LLM ### Detailed Analysis **1-hop:** * **Our method (Red):** Hit@1 score is approximately 0.92, with a small standard deviation. * **LLM+KG (Orange):** Hit@1 score is approximately 0.95, with a small standard deviation. * **LLM+QD (Magenta):** Hit@1 score is approximately 0.45, with a small standard deviation. * **LLM (Purple):** Hit@1 score is approximately 0.46, with a small standard deviation. **2-hop:** * **Our method (Red):** Hit@1 score is approximately 0.78, with a small standard deviation. * **LLM+KG (Orange):** Hit@1 score is approximately 0.58, with a small standard deviation. * **LLM+QD (Magenta):** Hit@1 score is approximately 0.37, with a small standard deviation. * **LLM (Purple):** Hit@1 score is approximately 0.38, with a small standard deviation. **3-hop:** * **Our method (Red):** Hit@1 score is approximately 0.63, with a small standard deviation. * **LLM+KG (Orange):** Hit@1 score is approximately 0.50, with a small standard deviation. * **LLM+QD (Magenta):** Hit@1 score is approximately 0.54, with a small standard deviation. * **LLM (Purple):** Hit@1 score is approximately 0.52, with a small standard deviation. ### Key Observations * "Our method" consistently outperforms the baseline models (LLM+QD and LLM) across all hop counts. * LLM+KG performs comparably to "Our method" for 1-hop questions, but its performance decreases more rapidly as the number of hops increases. * The performance of all models generally decreases as the number of hops increases, indicating that answering multi-hop questions is more challenging. * The standard deviations are relatively small, suggesting that the results are consistent. ### Interpretation The chart demonstrates that "Our method" is effective in answering questions from the MetaQA dataset, particularly for multi-hop questions. The addition of Knowledge Graph information (LLM+KG) improves performance for simple questions (1-hop), but its effectiveness diminishes for more complex questions. The LLM+QD and LLM baselines perform significantly worse, indicating the importance of the techniques used in "Our method" for reasoning and knowledge integration. The decreasing performance with increasing hops highlights the challenge of multi-hop reasoning in question answering.

## Bar Chart: MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines ### Overview This bar chart displays the Hit@1 scores, presented as mean values with standard deviation error bars, for four different models across three "hop" categories of the MetaQA dataset. The models are "Our method", "LLM+KG", "LLM+QD", and "LLM". The x-axis represents the MetaQA Dataset categories ("1-hop", "2-hop", "3-hop"), and the y-axis represents the Hit@1 Score, ranging from 0.0 to 1.0. ### Components/Axes * **Title:** "MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines" * **X-axis Title:** "MetaQA Dataset" * **X-axis Labels:** "1-hop", "2-hop", "3-hop" * **Y-axis Title:** "Hit@1 Score" * **Y-axis Scale:** 0.0 to 1.0, with major ticks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **Legend:** Located in the top-right corner of the chart. * **Title:** "Models" * **Entries:** * Red square: "Our method" * Orange square: "LLM+KG" * Magenta square: "LLM+QD" * Purple square: "LLM" ### Detailed Analysis The chart presents grouped bar charts for each "hop" category. Within each category, there are four bars representing the four models. **1-hop:** * **Our method (Red):** The bar reaches approximately 0.93. The error bar extends from approximately 0.91 to 0.95. * **LLM+KG (Orange):** The bar reaches approximately 0.96. The error bar extends from approximately 0.95 to 0.97. * **LLM+QD (Magenta):** The bar reaches approximately 0.45. The error bar extends from approximately 0.43 to 0.47. * **LLM (Purple):** The bar reaches approximately 0.47. The error bar extends from approximately 0.45 to 0.49. **2-hop:** * **Our method (Red):** The bar reaches approximately 0.78. The error bar extends from approximately 0.77 to 0.79. * **LLM+KG (Orange):** The bar reaches approximately 0.59. The error bar extends from approximately 0.58 to 0.60. * **LLM+QD (Magenta):** The bar reaches approximately 0.37. The error bar extends from approximately 0.36 to 0.38. * **LLM (Purple):** The bar reaches approximately 0.39. The error bar extends from approximately 0.38 to 0.40. **3-hop:** * **Our method (Red):** The bar reaches approximately 0.63. The error bar extends from approximately 0.62 to 0.64. * **LLM+KG (Orange):** The bar reaches approximately 0.50. The error bar extends from approximately 0.49 to 0.51. * **LLM+QD (Magenta):** The bar reaches approximately 0.54. The error bar extends from approximately 0.53 to 0.55. * **LLM (Purple):** The bar reaches approximately 0.52. The error bar extends from approximately 0.51 to 0.53. ### Key Observations * **Overall Performance:** "Our method" consistently achieves the highest Hit@1 scores across all three "hop" categories. * **Performance Drop with Hops:** For "Our method" and "LLM+KG", there is a noticeable decrease in Hit@1 scores as the number of hops increases from 1 to 2, and then a further decrease from 2 to 3. * **LLM-based Models:** The "LLM+QD" and "LLM" models show significantly lower performance compared to "Our method" and "LLM+KG" in the 1-hop and 2-hop categories. * **3-hop Anomaly:** In the 3-hop category, "LLM+QD" and "LLM" show a slight increase in performance compared to the 2-hop category, and their scores become closer to "LLM+KG". "Our method" still maintains the lead. * **Standard Deviation:** The standard deviations (indicated by error bars) are relatively small across all models and categories, suggesting consistent performance within each group. ### Interpretation This chart demonstrates the effectiveness of "Our method" in achieving higher Hit@1 scores on the MetaQA dataset compared to the baseline models, particularly as the complexity of the questions (indicated by the number of hops) increases. The significant performance gap between "Our method" and the LLM-based baselines in the 1-hop and 2-hop scenarios suggests that "Our method" is better at handling these types of questions. The observed drop in performance for "Our method" and "LLM+KG" with increasing hops is a common challenge in question-answering systems, as multi-hop reasoning is inherently more difficult. The slight improvement or stabilization of "LLM+QD" and "LLM" in the 3-hop category, while still lagging behind, might indicate different strengths or weaknesses in their underlying architectures or training data. The data suggests that "Our method" offers a superior approach for MetaQA tasks, especially for simpler question structures, and maintains a competitive edge even with more complex reasoning requirements. The error bars indicate a reasonable level of confidence in the reported mean scores.

## Bar Chart: MetaQA Hit@1 Scores ### Overview This bar chart presents the Hit@1 scores (Mean ± Std) for a new method and several baseline models on the MetaQA dataset, categorized by the number of hops (1-hop, 2-hop, and 3-hop). Error bars are included for each data point, representing the standard deviation. ### Components/Axes * **Title:** "MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines" - positioned at the top-center. * **X-axis:** "MetaQA Dataset" - with markers "1-hop", "2-hop", and "3-hop". * **Y-axis:** "Hit@1 Score" - ranging from 0.0 to 1.0. * **Legend:** Located in the top-right corner, listing the models: * "Our method" (Red) * "LLM+KG" (Orange) * "LLM+QD" (Purple) * "LLM" (Magenta) ### Detailed Analysis The chart consists of three groups of bars, one for each hop value (1-hop, 2-hop, 3-hop). Each group contains four bars, representing the Hit@1 score for each of the four models. Error bars are present on top of each bar. **1-hop:** * **Our method (Red):** Approximately 0.93 ± 0.02. The bar extends to roughly 0.95 and down to 0.91. * **LLM+KG (Orange):** Approximately 0.95 ± 0.02. The bar extends to roughly 0.97 and down to 0.93. * **LLM+QD (Purple):** Approximately 0.46 ± 0.03. The bar extends to roughly 0.49 and down to 0.43. * **LLM (Magenta):** Approximately 0.43 ± 0.03. The bar extends to roughly 0.46 and down to 0.40. **2-hop:** * **Our method (Red):** Approximately 0.78 ± 0.04. The bar extends to roughly 0.82 and down to 0.74. * **LLM+KG (Orange):** Approximately 0.62 ± 0.04. The bar extends to roughly 0.66 and down to 0.58. * **LLM+QD (Purple):** Approximately 0.39 ± 0.03. The bar extends to roughly 0.42 and down to 0.36. * **LLM (Magenta):** Approximately 0.36 ± 0.03. The bar extends to roughly 0.39 and down to 0.33. **3-hop:** * **Our method (Red):** Approximately 0.65 ± 0.04. The bar extends to roughly 0.69 and down to 0.61. * **LLM+KG (Orange):** Approximately 0.53 ± 0.04. The bar extends to roughly 0.57 and down to 0.49. * **LLM+QD (Purple):** Approximately 0.51 ± 0.04. The bar extends to roughly 0.55 and down to 0.47. * **LLM (Magenta):** Approximately 0.48 ± 0.04. The bar extends to roughly 0.52 and down to 0.44. ### Key Observations * "Our method" consistently outperforms the baseline models (LLM+KG, LLM+QD, and LLM) across all hop values. * LLM+KG generally performs better than LLM+QD and LLM. * The performance of all models decreases as the number of hops increases. * The error bars indicate that the standard deviation is relatively small, suggesting consistent results. ### Interpretation The data suggests that the proposed method is effective in improving the Hit@1 score on the MetaQA dataset, particularly when compared to the baseline models. The decrease in performance with increasing hop values indicates that the task becomes more challenging as the reasoning chain lengthens. The relatively small standard deviations suggest that the results are reliable and not heavily influenced by random variations. The consistent outperformance of "Our method" suggests it is more robust to the increased complexity of multi-hop reasoning. The gap between "Our method" and the baselines widens as the hop count increases, indicating that the proposed method's advantage is more pronounced in more complex scenarios.

## Bar Chart: MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines ### Overview This is a grouped bar chart comparing the performance of four different models on the MetaQA dataset across three levels of question complexity (1-hop, 2-hop, 3-hop). The performance metric is the Hit@1 Score, presented as the mean with standard deviation error bars. The chart demonstrates how model performance degrades as the reasoning complexity (number of hops) increases. ### Components/Axes * **Chart Title:** "MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines" * **Y-Axis:** * **Label:** "Hit@1 Score" * **Scale:** Linear, ranging from 0.0 to 1.0, with major tick marks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **X-Axis:** * **Label:** "MetaQA Dataset" * **Categories (from left to right):** "1-hop", "2-hop", "3-hop". These represent the complexity of the question-answer task. * **Legend:** * **Title:** "Models" * **Position:** Top-right corner of the plot area. * **Items (with associated colors):** 1. **Our method** (Red bar) 2. **LLM+KG** (Orange bar) 3. **LLM+QD** (Magenta/Pink bar) 4. **LLM** (Dark Purple bar) * **Data Representation:** For each X-axis category (hop level), there is a group of four bars, one for each model in the legend order. Each bar has a black error bar extending vertically from its top, representing the standard deviation (Std). ### Detailed Analysis **1-hop Category (Leftmost Group):** * **Our method (Red):** Highest score. Bar height is approximately **0.91**. Error bar extends from ~0.89 to ~0.93. * **LLM+KG (Orange):** Second highest. Bar height is approximately **0.95**. Error bar extends from ~0.93 to ~0.97. * **LLM+QD (Magenta):** Significantly lower. Bar height is approximately **0.45**. Error bar extends from ~0.42 to ~0.48. * **LLM (Purple):** Similar to LLM+QD. Bar height is approximately **0.46**. Error bar extends from ~0.43 to ~0.49. **2-hop Category (Middle Group):** * **Our method (Red):** Remains the highest. Bar height is approximately **0.78**. Error bar extends from ~0.76 to ~0.80. * **LLM+KG (Orange):** Second highest, but with a larger drop from 1-hop. Bar height is approximately **0.58**. Error bar extends from ~0.56 to ~0.60. * **LLM+QD (Magenta):** Lowest in this group. Bar height is approximately **0.37**. Error bar extends from ~0.34 to ~0.40. * **LLM (Purple):** Slightly higher than LLM+QD. Bar height is approximately **0.38**. Error bar extends from ~0.35 to ~0.41. **3-hop Category (Rightmost Group):** * **Our method (Red):** Still the highest, but with a further decline. Bar height is approximately **0.63**. Error bar extends from ~0.61 to ~0.65. * **LLM+KG (Orange):** Second highest. Bar height is approximately **0.49**. Error bar extends from ~0.47 to ~0.51. * **LLM+QD (Magenta):** Third highest. Bar height is approximately **0.54**. Error bar extends from ~0.52 to ~0.56. * **LLM (Purple):** Lowest in this group. Bar height is approximately **0.52**. Error bar extends from ~0.50 to ~0.54. **Trend Verification per Model:** * **Our method (Red):** Shows a clear, consistent downward trend as hop count increases (0.91 -> 0.78 -> 0.63). It is the top-performing model in all categories. * **LLM+KG (Orange):** Also shows a consistent downward trend (0.95 -> 0.58 -> 0.49). It is the second-best model in all categories. * **LLM+QD (Magenta):** Performance dips at 2-hop (0.45 -> 0.37) but recovers slightly at 3-hop (0.54). It is not consistently the worst or second-worst. * **LLM (Purple):** Performance dips at 2-hop (0.46 -> 0.38) and recovers slightly at 3-hop (0.52). It is generally the lowest or tied for lowest performer. ### Key Observations 1. **Performance Hierarchy:** A clear performance hierarchy is established across all complexity levels: **Our method > LLM+KG > (LLM+QD ≈ LLM)**. The gap between the top two models and the bottom two is substantial, especially at 1-hop and 2-hop. 2. **Impact of Complexity:** All models experience a decline in Hit@1 Score as the number of reasoning hops increases from 1 to 3. This indicates that multi-hop reasoning is a more challenging task for all evaluated systems. 3. **Error Bar Overlap:** The error bars for "Our method" and "LLM+KG" do not overlap with each other or with the other two models in the 1-hop and 2-hop categories, suggesting the performance differences are statistically significant. At 3-hop, the error bars for LLM+QD and LLM overlap, indicating their performance is not significantly different at this complexity level. 4. **Relative Resilience:** While all models decline, "Our method" shows the most resilience in absolute terms, maintaining a score above 0.6 even at 3-hop complexity. LLM+KG experiences the sharpest drop between 1-hop and 2-hop. ### Interpretation The data strongly suggests that the proposed model ("Our method") is the most effective for the MetaQA task, consistently outperforming the baselines regardless of question complexity. The inclusion of a Knowledge Graph (LLM+KG) provides a significant boost over the base LLM, particularly for simpler (1-hop) questions, but its advantage diminishes with increased complexity. The LLM+QD and base LLM models perform poorly in comparison, struggling even with 1-hop questions. The universal downward trend across all models underscores the inherent difficulty of scaling question-answering systems to handle longer reasoning chains. The fact that "Our method" degrades more gracefully suggests its architecture or training better captures the dependencies required for multi-hop inference. The chart serves as evidence that the authors' method advances the state-of-the-art for this specific benchmark, addressing a key challenge in knowledge-intensive QA systems.

## Bar Chart: MetaQA Hit@1 Scores (Mean ± Std) for Our Method and Baselines ### Overview The chart compares the Hit@1 scores (mean ± standard deviation) of four models ("Our method," "LLM+KG," "LLM+QD," "LLM") across three MetaQA datasets: "1-hop," "2-hop," and "3-hop." The y-axis represents the Hit@1 score (0–1), while the x-axis categorizes the datasets. Error bars indicate variability (standard deviation). ### Components/Axes - **X-axis (MetaQA Dataset)**: Labeled "1-hop," "2-hop," "3-hop" (left to right). - **Y-axis (Hit@1 Score)**: Ranges from 0.0 to 1.0 in increments of 0.2. - **Legend**: Located in the top-right corner, with four color-coded models: - Red: "Our method" - Orange: "LLM+KG" - Purple: "LLM+QD" - Dark purple: "LLM" - **Error Bars**: Vertical lines on top of each bar, representing ± standard deviation. ### Detailed Analysis #### 1-hop Dataset - **Our method**: ~0.92 (±0.03) - **LLM+KG**: ~0.95 (±0.02) - **LLM+QD**: ~0.45 (±0.03) - **LLM**: ~0.45 (±0.03) #### 2-hop Dataset - **Our method**: ~0.78 (±0.04) - **LLM+KG**: ~0.59 (±0.03) - **LLM+QD**: ~0.37 (±0.03) - **LLM**: ~0.38 (±0.03) #### 3-hop Dataset - **Our method**: ~0.63 (±0.04) - **LLM+KG**: ~0.49 (±0.03) - **LLM+QD**: ~0.54 (±0.03) - **LLM**: ~0.52 (±0.03) ### Key Observations 1. **"Our method" consistently outperforms all baselines** across all datasets, with the largest margin in the 1-hop dataset (~0.92 vs. 0.95 for "LLM+KG"). 2. **"LLM+KG" shows a significant drop in performance** from 1-hop to 2-hop (0.95 → 0.59) and further to 3-hop (0.49). 3. **"LLM+QD" and "LLM" exhibit similar performance** in 2-hop and 3-hop, but "LLM+QD" slightly outperforms "LLM" in 3-hop (0.54 vs. 0.52). 4. **Error bars indicate variability**: "Our method" has the smallest error bars (≤0.04), suggesting higher reliability. ### Interpretation The data demonstrates that **"Our method" is the most effective** for MetaQA tasks, particularly in 1-hop scenarios. The decline in performance for "LLM+KG" across datasets suggests it struggles with increasing complexity (hop count). "LLM+QD" and "LLM" perform comparably but lag behind "Our method," indicating potential limitations in their design or training. The error bars highlight that "Our method" has the most consistent results, making it a robust choice for MetaQA.