2602.00007

# PPoGA: Predictive Plan-on-Graph with Action for Knowledge Graph Question Answering **Authors**: MinGyu Jeon, SuWan Cho, JaeYoung Shu > MODULABS (2018) ## Abstract Large Language Models (LLMs) augmented with Knowledge Graphs (KGs) have advanced complex question answering, yet they often remain susceptible to failure when their initial high-level reasoning plan is flawed. This limitation, analogous to cognitive functional fixedness, prevents agents from restructuring their approach, leading them to pursue unworkable solutions. To address this, we propose PPoGA (Predictive Plan-on-Graph with Action), a novel KGQA framework inspired by human cognitive control and problem-solving. PPoGA incorporates a Planner-Executor architecture to separate high-level strategy from low-level execution and leverages a Predictive Processing mechanism to anticipate outcomes. The core innovation of our work is a self-correction mechanism that empowers the agent to perform not only Path Correction for local execution errors but also Plan Correction by identifying, discarding, and reformulating the entire plan when it proves ineffective. We conduct extensive experiments on three challenging multi-hop KGQA benchmarks: GrailQA, CWQ, and WebQSP. The results demonstrate that PPoGA achieves state-of-the-art performance, significantly outperforming existing methods. Our work highlights the critical importance of metacognitive abilities like problem restructuring for building more robust and flexible AI reasoning systems. Knowledge Graph Question Answering, Large Language Models, Self-Correction, Predictive Processing, Neuro-Symbolic AI copyright: acmlicensed journalyear: 2018 doi: XXXXXXX.XXXXXXX conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY isbn: 978-1-4503-XXXX-X/2018/06 ccs: Computing methodologies Reasoning about belief and knowledge ## 1. Introduction Large Language Models (LLMs) have recently demonstrated unprecedented success in few-shot and zero-shot generalization, showcasing their potential to solve complex reasoning tasks (Brown et al., 2020; Chowdhery et al., 2022; Wei et al., 2022; Ouyang et al., 2022). However, LLMs face inherent limitations such as the computational opacity of their internal knowledge, a lack of up-to-date information, and the phenomenon of hallucination, where they generate factually incorrect information (Chen et al., 2024; Sun et al., 2023a, b; Ji et al., 2023). To mitigate these limitations, the KG-augmented LLM paradigm has emerged as a promising solution, integrating Knowledge Graphs (KGs)—which provide explicit and editable factual knowledge—into the reasoning process of LLMs (Sun et al., 2023a, b; Pan et al., 2024; Yasunaga et al., 2021; Baek et al., 2023). Early studies on KG-based reasoning primarily extended the Chain-of-Thought (CoT) to explore reasoning paths within the KG. However, this approach relied solely on the internal reasoning of LLMs, leading to the generation of inaccurate paths that diverge from the actual KG structure (Wei et al., 2022; Press et al., 2022; Creswell et al., 2022). Addressing this issue, Plan-on-Graph (PoG) introduced an advanced methodology that decomposes a question into multiple sub-goals, dynamically explores reasoning paths within the KG, and self-corrects its path upon encountering errors (Chen et al., 2024). However, PoG’s self-correction mechanism has a fundamental limitation: it is confined to analytical solving, where an efficient solution is sought within a given framework (Wertheimer, 1959; Duncker, 1945). If the initial high-level plan—the ”framework” itself—is unsuitable for solving the problem, the agent can become functionally fixed on that plan, failing to redefine the problem space and ultimately leading to failure (Duncker, 1945; Newell and Simon, 1972). To overcome this limitation, this paper proposes PPoGA (Predictive Plan-on-Graph with Action), a novel framework that computationally models the multi-layered and flexible problem-solving methods of humans. PPoGA begins by mimicking the Hierarchical Control of Action in humans, explicitly separating roles into a Planner, which formulates high-level strategies, and an Executor, which carries out specific procedures. This division reduces the cognitive load on each component, thereby maximizing the efficiency and stability of the entire system (Wang et al., 2024; Norman and Shallice, 1986; Kahneman, 2011). Building on this structure, PPoGA incorporates a Predictive Processing mechanism, inspired by how the brain learns from prediction errors, to deepen its reasoning by simulating the outcomes of actions before they are taken (Clark, 2013; Friston, 2010). Furthermore, the core contribution of PPoGA lies in its self-correction mechanism, which emulates the dual human ability to find solutions within a problem’s framework and to restructure the problem when the framework itself is flawed (Wertheimer, 1959). Through this mechanism, PPoGA performs both Path Correction —optimizing paths within a given plan—and Plan Correction —recognizing the failure of the plan itself and formulating a new strategy. This enables PPoGA to achieve a higher level of robust and flexible problem-solving capabilities compared to previous studies (Flavell, 1979). The key contributions of this paper are as follows: - First, we propose PPoGA, a novel Knowledge Graph Question Answering (KGQA) framework that integrates a Planner-Executor role separation, prediction-based reasoning, and dynamic Path Correction into a single, coherent architecture. - Second, we are the first to propose and implement a self-correction mechanism that allows an agent to perform both Path Correction for execution-level errors and Plan Correction by recognizing plan failure and reformulating strategy. - Third, through experiments on complex KGQA benchmarks, we demonstrate that PPoGA achieves higher accuracy and robustness than existing static plan-based agents. ## 2. Methodology In this section, we elucidate the architecture and operational workflow of our proposed framework, PPoGA (Predictive Plan-on-Graph with Action). Our method is predicated on the cognitive principle of Hierarchical Control of Action, which separates high-level strategic reasoning from low-level environmental interaction. This is realized through a dual-component architecture comprising a Planner and an Executor, whose operations are coordinated via a unified, three-tiered Integrated Memory Architecture. The interplay between these components enables a dynamic, multi-stage workflow designed for robust and adaptive problem-solving on knowledge graphs. The Planner, built upon a LLM, serves as the cognitive core of the agent. It is responsible for high-level reasoning tasks, including initial strategy formulation, task decomposition into a multi-step plan, outcome prediction, and self-correction based on environmental feedback. The Executor acts as the interface to the external environment—in this case, a KG. It translates abstract instructions from the Planner into executable SPARQL queries, executes them, and returns structured observations, thereby closing the loop between reasoning and action. All interactions and state information are mediated through the Integrated Memory Architecture, which provides the Planner with comprehensive situational awareness throughout the problem-solving process. ### 2.1. The PPoGA Workflow The PPoGA workflow emulates a structured but flexible human-like reasoning process, beginning with Decomposition and proceeding through an iterative Step Cycle and an Evaluation phase, as illustrated in Figure 1. Upon receiving a question, the Planner first performs Decomposition, creating a multi-step strategic plan to find the answer. This initial plan is a dynamic blueprint composed of discrete steps, where each step encapsulates a unique objective, a description, and a status tracker (e.g., not started, in progress, completed). Once the plan is formulated, the agent executes it sequentially via an iterative, four-stage Step Cycle: Predict, Act, Observe, and Think. The cycle begins with Prediction, where the Planner simulates the expected outcome of the current step’s objective before any action is taken. This proactive mechanism allows the agent to anticipate and identify potential deviations. Following prediction, the Action phase, termed Path Exploration, is executed by the Executor to retrieve information from the KG. This process starts by identifying the topic entity from the user’s query. The Executor then explores the KG by retrieving all connected relations and subsequent entities to form a set of candidate triples. To manage combinatorial explosion, if the number of candidates exceeds a predefined threshold (e.g., 70), an embedding-based pruning mechanism is employed to retain only the most semantically relevant candidates. From this pruned set, the Executor selects the single most promising entity, updating the agent’s memory with the choice and its corresponding rationale. The result of this action forms the Observation, which is then compared against the initial prediction to compute an error signal. This comparison informs the Think stage, where the Planner generates an internal reflection on the outcome, leading to a decision in the final phase. The culmination of each Step Cycle is the Evaluation and Self-Correction phase, where the Planner determines the next course of action. If the step’s objective is met, the agent proceeds to the next step in the plan. If the action was unsuccessful but the overall plan remains sound, it initiates Path Correction, re-attempting the current step while leveraging memory to avoid previously failed exploration paths. Should repeated attempts fail, the agent concludes the plan itself may be flawed and triggers Plan Correction (Replan), returning to the decomposition phase to formulate a new strategy based on the knowledge gathered thus far. The workflow finishes when the agent determines it has sufficient information to synthesize a final answer. <details> <summary>images/PPoGA_workflow_v4.png Details</summary>  ### Visual Description ## Diagram Type: Multi-Component Process Flowchart (AI/Reasoning System Architecture) ### Overview The image is a detailed flowchart illustrating a **reasoning system** (likely for AI problem-solving) with five interconnected components: *Plan & Reasoning*, *Executor & Exploration*, *Memory Architecture*, *Evaluation & Correction*, and *Flow control & Process Loop*. Each component contains nested sub-elements (rectangles/rounded rectangles) with text labels, connected by arrows to show process flow and feedback loops. ### Components/Axes (Sub-Components and Labels) The diagram is divided into five main light-gray boxes, each with distinct color-coded sub-components: #### 1. **Plan & Reasoning (Left Box, Blue Sub-Components)** - Top: *“Question”* (text) → *“Planner”* (blue rounded rectangle) → *“Decompose”* (blue rectangle) → *“Step_1”*, *“Step_2”*, ..., *“Step_N”* (blue rectangles; “...” indicates multiple steps). - Below: *“Step Cycle”* (blue rectangle) with sub-components: *“Predict”*, *“Action”*, *“Observation”*, *“Think”* (blue rounded rectangles). - Bottom: *“Evaluate”* (blue rectangle). #### 2. **Executor & Exploration (Middle Box, Green Sub-Components)** - Top: *“Executor”* (green rounded rectangle). - Below: *“Path Exploration”* (green rectangle) with sub-components: *“Entity Select(From data)”*, *“Relation Explore”*, *“Entity Explore”*, *“Prune Candidates”*, *“Entity Select(From Candidates)”* (green rounded rectangles). #### 3. **Memory Architecture (Right Box, Yellow Sub-Components)** Three layers (yellow rectangles) store data: - *“Strategic Layer”*: *“Question”*, *“Plan”*, *“Replan Count”* (yellow rounded rectangles). - *“Step Cycle Layer”*: *“Current Step”*, *“Predict”*, *“Observation”*, *“Think”* (yellow rounded rectangles). - *“Knowledge Layer”*: *“Explored Entities”*, *“Explored Relations”*, *“Exploration Path”*, *“Reasoning Chains”* (yellow rounded rectangles). #### 4. **Evaluation & Correction (Bottom Middle Box, Pink Sub-Components)** Two feedback paths: - *“Correct Path”* (pink rounded rectangle) → *“Path Correction”* (pink rectangle) → *“Step Cycle”* (blue rectangle). - *“Replan”* (pink rounded rectangle) → *“Plan Correction”* (pink rectangle) → *“Decompose”* (blue rectangle). #### 5. **Flow control & Process Loop (Bottom Right Box, Orange/Red Sub-Components)** Two progression paths: - *“Proceed”* (orange rounded rectangle) → *“Next Step”* (orange rectangle) → *“Step_2”* (blue rectangle). - *“Finish”* (orange rounded rectangle) → *“Final Answer”* (red rectangle) → *“Answer”* (red rectangle). ### Detailed Analysis (Component Relationships and Flow) - **Plan & Reasoning** initiates with a *“Question”*, processed by *“Planner”* and *“Decompose”* into steps (Step_1 to Step_N). Each step enters *“Step Cycle”* (Predict → Action → Observation → Think), then *“Evaluate”*. - **Executor & Exploration** handles execution: *“Executor”* oversees *“Path Exploration”* (entity/relation exploration, pruning, selection). - **Memory Architecture** stores data across layers: *Strategic* (question, plan, replan count), *Step Cycle* (current step, predict, observation, think), *Knowledge* (explored entities/relations, path, reasoning chains). - **Evaluation & Correction** corrects paths or replans: *“Correct Path”* fixes the path, *“Replan”* adjusts the plan, feeding back to *“Step Cycle”* or *“Decompose”*. - **Flow control & Process Loop** manages progression: *“Proceed”* moves to the next step, *“Finish”* outputs the final answer. ### Key Observations - **Color Coding:** Blue (Plan & Reasoning), Green (Executor & Exploration), Yellow (Memory Architecture), Pink (Evaluation & Correction), Orange (Flow control), Red (Final Answer). - **Flow Direction:** Arrows show sequential and feedback loops (e.g., *Evaluate* feeds back to *Step Cycle* or *Decompose*). - **Sub-Component Hierarchy:** Each main box contains nested sub-components (e.g., *Step Cycle* has *Predict*, *Action*, etc.). ### Interpretation This diagram models a **cognitive/AI agent’s decision-making process** for complex tasks. It emphasizes: - **Iterative Learning:** Feedback loops (replan, path correction) refine plans and paths. - **Structured Memory:** Layers (strategic, step, knowledge) support planning, execution, and knowledge retention. - **Modular Design:** Components (plan, execute, evaluate, control) work in tandem to break down tasks, explore solutions, and correct errors. The architecture suggests a system designed to handle ambiguity, learn from experience, and produce reasoned answers—similar to how humans or advanced AI agents solve problems. *(No numerical data or charts; this is a process diagram with text labels and flow relationships.)* </details> Figure 1. The PPoGA Workflow and Architecture. The framework consists of a Planner (reasoning) and an Executor (action), which interact through a three-layered Integrated Memory Architecture. The workflow follows an iterative cycle of Decomposition, a four-stage Step Cycle (Predict, Act, Observe, Think), and Evaluation, enabling both Plan and Path Correction. : ### 2.2. Integrated Memory Architecture The entire workflow is underpinned by the Integrated Memory Architecture, a centralized cognitive workspace structured into three interconnected layers that operate at different levels of abstraction. The Strategic Layer maintains the highest-level context, storing the initial question, the overall plan, and metadata such as a replan counter to prevent infinite loops; it provides the foundational knowledge required for Plan Correction. Operating at the level of a single action, the Step Cycle Layer holds transient state information like the current step’s objective, predictions, and observations, which is crucial for enabling Path Correction. Finally, the Knowledge Layer serves as the persistent repository for all factual information gathered from the KG, accumulating explored entities, relations, and reasoning chains to ensure that knowledge is retained and redundant actions are avoided. ## 3. Experiment ### 3.1. Experimental Setup We demonstrate the effectiveness of the proposed framework, PPoGA (Predictive Plan-on-Graph with Action), on the well-established multi-hop KGQA benchmarks GrailQA (Gu et al., 2021), CWQ (Talmor and Berant, 2018), and WebQSP (Yih et al., 2016). The experimental settings are consistent with previous studies such as ToG and PoG (Sun et al., 2023b; Chen et al., 2024), and the same testing samples are utilized for computational efficiency. As in previous KGQA literature (Gu et al., 2021; Jiang et al., 2023b; Luo et al., 2024), we use Exact Match Accuracy (Hits@1) as the evaluation metric (Li et al., 2023; Wang et al., 2023a). This criterion measures whether the entity predicted by the model exactly matches the ground truth answer (Gu et al., 2021; Sun et al., 2023b). The comparison baselines follow PoG (Chen et al., 2024) and can be classified into categories: - LLM-only: a conventional language model without KG integration (Brown et al., 2020; Wei et al., 2022; Wang et al., 2023b; Zhou et al., 2023). - Fine-tuned KG-augmented LLM: a model fine-tuned on KGQA datasets with parameter updates (Jiang et al., 2023b; Shu et al., 2022; Luo et al., 2024; Yu et al., 2023; Zhang et al., 2023; Ye et al., 2022). - Prompting KG-augmented LLM (GPT-3.5): a prompting-based reasoning model using GPT-3.5 (Li et al., 2023; Wang et al., 2023a; Jiang et al., 2023a; Xiong et al., 2024; Sun et al., 2023b). ### 3.2. Results We evaluate PPoGA across three standard benchmarks. The GrailQA benchmark measures the ability of models to generalize under four evaluation settings: Overall, I.I.D., Compositional, and Zero-Shot (Gu et al., 2021). These configurations are designed to evaluate, respectively, overall performance, in-domain reasoning, compositional generalization, and out-of-distribution reasoning capability (Chen et al., 2024; Sun et al., 2023b). Additionally, we assess performance on Complex WebQuestions (CWQ) (Talmor and Berant, 2018) and WebQuestionsSP (WebQSP) (Yih et al., 2016) to validate the framework’s effectiveness on other complex question-answering datasets. The main results are presented in Table 1 (for GrailQA) and Table 2 (for CWQ and WebQSP). Our experiments are conducted using GPT-3.5 to ensure a fair and direct comparison against the prompting-based baselines in the same category. PPoGA consistently achieves state-of-the-art performance among all Prompting KG-Augmented LLM (GPT-3.5) baselines across all three datasets. Most notably, PPoGA demonstrates a clear and consistent performance improvement over its direct predecessor, PoG (Chen et al., 2024). On the GrailQA benchmark (Table 1), PPoGA achieves an overall accuracy of 78.9, surpassing PoG’s 76.5. This trend holds across the generalization splits, especially in the challenging Zero-shot split (82.3 vs. 81.7). This superiority is further confirmed on CWQ and WebQSP (Table 2), where PPoGA (64.5 on CWQ, 83.1 on WebQSP) outperforms PoG | Method LLM-Only IO Prompt (Brown et al., 2020) | Overall 29.4 | I.I.D. - | Compositional - | Zero-shot - | | --- | --- | --- | --- | --- | | CoT (Wei et al., 2022) | 28.1 | - | - | - | | SC (Wang et al., 2023b) | 29.6 | - | - | - | | Fine-Tuned KG-Augmented LLM | | | | | | RnG-KBQA (Ye et al., 2022) | 68.8 | 86.2 | 63.8 | 63.0 | | TIARA (Shu et al., 2022) | 73.0 | 87.8 | 69.2 | 68.0 | | FC-KBQA (Zhang et al., 2023) | 73.2 | 88.5 | 70.0 | 67.6 | | Pangu (Gu et al., 2023) | 75.4 | 84.4 | 74.6 | 71.6 | | FlexKBQA (Li et al., 2024) | 62.8 | 71.3 | 59.1 | 60.6 | | GAIN (Shu and Yu, 2024) | 76.3 | 88.5 | 73.7 | 71.8 | | Prompting KG-Augmented LLM (GPT-3.5) | | | | | | KB-BINDER (Li et al., 2023) | 50.6 | - | - | - | | ToG (Sun et al., 2023b) | 68.7 | 70.1 | 56.1 | 72.7 | | PoG (Chen et al., 2024) | 76.5 | 76.3 | 62.1 | 81.7 | | PPoGA (ours) | 78.9 | 78.6 | 64.7 | 82.3 | Table 1. Performance comparison of different methods on GrailQA. | Method LLM-Only IO Prompt (Brown et al., 2020) | CWQ 37.6 | WebQSP 63.3 | | --- | --- | --- | | CoT (Wei et al., 2022) | 38.8 | 62.2 | | SC (Wang et al., 2023b) | 45.4 | 61.1 | | Fine-Tuned KG-Augmented LLM | | | | UniKGQA (Jiang et al., 2023b) | 51.2 | 79.1 | | TIARA (Shu et al., 2022) | - | 75.2 | | RE-KBQA (Cao et al., 2023) | 50.3 | 74.6 | | DeCAF (Yu et al., 2023) | 70.4 | 82.1 | | ROG (Luo et al., 2024) | 62.6 | 85.7 | | Prompting KG-Augmented LLM (GPT-3.5) | | | | KD-COT (Wang et al., 2023a) | 50.5 | 73.7 | | KB-BINDER (Li et al., 2023) | 54.3 | 74.4 | | StructGPT (Jiang et al., 2023a) | 57.1 | 72.6 | | ToG (Sun et al., 2023b) | - | 76.2 | | PoG (Chen et al., 2024) | 63.2 | 82.0 | | PPoGA (ours) | 64.5 | 83.1 | Table 2. Performance comparison (Hits@1) on CWQ and WebQSP, aligned with PoG baselines. ## 4. Conclusion In this paper, we introduced PPoGA (Predictive Plan-on-Graph with Action), a novel framework for complex KGQA that addresses the critical limitation of functional fixedness in existing planning-based agents. By computationally modeling principles of human cognition, PPoGA moves beyond simple error correction to enable true problem restructuring. Our core contributions are the explicit separation of high-level reasoning and low-level execution through a Planner-Executor architecture, the integration of a Predictive Processing mechanism for proactive error anticipation, and, most importantly, the introduction of a Two-Level Self-Correction mechanism. This dual-correction system empowers the agent to perform not only Path Corrections at the execution level but also Plan Corrections by identifying and abandoning flawed high-level plans. Our experimental results on three challenging KGQA benchmarks empirically validate the effectiveness of our approach. PPoGA consistently outperforms existing state-of-the-art methods, demonstrating that its ability to flexibly adapt and reformulate strategies leads to superior accuracy and robustness. These findings underscore the value of building agents that can reason about their own reasoning process, a key aspect of metacognition. The success of PPoGA suggests a promising direction for developing more autonomous and resilient AI systems. Future work could explore applying this hierarchical and self-correcting cognitive architecture to other complex, multi-step reasoning domains such as embodied AI, software engineering, and scientific discovery, where the ability to dynamically restructure a problem is paramount. Acknowledgements. This research was supported by Brian Impact Foundation, a non-profit organization dedicated to the advancement of science and technology for all. ## References - (1) - Baek et al. (2023) Jinyoung Baek, Sebon O’Brien, and Jong-Hoon Lee. 2023. Knowledge graph-augmented language models for financial question answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 10133–10147. - Brown et al. (2020) Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems. 1877–1901. - Cao et al. (2023) Yong Cao, Xianzhi Li, Huiwen Liu, Wen Dai, Shuai Chen, Bin Wang, Min Chen, and Daniel Hershcovich. 2023. Pay more attention to relation exploration for knowledge base question answering. arXiv preprint arXiv:2305.02118 (2023). - Chen et al. (2024) Lin Chen, Zirui Wang, Yeyu Zhang, Yuxiang Zhao, Jifan Tang, Weize Chen, Yong Zhang, and Yunjun Chen. 2024. Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs. arXiv preprint arXiv:2402.13875 (2024). - Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. PaLM: Scaling Language Modeling with Pathways. arXiv preprint arXiv:2204.02311 (2022). - Clark (2013) Andy Clark. 2013. Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science. Behavioral and Brain Sciences 36, 3 (2013), 181–204. - Creswell et al. (2022) Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022. Faithful chain-of-thought reasoning. arXiv preprint arXiv:2201.11903 (2022). - Duncker (1945) Karl Duncker. 1945. On problem-solving. In Psychological Monographs. Vol. 58. American Psychological Association, i–113. - Flavell (1979) John H Flavell. 1979. Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist 34, 10 (1979), 906. - Friston (2010) Karl Friston. 2010. The free-energy principle: a unified brain theory? Nature Reviews Neuroscience 11, 2 (2010), 127–138. - Gu et al. (2023) Yu Gu, Xiang Deng, and Yu Su. 2023. Don’t Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 4928–4949. - Gu et al. (2021) Yu Gu, Sue Kase, Michelle Vanni, Brian Sadler, Percy Liang, Xifeng Yan, and Yu Su. 2021. Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases. In Proceedings of the Web Conference 2021. 3477–3488. - Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys (2023). - Jiang et al. (2023a) Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023a. StructGPT: A General Framework for Large Language Model to Reason over Structured Data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 9237–9251. - Jiang et al. (2023b) Jinhao Jiang, Kun Zhou, Xin Zhao, and Ji-Rong Wen. 2023b. UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering over Knowledge Graphs. In International Conference on Learning Representations (ICLR). - Kahneman (2011) Daniel Kahneman. 2011. Thinking, Fast and Slow. Farrar, Straus and Giroux. - Li et al. (2023) Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, and Wenhu Chen. 2023. KB-BINDER: Few-Shot In-Context Learning on Knowledge Base Question Answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 6966–6980. - Li et al. (2024) Zhenyu Li, Sunqi Fan, Yu Gu, Xiuxing Li, Zhichao Duan, Bowen Dong, Ning Liu, and Jianyong Wang. 2024. FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18608–18616. - Luo et al. (2024) Linhao Luo, Yuan-Fang Li, Reza Haf, and Shirui Pan. 2024. Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning. In The Twelfth International Conference on Learning Representations (ICLR). - Newell and Simon (1972) Allen Newell and Herbert A Simon. 1972. Human Problem Solving. Prentice-Hall. - Norman and Shallice (1986) Donald A Norman and Tim Shallice. 1986. Attention to action: Willed and automatic control of behavior. In Consciousness and self-regulation. Springer, 1–18. - Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems. - Pan et al. (2024) Shirui Pan, Lin Chen, Zirui Wang, Yeyu Zhang, Yuxiang Zhao, Jifan Tang, Weize Chen, Yong Zhang, and Yunjun Chen. 2024. Unifying Large Language Models and Knowledge Graphs: A Roadmap. arXiv preprint arXiv:2401.01061 (2024). - Press et al. (2022) Ofir Press, Omer Levy, and Timo Schick. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350 (2022). - Shu and Yu (2024) Yiheng Shu and Zhiwei Yu. 2024. Distribution Shifts are Bottlenecks: Extensive Evaluation for Grounding Language Models to Knowledge Bases. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop. 71–88. - Shu et al. (2022) Yiheng Shu, Zhiwei Yu, Yuhan Li, Börje Karlsson, Tingting Ma, Yuzhong Qu, and Chin-Yew Lin. 2022. TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 8108–8121. - Sun et al. (2023b) Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel Ni, Heung-Yeung Shum, and Jian Guo. 2023b. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph. arXiv preprint arXiv:2310.01061 (2023). - Sun et al. (2023a) Zhen Sun, Zirui Wang, Yeyu Zhang, Yuxiang Zhao, Jifan Tang, Weize Chen, Yong Zhang, and Yunjun Chen. 2023a. Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning. arXiv preprint arXiv:2307.07697 (2023). - Talmor and Berant (2018) Alon Talmor and Jonathan Berant. 2018. The Web as a Knowledge-Base for Answering Complex Questions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Vol. 1. 641–651. - Wang et al. (2024) Jhen-Dong Wang, Yi-Lin Su, Tung-Cheng Chen, Hung-yi Chu, and Yun-Nung Chen. 2024. PLAN-AND-ACT: Improving Planning of Agents for Long-Horizon Tasks. arXiv preprint arXiv:2403.09572 (2024). - Wang et al. (2023a) Keheng Wang, Feiyu Duan, Sirui Wang, Peiguang Li, Yunsen Xian, Chuantao Yin, Wenge Rong, and Zhang Xiong. 2023a. Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-Intensive Question Answering. arXiv preprint arXiv:2308.13259 (2023). - Wang et al. (2023b) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, and Denny Zhou. 2023b. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In International Conference on Learning Representations. - Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems, Vol. 35. 24824–24837. - Wertheimer (1959) Max Wertheimer. 1959. Productive Thinking. Harper & Row. - Xiong et al. (2024) Guanming Xiong, Junwei Bao, and Wen Zhao. 2024. Interactive-KBQA: Multi-turn Interactions for Knowledge Base Question Answering with Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 10561–10582. - Yasunaga et al. (2021) Michihiro Yasunaga, Jungo Kasai, and Dragomir Radev. 2021. QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 535–546. - Ye et al. (2022) Xi Ye, Semih Yavuz, Kazuma Hashimoto, Yingbo Zhou, and Caiming Xiong. 2022. RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 6032–6043. - Yih et al. (2016) Wen-tau Yih, Matthew Richardson, Christopher Meek, Ming-Wei Chang, and Jina Suh. 2016. The Value of Semantic Parse Labeling for Knowledge Base Question Answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 201–206. - Yu et al. (2023) Donghan Yu, Sheng Zhang, Patrick Ng, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Yiqun Hu, Zhiguo Wang, and Bing Xiang. 2023. DeCAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases. In The International Conference on Learning Representations. - Zhang et al. (2023) Lingxi Zhang, Jing Zhang, Yanling Wang, Shulin Cao, Xinmei Huang, Cuiping Li, Hong Chen, and Juanzi Li. 2023. FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. - Zhou et al. (2023) Denny Zhou, Nathanael Sch”arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, and Quoc V Le. 2023. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. International Conference on Learning Representations (2023).