2504.10499v2

# Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey **Authors**: Zulun Zhu, Tiancheng Huang, Kai Wang, Junda Ye, Xinghe Chen, Siqiang Luo > Nanyang Technological UniversitySingaporeZULUN001@ntu.edu.sg > Nanyang Technological UniversitySingaporetiancheng.huang@ntu.edu.sg > Nanyang Technological UniversitySingaporekai˙wang@ntu.edu.sg > Beijing University of Posts and TelecommunicationsChinajundaye@bupt.edu.cn > Nanyang Technological UniversitySingaporexinghe001@e.ntu.edu.sg > Nanyang Technological UniversitySingaporesiqiang.luo@ntu.edu.sg Abstract. Large language models (LLMs) struggle with the factual error during inference due to the lack of sufficient training data and the most updated knowledge, leading to the hallucination problem. Retrieval-Augmented Generation (RAG) has gained attention as a promising solution to address the limitation of LLMs, by retrieving relevant information from external source to generate more accurate answers to the questions. Given the pervasive presence of structured knowledge in the external source, considerable strides in RAG have been made to employ the techniques related to graphs and achieve more complex reasoning based on the topological information between knowledge entities. Although recent surveys summarize RAG pipelines, they largely restrict the role of graphs to knowledge-graph traversal, leaving the broader impacts of graph structure on retrieval, prompting, and pipeline control underexplored. This survey offers a novel perspective on the functionality of graphs within RAG and their impact on enhancing performance across a wide range of graph-structured data. It provides a detailed breakdown of the roles that graphs play in RAG, covering database construction, algorithms, pipelines, and tasks. Finally, it identifies current challenges and outline future research directions, aiming to inspire further developments in this field. Our graph-centered analysis highlights the commonalities and differences in existing methods, setting the stage for future researchers in areas such as graph learning, database systems, and natural language processing. Graph, Large language model, Retrieval-augmented generation. ccs: Information systems Information retrieval ccs: Computing methodologies Natural language processing 1. Introduction Large Language Models (LLMs) have demonstrated high effectiveness in a wide spectrum of human-centric tasks, including text summarization (Ding et al., 2024), question answering (Dehghan et al., 2024), machine translation (Feng et al., 2024), and code generation (Liu et al., 2024c). Early models like BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) introduced advanced techniques for leveraging contextual embeddings, significantly enhancing performance of different natural language understanding tasks. Inspired by these fundamental works, models such as GPT-4 (OpenAI, 2024), LLaMA 2 (Touvron et al., 2023), and more recently Deepseek-R1 (DeepSeek-AI et al., 2025), have set new benchmarks by utilizing transformer decoder-only architectures and scaling up model sizes. These LLMs excel at generating human-like responses, adapting to a wide range of domains, and supporting complex reasoning. All of these advancements have led to a growing consensus that LLMs represent a key pathway to artificial general intelligence (AGI) (Bubeck et al., 2023). Despite these successes, LLMs face two major challenges: they are prone to factual errors and hallucinations, and they frequently struggle with real-world knowledge that inherently represents structured information (Lavrinovics et al., 2025; Agrawal et al., 2023; Zhu et al., 2024a). The first challenge arises because the retraining of LLMs to incorporate new or domain-specific knowledge is computationally expensive (Fatemi et al., 2023), leaving models susceptible to producing logically coherent but factually incorrect inferences, which is also known as the hallucination problem (Shuster et al., 2021). For the second challenge, knowledge graphs like DBpedia (Auer et al., 2007b), Wikidata (Vrandečić and Krötzsch, 2014), and Freebase (Bollacker et al., 2008) are primary examples of how entities and relationships are organized in a structured format, yet existing LLM-based approaches often fail to fully exploit these relational patterns, especially for complex or multi-hop reasoning tasks. These two challenges call for retrieval mechanisms that enable LLMs to dynamically access external and structured knowledge during inference. Retrieval-Augmented Generation (RAG) helps solve the first challenges by retrieving relevant external information, thereby reducing hallucinations and enhancing factual accuracy (Gao et al., 2024c; Huang and Huang, 2024). With respect to the second challenge, the integration of graph data management with RAG further enhances these capabilities by effectively organizing knowledge into organized forms such as knowledge graphs (Guo et al., 2023). This integration captures complex relational patterns and supports multi-hop reasoning tasks such as node classification (Chen et al., 2024c; Zhang et al., 2024b), link prediction (Dettmers et al., 2018; Pahuja et al., 2023), and subgraph extraction (Gutiérrez et al., 2024; Luo et al., 2023; Li et al., 2024a). Consequently, graph-enhanced RAG provides a powerful approach to improving LLM’s factual reliability and their ability to execute complex, structured reasoning tasks. The application of graph techniques in RAG has rapidly expanded, leading several recent surveys on LLMs to discuss graph-based methods, such as Han’s survey, Peng’s survey in 2024 (Peng et al., 2024) and Zhang et al.’s more recent work in 2025 (Zhang et al., 2025). While prior surveys provide a useful synthesis of RAG architectures and applications with graphs, they are largely pipeline-centric and often conflate graph-based RAG with knowledge-graph traversal. Consequently, they devote limited attention to the underlying graph techniques, and provide mainly high-level rather than task-specific guidance for applying graph techniques in LLM pipelines. In contrast, this paper fills this notable gap by providing a meticulous and detailed decomposition of the RAG framework, broadening the definition of graph-based RAG and thoroughly examining the concrete graph techniques applied in representative works. We explicitly discuss the rationale behind integrating these graph methods into RAG systems and highlight their specific advantages in enhancing answer accuracy and efficiency. Furthermore, based on insights drawn from over 200 recent RAG studies, we systematically categorize the use of graph techniques across the core components of RAG—including database construction, retrieval or prompting algorithms, processing pipelines, and task scenarios. Unlike previous surveys (Peng et al., 2024; Fan et al., 2024a; Hu and Lu, 2024; Gao et al., 2024c), this work offers an in-depth exploration from the viewpoint of graph data management with respect to database construction, query algorithms, prompt structure, and pipeline design, presenting novel insights into how detailed graph methods can significantly drive advancements in RAG research. The contributions of this work are summarized as follows: $\bullet$ Detailed and Case-by-Case Introduction of Graph Techniques in RAG: We meticulously introduce concrete graph techniques employed in RAG, analyzing them case by case, unlike the brief and superficial descriptions provided in existing surveys. This systematic and detailed review clarifies both the rationale for selecting these methods and their practical benefits in enhancing accuracy and efficiency. $\bullet$ Novel Perspective and Taxonomy: We introduce a novel perspective on the roles of graphs in RAG, encompassing database construction, algorithms, pipelines, and tasks. Additionally, we propose a taxonomy to categorize existing RAG methods and provide insightful discussion of their their pros and cons. $\bullet$ Comprehensive Resource and Future Directions: Based on an extensive review of over 200 studies, we provide a comprehensible resource to guide graph researchers in contributing to this evolving field. We introduce the representative works in detail to demonstrate the functionality of graphs to enhance the reasoning process of LLMs. This work lays the foundation for future advancements in areas such as graph learning, database systems, and natural language processing. 2. Comparison with Existing Surveys Surveys of LLMs. A large number of surveys (Yang et al., 2023; Huang and Chang, 2022; Naveed et al., 2023; Zhao et al., 2023b; Minaee et al., 2024; Hadi et al., 2023; Myers et al., 2024; Besta et al., 2024b) have explored LLMs from various perspectives over the years, reflecting their growing prominence in AI research. Early works (Yang et al., 2023; Hadi et al., 2023) provide comprehensive overviews of foundational models like BERT (Devlin et al., 2019) and GPT-3 (Brown et al., 2020), detailing their architectures, design principles, and advancements compared to earlier models. These surveys emphasize the evolution of LLMs and their capacity to handle zero-shot and few-shot learning tasks, marking a shift from task-specific fine-tuning. Other studies (Zhao et al., 2023b; Minaee et al., 2024; Shen et al., 2023) focus on specific technical aspects, such as pretraining methods, fine-tuning strategies, and alignment techniques for generating reliable outputs. They also address challenges like mitigating biases, ensuring factual accuracy, and aligning outputs with user intent. Surveys of RAG. Building on the foundation of LLM surveys, an increasing number of works have concentrated on RAG, exploring its workflow and the integration of external knowledge to enhance generative accuracy. Some surveys (Fan et al., 2024a; Hu and Lu, 2024; Huang and Huang, 2024; Gao et al., 2024c; Yu et al., 2024; Wu et al., 2024d; Zhu et al., 2023) delve into the architecture of RAG, detailing how retrieval mechanisms interact with generative models to create a seamless pipeline for producing context-aware responses. These studies often highlight the modularity of RAG workflows, emphasizing the flexibility of incorporating different retrieval strategies, such as vector-based retrieval and hybrid methods, to optimize performance across diverse tasks. Other surveys (Tonmoy et al., 2024; Huang et al., 2023a; Rawte et al., 2023; Zhang et al., 2023) prioritize addressing the hallucination problem, analyzing how external information retrieval mitigates factual inaccuracies, and ensures the generated responses align closely with real-world knowledge. They discuss the effectiveness of external retrieval in outputs of grounding models and reducing the risk of plausible but incorrect information generation. Additionally, some works (Zhao et al., 2024b, a; Zhou et al., 2024b) explore the broader applications of RAG, demonstrating its utility in tasks like question answering, summarization, and domain-specific knowledge generation. However, given the importance of graphs as a fundamental structure in RAG, graph-related applications are not a central focus of these works. Surveys of LLMs on graphs. Existing surveys of LLMs on graphs can broadly be categorized into two categories. The first category focuses on how LLMs can enhance prediction tasks on graphs by utilizing their ability to comprehend natural language (Fan et al., 2024b; Mao et al., 2024; Liu et al., 2024d; Li et al., 2024c; Jin et al., 2024a). These surveys examine techniques such as (i) feature augmentation, where LLMs enhance node or edge attributes; (ii) feature alignment, which bridges textual and graph representations; and (iii) structural enhancement, leveraging LLMs to model intricate graph relationships and topological patterns. These works emphasize the potential of LLMs in boosting the performance of tasks like node classification, link prediction, and graph-based recommendation systems. Surveys in another category (Pan et al., 2023, 2024; Chen, 2024) investigate how LLMs enhance traditional symbolic knowledge bases through three key approaches: (i) deploying LLMs as knowledge graph builders and controllers; (ii) leveraging structured knowledge to improve LLM pretraining; and (iii) enabling LLM-augmented symbolic reasoning to refine logical inference. With the approaches above, these methods showcase the potential of LLMs to enrich and expand the capabilities of knowledge graphs in both construction and reasoning tasks. Another important related work (Peng et al., 2024) examines the general workflow of how RAG leverages external knowledge graphs to enhance LLM predictions. Comparison with existing surveys on graph-based RAG. Existing surveys on graph-based RAG (Han et al., 2024; Peng et al., 2024) provide a valuable synthesis of the RAG workflow and have helped stabilize terminology and component boundaries. At the same time, they are largely component-centric and tend to equate graph-based RAG with knowledge-graph traversal, which narrows the view of what graphs contribute to retrieval and reasoning. This perspective gives limited attention to analysis grounded in data-management practice and offers only modest, task-level guidance for graph researchers who wish to apply graph techniques in LLM-centric pipelines. Our survey addresses these gaps along three axes while building on the strengths of prior work. First, it adopts a broader definition of graph-based RAG, encompassing graph-powered databases beyond curated knowledge graphs and clarifying how structural signals inform retrieval and prompting. Second, it provides a deeper analysis from the perspective of graph data management, treating retrieval, prompting, and pipeline topology as design problems with explicit trade-offs in accuracy, latency, token budget, freshness, and provenance. Third, it serves as a comprehensible resource for graph researchers, offering task-wise guidance and decision criteria that indicate when graph structure materially improves outcomes and how to apply it in practice. Together, these contributions complement existing surveys by shifting the emphasis from pipeline components to the roles graphs play in RAG and by translating those roles into actionable methodology. 3. Foundations of Graph-Enhanced RAG <details> <summary>figures/overview_of_RAG.jpg Details</summary>  ### Visual Description ## Flowchart: Collaboration Process for General Theory of Relativity ### Overview This flowchart illustrates the process of answering a user query about the collaboration between Albert Einstein and Marcel Grossmann in developing the general theory of relativity. It shows how a language model (LLM) retrieves historical information, processes it through prompting, and generates a response explaining their scientific partnership. ### Components/Axes 1. **User Query** (Top-left): - Text: "How did the collaboration between Albert Einstein and Marcel Grossmann lead to the development of the general theory of relativity?" - Icon: Silhouette of a person raising a hand 2. **Database** (Bottom-left): - Icon: Stacked disks representing data storage - Label: "Database" 3. **Retrieval** (Center-left): - Icon: Magnifying glass over text lines - Label: "Retrieval" - Outputs: Documents labeled "Albert Einstein" and "Marcel Grossmann" 4. **Prompting** (Center-right): - Icon: Code brackets and document pages - Label: "Prompting" - Text: "<Albert Einstein>, apply, <Riemannian geometry>, introduced by, <Marcel Grossmann>" 5. **LLM** (Bottom-right): - Icon: Snowflake symbolizing neural network - Label: "LLM" 6. **Output Response** (Top-right): - Text: "Grossmann introduced Einstein to Riemannian geometry and used his expertise in tensor calculus to help his physical ideas, providing the mathematical foundation for the general theory of relativity." - Icon: Robot head ### Detailed Analysis - **Flow Direction**: Left-to-right progression from query to response - **Key Connections**: - User query → Database → Retrieval (scientist documents) → Prompting → LLM → Output response - **Textual Elements**: - All labels and text blocks are explicitly annotated - Scientist names appear in both retrieval documents and prompting structure ### Key Observations 1. The process emphasizes historical accuracy through structured information retrieval 2. The prompting stage explicitly connects Einstein's physical concepts with Grossmann's mathematical contributions 3. The LLM synthesizes retrieved information into a coherent explanation 4. No numerical data or quantitative elements present ### Interpretation This diagram demonstrates how AI systems can: 1. Access historical scientific records 2. Contextually link interdisciplinary contributions (physics + mathematics) 3. Generate explanations that highlight collaborative scientific processes The flowchart reveals that Grossmann's expertise in Riemannian geometry and tensor calculus provided the mathematical framework Einstein needed to formalize his physical theories, illustrating how cross-disciplinary collaboration drives scientific breakthroughs. </details> Figure 1. Paradigm overview of RAG 3.1. Overall Workflow of RAG The standard RAG pipeline combines external knowledge retrieval with the generation abilities of LLMs, creating a robust pipeline for producing contextually accurate responses. We present an example for demonstration in Figure 1. In this scenario, a user query explores the development of the general theory of relativity, which involves background knowledge about scientists Albert Einstein and Marcel Grossmann. As this information may exceed the capacity of an LLM, RAG extends its scope by retrieving relevant knowledge from an external database. This process constructs a more informative prompt, thereby improving response accuracy. The complete workflow can be divided into the following four steps: (a) User query. The workflow begins with a user-provided query, typically in natural language, such as a question or request for specific information. In our running example, the user initiates a dialogue to explore the development of the general theory of relativity. Additionally, the query includes a constraint specifying that the response should focus on the collaboration between Albert Einstein and Marcel Grossmann. (b) Retrieval module. The query is passed to a retrieval system that interacts with an external knowledge database, such as a document corpus, knowledge graph, or vector store. Then, the retrieval system identifies and fetches the information most closely aligned with the query. The example query involves three key entities essential for answering the question: general theory of relativity, Albert Einstein, and Marcel Grossmann. The RAG system retrieves relevant information associated with these entities. In the context of a knowledge graph, the retrieved knowledge can be represented as triples, such as ¡ Marcel Grossmann, introduced Riemannian geometry to, Albert Einstein ¿. (c) Prompting module. The retrieved knowledge is transformed and integrated into the original query, ensuring that the model has access to up-to-date and domain-specific knowledge for generating accurate responses. Based on the key information retrieved to address the development of the general theory of relativity, the RAG system integrates external knowledge and reformats it to align with the LLM’s input style. For example, it may generate a statement such as: ” Albert Einstein applied Riemannian geometry in developing the general theory of relativity, a concept introduced to him by Marcel Grossmann during their collaboration.” This prompting strategy helps the LLM more effectively process and incorporate new information naturally into its responses. (d) Output response. Finally, the LLM leverages the enriched context to generate a response that is both accurate and highly relevant to the user’s query. By incorporating external knowledge, the model not only captures the core rationale—such as the role of Riemannian geometry in the development of the general theory of relativity —but also structures the information in a more coherent and explanatory manner. This ensures that the response not only conveys factual accuracy but also presents the historical and conceptual development of the general theory of relativity in a clearer, more logically connected way. 3.2. RAG with Graph Data Management Graphs, with their inherent ability to model complex relationships and dependencies, serve as powerful tools for structuring external knowledge and enabling sophisticated reasoning in RAG workflows. Existing literature (Jin et al., 2024a; Peng et al., 2024) primarily focuses on how LLMs enhance graph performance or provide a brief introduction of the RAG process involving graphs. However, several critical questions remain unanswered: What benefits does the graph structure offer to RAG? Which graph patterns can enhance the effectiveness or efficiency of LLMs responses? What future directions should be pursued to develop graph algorithms that synergize with LLMs? To address these compelling questions, we approach this survey explicitly from the perspective of graph data management, systematically investigating how graph techniques enhance each core stage of the RAG process, as illustrated in Figure 2. Specifically, we highlight how graph data management methods significantly (i) improve knowledge base construction; (ii) optimize retrieval and prompting algorithms through graph-based indexing, querying, and reasoning; (iii) streamline data processing pipelines via graph-structured workflows; and (iv) effectively support graph-oriented tasks. Graph-Powered Databases. In Section 4, We will explore the methods used to construct auxiliary databases for RAG. The discussion will be organized into two categories: existing knowledge graphs and the graph from the texts. For each category, we will analyze the representative graph techniques employed in existing literature and examine how they contribute to building effective auxiliary databases. This analysis will provide insights into how these databases are prepared to support subsequent retrieval tasks in RAG systems. Graph-Driven R&P (retrieval and prompting). In Section 5, we will examine the graph algorithms used in RAG from two perspectives: graph retrieval and prompting (R&P). For graph retrieval, we will categorize existing graph techniques into non-parameterized methods and learning-based methods, analyzing their effectiveness and efficiency in retrieving relevant information. For Graph Prompting, we will explore two distinct classes: Graph-Structure Prompting and Text Prompting, focusing on how they transform graph information into formats that can be utilized by LLMs. For each category, we present concrete examples to illustrate the specific techniques and their practical applications. Graph-Structured Pipelines. In Section 6, we categorize the pipelines in RAG into three distinct types: Sequential Pipeline, Loop Pipeline, and Tree Pipeline. This classification is based on the way where each method structures the overall pipeline, reflecting the underlying graph-like relationships between different module. By capturing the topological characteristics of these pipelines, we aim to shed light on how their designs influence efficiency, scalability, and reasoning capabilities. Through case studies of representative projects, we will analyze how these different pipeline facilitate the integration of retrieval and generation, ultimately enhancing the performance of LLMs in diverse scenarios. Graph-Oriented Tasks. Finally, in Section 7, we systematically categorize the applications of RAG into three key tasks: (i) knowledge graph question answering (KGQA) tasks, which involves responding to natural language queries using structured knowledge; (ii) graph-centric tasks (e.g., node classification, link prediction), leveraging reasoning property of LLMs to enhance graph reasoning; and (iii) domain-specific applications (e.g., biomedicine, finance), where graphs integrate domain knowledge to refine retrieval precision. For each category, we analyze how different RAG frameworks harness graph structures to optimize retrieval mechanisms, improve contextual relevance, and strengthen inferential accuracy. <details> <summary>figures/overview_of_graph_in_RAG.jpg Details</summary>  ### Visual Description ## Flowchart: Topological RAG Architecture ### Overview The diagram illustrates a hierarchical framework for "Topological RAG" (Retrieval-Augmented Generation), categorizing graph-based methodologies into four main branches: **Graph-Powered Database**, **Graph-Driven R&P**, **Graph-Structured Pipeline**, and **Graph-Oriented Tasks**. Each branch further subdivides into specific techniques, tools, or applications, emphasizing the integration of graph technologies in RAG systems. --- ### Components/Axes - **Main Branches** (colored in blue, orange, and purple): 1. **Graph-Powered Database** (blue) 2. **Graph-Driven R&P** (orange) 3. **Graph-Structured Pipeline** (blue) 4. **Graph-Oriented Tasks** (purple) - **Subcategories** (green bubbles with labels): - **Graph-Powered Database**: - Knowledge Graph (FreeBase, WebQSP, KG-FiD) - Text to Graph (MultiQG) - Graph Database (HipoRAG, REANO) - **Graph-Driven R&P**: - Graph Retrieval (HipoRAG, REANO) - Graph Prompting (GRAG, KCQA) - Knowledge Graph (FreeBase, WebQSP, KG-FiD) - **Graph-Structured Pipeline**: - Sequential (DALK, Keqing) - Loop (ArcaneQA, GNN-Ret) - Tree (EWEK-QA, HamQA) - **Graph-Oriented Tasks**: - Graph Task (LLMs-EP, GraphGPT) - Specific Domain (KGQA, G-Retriever, MedGraphRAG, HyKGE, Golden) --- ### Detailed Analysis - **Graph-Powered Database**: Focuses on foundational graph technologies for data storage and retrieval. Tools like **HipoRAG** and **REANO** are listed under both this branch and **Graph-Driven R&P**, suggesting their dual role in database management and retrieval/prompting workflows. - **Graph-Driven R&P**: Highlights retrieval and prompting strategies, with **Graph Prompting** (GRAG, KCQA) and **Graph Retrieval** (HipoRAG, REANO) as core components. The repetition of **Knowledge Graph** tools indicates their centrality to both database and R&P layers. - **Graph-Structured Pipeline**: Organizes processing workflows into **Sequential**, **Loop**, and **Tree** structures, each with domain-specific tools (e.g., **DALK** for sequential processing, **GNN-Ret** for loop-based retrieval). - **Graph-Oriented Tasks**: Addresses specialized applications, including **Graph Task** (LLMs-EP, GraphGPT) and **Specific Domain** (KGQA, MedGraphRAG, HyKGE). The inclusion of **Golden** (likely a reference to gold-standard datasets) underscores the importance of benchmarking. --- ### Key Observations 1. **Overlap in Tools**: Tools like **HipoRAG**, **REANO**, **FreeBase**, and **WebQSP** appear in multiple branches, indicating their versatility across database, retrieval, and prompting tasks. 2. **Hierarchical Structure**: The diagram emphasizes a layered approach, where foundational databases (e.g., **Knowledge Graph**) support higher-level processes like **Graph Prompting** and **Graph-Structured Pipelines**. 3. **Domain Specialization**: The **Specific Domain** subcategory under **Graph-Oriented Tasks** highlights tailored applications (e.g., **MedGraphRAG** for medical graphs, **KGQA** for knowledge graph question answering). --- ### Interpretation The diagram presents a comprehensive taxonomy for graph-driven RAG systems, illustrating how graph technologies are stratified into database infrastructure, retrieval/prompting strategies, processing pipelines, and domain-specific applications. The repetition of tools across branches suggests a modular design where components can be reused or adapted for different stages of the RAG workflow. For example: - **HipoRAG** and **REANO** bridge database management and retrieval, acting as both storage solutions and retrieval engines. - **Graph Prompting** (GRAG, KCQA) and **Graph Retrieval** (HipoRAG, REANO) are tightly coupled, reflecting the interdependence of data access and query formulation in RAG systems. - The **Specific Domain** subcategory underscores the need for domain adaptation, with tools like **MedGraphRAG** and **KGQA** addressing niche use cases. This framework provides a roadmap for integrating graph-based methods into RAG pipelines, emphasizing scalability, modularity, and domain-specific optimization. </details> Figure 2. RAG with graph data management 4. Graph-Powered Databases Database construction serves as a foundational step in the RAG paradigm, as it organizes and stores external knowledge to facilitate effective information retrieval. Specifically, the database in graph-based RAG captures entities and relationships from plain textual knowledge into a structured format, enabling efficient access to local (immediate neighbors or direct relationships) and global (overall connectivity or multi-hop paths) topological information. Formally, let $\mathcal{E}$ denote the set of entities and $\mathcal{R}$ denote the set of relationships; a particular fact is represented as a triplet: $\mathbf{t}=(e_{h},r,e_{t})∈\mathcal{E}×\mathcal{R}×\mathcal{E}$ , where $e_{h}$ and $e_{t}$ denote the head and tail entities, respectively, and $r$ denotes the relationship linking them. A graph database $\mathcal{G}$ thus consists of a collection of such triples: $\mathcal{G}={(e_{h},r,e_{t})}.$ In practice, various types of graph databases have been utilized in the RAG literature (summarized in Table 1), each offering distinct characteristics suited to different retrieval and reasoning tasks. In the following sections, we first introduce the existing graph databases and then discuss widely adopted techniques for generating such databases from textual data. 4.1. Existing Knowledge Graphs 4.1.1. A Unified View In this section, we focus on existing knowledge graphs, which play a crucial role in many RAG systems by serving as structured repositories of factual knowledge. To enable efficient querying and traversal of entities and their relationships, many existing methods (Hu et al., 2024; Baek et al., 2023b; Jiang et al., 2023b) directly retrieve knowledge relevant to the query from well-established knowledge graphs such as FreeBase (Bollacker et al., 2008), T-REx (ElSahar et al., 2018), and WebQSP (Yih et al., 2016). These widely used databases offer comprehensive scopes for knowledge retrieval, encompassing a vast array of entities and relationships across diverse domains. For instance, T-REx (ElSahar et al., 2018) provides extensive alignments between natural language expressions and knowledge base triples, facilitating the integration of structured data with textual information. By organizing nodes and edges along with their textual attributes, these structured repositories allow RAG systems to efficiently access additional topological information, supporting advanced operations such as entity retrieval, relationship discovery, and complex multi-hop queries. 4.1.2. Existing Knowledge Graphs Table 1. Summary of commonly used KGs and the methods constructing KG. | Categories | Pros | Cons | References | | --- | --- | --- | --- | | Existing KGs | Broad coverage, reliable quality | General knowledge, less adaptable | BioRED (Luo et al., 2022), QALD-9-plus (Perevalov et al., 2022), OpenbookQA (Mihaylov et al., 2018), CREAK (Onoe et al., 2021), TriviaQA (Joshi et al., 2017), HotpotQA (Yang et al., 2018), Mintaka (Sen et al., 2022), MedQA (Jin et al., 2020), TUDataset (Morris et al., 2020), CWQ (Talmor and Berant, 2018), Beyond I.I.D. (Gu et al., 2021), CommonsenseQA (Talmor et al., 2019), SocialIQA (Sap et al., 2019b), PIQA (Bisk et al., 2020), RiddleSense (Lin et al., 2021), Freebase (Jiang et al., 2019), ATOMIC (Sap et al., 2019a), FactKG (Kim et al., 2023b), MultiHop-RAG (Tang and Yang, 2024), T-REx (ElSahar et al., 2018), DBpedia (Auer et al., 2007a), Yago (Suchanek et al., 2007) | | Graphs generated from texts | Domain-specific, easily updated | LM-dependent, computationally intensive | GraphRAG (Edge et al., 2024), GRBK (Delile et al., 2024), ATLANTIC (Munikoti et al., 2023), GNN-Ret (Li et al., 2024f), HippoRAG (Gutiérrez et al., 2024), DALK (Li et al., 2024a), KGP (Wang et al., 2024d), OpenSCR (Han et al., 2023), MindMap (Wen et al., 2024), FABULA (Ranade and Joshi, 2023), GER (Wu et al., 2023a), FoodGPT (Qi et al., 2023), ChatKBQA (Luo et al., 2024a), MultiQG (Lan and Jiang, 2020), HSGE (Sun et al., 2023), ReTraCk (Chen et al., 2021), RNG-KBQA (Ye et al., 2021), ArcaneQA (Gu and Su, 2022), HybridRAG (Sarmah et al., 2024), EWEK-QA (Dehghan et al., 2024), KG-FiD (Yu et al., 2022), REANO (Fang et al., 2024), MedGraphRAG (Wu et al., 2024b), MINERVA (Das et al., 2018) | Freebase (Bollacker et al., 2008) is a large-scale, structured knowledge database designed to organize and store general human knowledge. It combines the efficiency and scalability of structured databases with the rich semantic representation found in sources like Wikipedia (Bridge, 2001), enabling flexible integration of diverse knowledge. By structuring information into tuples, Freebase facilitates efficient querying, entity linking, and knowledge retrieval across multiple domains. Freebase features an HTTP-based API powered by the Metaweb Query Language (MQL) (Flanagan, 2007), which facilitates intuitive object-oriented queries and schema evolution. Its complete normalization philosophy ensures unique global identifiers (GUIDs) for real-world entities, while its lightweight typing system supports multiple data. In the context of RAG, Freebase serves as a powerful auxiliary database, enabling methods to retrieve structured knowledge for enhanced reasoning and answer generation. With its ability to handle vast datasets and diverse relationships, Freebase provides a robust foundation for integrating external knowledge into RAG systems, improving their capability to process complex queries and generate context-aware responses. T-REx (ElSahar et al., 2018) is a large-scale dataset designed to align natural language from Wikipedia abstracts with knowledge base (KB) triples from Wikidata. Addressing the alignment problem, T-REx connects unstructured text with structured knowledge representations by providing 11 million triples aligned with over 3 million Wikipedia abstracts, covering more than 600 unique predicates. Its customizable alignment pipeline incorporates techniques like predicate linking, co-reference resolution, and distant supervision to create high-quality alignments. Compared to existing datasets, T-REx provides a larger scale, broader predicate coverage, and high accuracy. This dataset is instrumental in advancing tasks like relation extraction, KB population, and question answering, and serves as a foundational resource for RAG systems to retrieve structured knowledge and improve alignment for complex reasoning. <details> <summary>x1.png Details</summary>  ### Visual Description ## Timeline Diagram: Evolution of Knowledge Base Technologies ### Overview The image depicts a horizontal timeline illustrating the progression of knowledge base technologies and graph database systems from 2015 to 2025. The timeline is represented by a blue line with six circular markers connected by arrows, each pointing to a specific technology or service with associated dates. ### Components/Axes - **Horizontal Line**: Represents chronological progression (2015-2025) - **Blue Dots**: Six milestone markers - **Arrows**: Connect timeline markers to descriptive labels - **Labels**: Technology/service names with dates - **No traditional axes or legends present** ### Detailed Analysis 1. **Wikidata Query Service** (2015-09) - Leftmost marker, earliest date - Represents initial knowledge base access point 2. **Amazon Neptune** (2018-05) - Second marker, 2.5 years after Wikidata - Indicates cloud-based graph database adoption 3. **Neo4j Vector Indexes** (2023-10) - Third marker, 5 years after Neptune - Shows integration of vector search capabilities 4. **ArangoDB** (2024-03) - Fourth marker, 4 months after Neo4j - Represents multi-model database expansion 5. **GQL Graph Query Language** (2024-04) - Fifth marker, 1 month after ArangoDB - Highlights query language standardization 6. **NebulaGraph** (2024-08) - Sixth marker, 4 months after GQL - Demonstrates continued graph database development 7. **Bedrock Knowledge Bases on Neptune** (2025-03) - Rightmost marker, 7 months after NebulaGraph - Indicates AWS integration with Neptune ### Key Observations - **Temporal Gaps**: Significant intervals between some milestones (e.g., 2018-05 to 2023-10: 5 years) - **Technical Evolution**: Progression from basic query services to specialized graph technologies - **Neptune Dominance**: Two Neptune-related entries (2018 and 2025) - **Query Language Focus**: Two entries related to query systems (Neo4j indexes and GQL) - **Chronological Order**: Strict left-to-right progression without backtracking ### Interpretation The timeline demonstrates a clear trajectory in knowledge base technology development: 1. **Early Adoption Phase** (2015-2018): Establishment of foundational services (Wikidata, Neptune) 2. **Specialization Period** (2018-2023): Emergence of vector indexes and graph-specific capabilities 3. **Standardization Phase** (2023-2024): Development of query languages and multi-model databases 4. **Integration Era** (2024-2025): AWS ecosystem expansion with Bedrock Knowledge Bases Notable patterns include the increasing frequency of Neptune-related entries and the strategic timing of query language development following database implementations. The 5-year gap between Neptune (2018) and Neo4j indexes (2023) suggests a period of maturation in graph technology before specialized indexing became critical. The diagram implies a growing emphasis on graph technologies for knowledge management, with AWS Neptune serving as both an early adopter platform and later integration point for advanced knowledge bases. </details> Figure 3. Timeline of recent graph database 4.1.3. Industrial Graph Database for LLM Recently, widely used graph databases have moved from early research backends to platforms that support LLM use cases with built-in embedding search, standard query languages, and managed GraphRAG pipelines. As shown in Figure 3, we list these important databases on a timeline: Wikidata’s public query service as a common open graph backend https://query.wikidata.org/; Amazon Neptune as a managed graph service, later adding Neptune Analytics for large-scale graph and embedding analytics https://aws.amazon.com/cn/neptune/; Neo4j with mature property-graph capabilities and native vector indexes for hybrid retrieval https://neo4j.com/release/neo4j-4-0/; JanusGraph as common open-source choices for large or real-time workloads https://janusgraph.org/; ArangoDB as a multi-model system that added vector indexes https://www.arangodb.com/docs/3.12/release-notes/; vendors beginning to productize GraphRAG (for example, NebulaGraph; Bedrock Knowledge Bases on Neptune) https://www.nebula-graph.io/posts/announcing_nebulagraph_RAG; and the GQL graph query language becoming an ISO standard that aligns vendor ecosystems https://www.iso.org/standard/76120.html. 4.1.4. Trend of Graph Database Construction In the LLM era, graph databases have evolved along four directions that directly address freshness, cost of construction, and LLM integration. First, standardization has begun to reduce vendor lock-in and smooth model portability. For example, the publication of graph query language (e.g., ISO/IEC 39075:2024 https://opencypher.org/, https://www.iso.org/standard/76120.html, which formalizes a property-graph data and query language and enables more consistent schemas, queries, and tooling across systems. Second, hybrid graph–vector indexing has become more popular so that semantic similarity can be combined with graph traversal. For instance, Neo4j made vector indexes generally available and documents RAG workflows on top of them https://neo4j.com/docs/cypher-manual/5/indexes/semantic-indexes/vector-indexes/, while ArangoDB introduced native vector indexes and vector search functions of its declarative language https://docs.arangodb.com/stable/. Third, to lower update latency and construction overhead, change-data-capture (CDC) and streaming ingestion became first-class, such as Neo4j’s CDC feature and Kafka connector strategies that propagate graph updates as events, reducing the gap between source changes and graph availability for RAG/agent pipelines https://neo4j.com/blog/developer/change-data-capture-cdc-ga/. Finally, the database construction increasingly blends general encyclopedic KGs with domain KGs for higher-quality grounding in graph-based RAG systems, a trend visible in community benchmarks such as GR-Bench (Jin et al., 2024b) and GraphQA (He et al., 2024). In summary, these updates reduce data-to-serving latency and integration cost, making graph databases easier to operate at scale while preserving the multi-hop structure and provenance required by LLM applications. 4.2. Graphs Generated from Texts 4.2.1. A Unified View In addition to reusing existing knowledge graphs, we consider a broader class of graph-powered databases constructed from text. A common route is transforming plain text into a knowledge graph through artificial means, which is known as open information extraction (OpenIE) (Angeli et al., 2015; Etzioni et al., 2008; Zhou et al., 2022). This process extracts key information—such as entities, relationships, and contextual meanings—from textual documents. An instruction-tuned language model then identifies these entities and their relationships to generate structured triples. As illustrated in Figure 4, given a paragraph describing the collaboration between Albert Einstein and Marcel Grossmann, the instruction-tuned LM captures the relationships among entities and converts them into structured triples. In longer contexts, multiple relationships emerge, linking entities to various neighbors and ultimately forming a complex graph. Beyond vanilla knowledge graphs, we also include multiple forms graphs distilled from text: (1) document-level structures such as entity–sentence and sentence–sentence links (HippoRAG (Gutiérrez et al., 2024), ATLANTIC (Munikoti et al., 2023), GNN-Ret (Li et al., 2024f), OpenSCR (Han et al., 2023), FiD (Yu et al., 2022), MedGraphRAG (Wu et al., 2024b)); (2) hierarchical graphs with each node as a subgraph (DALK (Li et al., 2024a), GER (Wu et al., 2023a)); (3) rule-based graph where each node aggregates multiple entities or facts consistent with a defined rule schema (FABULA (Ranade and Joshi, 2023), StarRAG (Zhu et al., 2025), ReTraCk (Chen et al., 2021)). These constructions produce property graphs with additional information and traceable evidence, enabling unique feature extractions from the knowledge base. <details> <summary>x2.png Details</summary>  ### Visual Description ## Diagram: Collaboration and Knowledge Transfer in the Development of the General Theory of Relativity ### Overview The image is a conceptual diagram illustrating the intellectual and professional relationships between mathematician Marcel Grossmann and physicist Albert Einstein during their collaboration at Zurich Polytechnic. It highlights the transfer of specialized knowledge (Riemannian Geometry and Tensor Calculus) and its application in developing the General Theory of Relativity. ### Components/Axes - **Nodes**: - **Text Box (Left)**: Describes Grossmann’s expertise in tensor calculus and Riemannian geometry, their collaboration at Zurich Polytechnic, and Einstein’s application of Riemannian geometry to develop the General Theory of Relativity. - **Diagram (Right)**: - **Concepts**: Tensor Calculus, Riemannian Geometry, General Theory of Relativity. - **People**: Grossmann, Einstein. - **Institutions**: Zurich Polytechnic. - **Arrows**: - **Directional Relationships**: - "specialized_in" (e.g., Grossmann → Tensor Calculus, Grossmann → Riemannian Geometry). - "collaborated_with" (Grossmann ↔ Einstein). - "studied_at" (Einstein → Zurich Polytechnic). - "developed" (Einstein → General Theory of Relativity). - "learned_from" (Einstein ← Riemannian Geometry). ### Detailed Analysis - **Text Box Content**: - Grossmann specialized in tensor calculus and Riemannian geometry. - Collaborated with Einstein at Zurich Polytechnic. - Einstein learned Riemannian geometry from Grossmann and applied it to develop the General Theory of Relativity. - **Diagram Relationships**: - Grossmann is centrally connected to Tensor Calculus and Riemannian Geometry via "specialized_in" arrows. - Einstein is linked to Grossmann via "collaborated_with" and to Riemannian Geometry via "learned_from." - Einstein is connected to Zurich Polytechnic via "studied_at" and to the General Theory of Relativity via "developed." ### Key Observations 1. **Central Role of Grossmann**: Grossmann’s expertise in Riemannian Geometry is the foundational node, with bidirectional influence on Einstein. 2. **Knowledge Flow**: Einstein’s development of the General Theory of Relativity is directly tied to learning Riemannian Geometry from Grossmann. 3. **Institutional Context**: Zurich Polytechnic serves as the shared environment enabling their collaboration. ### Interpretation The diagram underscores the critical role of interdisciplinary collaboration in scientific breakthroughs. Grossmann’s mathematical expertise in Riemannian Geometry provided Einstein with the conceptual framework necessary to formulate the General Theory of Relativity. The bidirectional arrows between Grossmann and Einstein emphasize mutual intellectual exchange, while the institutional node (Zurich Polytechnic) anchors their partnership. This visualizes how specialized knowledge transfer and collaborative problem-solving can lead to paradigm-shifting theories. **Note**: The image contains no numerical data, charts, or quantitative trends. All information is qualitative and derived from textual labels and relational arrows. </details> Figure 4. From texts to knowledge graph 4.2.2. Graph Generation Methods Based on Texts KG-FiD (Yu et al., 2022) is a graph-based framework that constructs knowledge graphs directly from textual data to facilitate more effective information retrieval. Specifically, it begins by extracting text segments from the dataset and linking them to their associated entities. Using the BERT model for passage encoding, KG-FiD computes similarity scores and reranks the passages within entities to enhance retrieval accuracy. Additionally, a Graph Attention Network (GAT) is utilized to propagate information across the graph, leveraging relationships between entities to retrieve relevant node information. This approach ensures that the graph representation is both contextually rich and informative, leading to improved retrieval outcomes. MultiQG (Lan and Jiang, 2020) proposes a method to answer multi-hop complex queries over knowledge bases by generating query graphs with constraints. The approach integrates constraints early in the process to guide the construction of query graphs, effectively reducing the search space. Candidate query graphs are ranked based on their embedding similarity with the question using a neural network, ensuring accurate alignment. By combining beam search and semantic matching, the method attains state-of-the-art performance on standard benchmark datasets like ComplexWebQuestions, demonstrating significant improvements in precision and F1 scores. This framework addresses the limitations of traditional RAG systems, which struggle to handle constraints effectively, improving accuracy in complex question. 4.3. Comparison between Existing and Generated Knowledge Graphs Both existing knowledge graphs and text-derived graphs serve as essential components of graph-powered databases, each with distinct advantages and limitations. Existing knowledge graphs offer structured, high-precision information with efficient querying, making them well-suited for tasks requiring fast and reliable entity retrieval. However, they are inherently static and require manual updates to incorporate new knowledge. In contrast, graphs constructed from text provide greater adaptability by dynamically extracting relationships from unstructured data. While this approach enhances knowledge coverage and supports evolving information, it introduces potential workload due to reliance on language models and probabilistic extraction methods. Moving forward, hybrid approaches that integrate raw graphs with text-derived structures could offer a balanced solution, leveraging both efficiency and adaptability. Additionally, improving incremental learning for knowledge graph updates, enhancing entity-linking accuracy, and developing scalable retrieval mechanisms will be crucial for optimizing graph-powered databases. 5. Graph-Driven R&P In the context of RAG, graphs play a crucial role in both retrieval and prompting (R&P) for LLMs– enabling the retrieval of structured knowledge and facilitating the prompting of LLMs. Specifically, graph retrieval refers to the algorithms designed to extract additional knowledge from the graph, which serves to enrich the reasoning capabilities of the LLMs. This process involves traversing the graph to find relevant nodes, edges, or subgraphs that provide valuable context or information necessary to enhance the accuracy and depth of the model’s responses. On the other hand, graph prompting focuses on transforming the retrieved graph structure into a textual format that is easily understood by the LLMs. This transformation ensures that the complex relationships and connections within the graph are effectively communicated in natural language, enabling the LLMs to leverage the structured knowledge in its generative process. In the subsequent sections, we explore a range of graph-based retrieval and prompting techniques, highlighting representative approaches. The broader set of approaches, along with our classification, is presented in Table 2 and Table 3. Table 2. Summary of retrieval algorithms. | Retrieval Algorithms | Pros | Cons | References | | | --- | --- | --- | --- | --- | | Deterministic Algorithms | Precise, reliable | Intensive computation | KG-GPT (Kim et al., 2023a), GLBK (Delile et al., 2024), GRAG (Hu et al., 2024), HyKGE (Jiang et al., 2024c), Knowledge Solver (Feng et al., 2023), KnowledGPT (Wang et al., 2023c), GGE (Gao et al., 2022), Engine (Zhu et al., 2024c), GraphBridge (Wang et al., 2024c), MMGCN (Wei et al., 2019), NLGraph (Wang et al., 2024a), GraphEval2000 (Wu et al., 2024c) | | | Non-parameterized Algorithms | Probabilistic Algorithms | Scalable, strong adaptability | Less precise, less interpretable | HippoRAG (Gutiérrez et al., 2024), MindMap (Wen et al., 2024), OreoLM (Hu et al., 2022), MultiQG (Lan and Jiang, 2020), MINERVA (Das et al., 2018), MuseGraph (Tan et al., 2024a), Walklm (Tan et al., 2024b) | | Heuristic-Based Algorithms | Efficient, scalable | Uncertain results depend on heuristics | RA-SIM (Peng and Yang, 2024), KG-Rank (Yang et al., 2024a), SKP (Dong et al., 2023a), NuTrea (Choi et al., 2023), Zeshel (Logeswaran et al., 2019), Conll (Hoffart et al., 2011), RNG-KBQA (Ye et al., 2021), ArcaneQA (Gu and Su, 2022), HybridRAG (Sarmah et al., 2024),MedGraphRAG (Wu et al., 2024b), TOG2 (Ma et al., 2024), DepsRAG (Alhanahnah et al., 2024), KELP (Liu et al., 2024b), KGQA (Sen et al., 2023), Graph-LLM (Chen et al., 2024c), GLBench (Li et al., 2024d) | | | Learning-based Algorithms | Convolutional-based Algorithms | Efficient local structure modeling | Limited global information | GNN-RAG (Mavromatis and Karypis, 2024), MHKG (Feng et al., 2020), RA-SIM (Peng and Yang, 2024), GenKGQA (Gao et al., 2024b), GRAG (Hu et al., 2024), GNN-Ret (Li et al., 2024f), ETD (Liu et al., 2024a), NuTrea (Choi et al., 2023), KAM-CoT (Mondal et al., 2024), ConvE (Dettmers et al., 2018), REANO (Fang et al., 2024), SURGE (Kang et al., 2022), GraphGPT (Tang et al., 2024) | | Attention-Based Algorithms | Global structure modeling | High computational cost | ATLANTIC (Munikoti et al., 2023), OpenSCR (Han et al., 2023), GGE (Gao et al., 2022), GER (Wu et al., 2023a), HSGE (Sun et al., 2023), HamQA (Dong et al., 2023b), KG-FiD (Yu et al., 2022), Fact (Opsahl, 2024) | | 5.1. Retrieval with Non-parameterized Algorithms 5.1.1. A Unified View The non-parameterized methods in the retrieval process of RAG focus on extracting relevant knowledge from the graph using predefined rules, without involving any trainable parameters. Specifically, given the query $q$ and the knowledge graph $\mathcal{G}$ , the retrieval response $R_{q}$ from the graph can be formulated as: $R_{q}=f(\mathcal{G},e(q)),$ where $e(·)$ is the entity or relationship extraction function and $f(·)$ is a deterministic function that follows predefined rules. To minimize overlaps of these deterministic functions and ensure the uniqueness of each group, we categorize the methods into the following three classes: Deterministic Graph Algorithms, Probabilistic Graph Algorithms, and Heuristic-Based Graph Algorithms. We outline the characteristics of these methods below and provide a visual representation in Figure 5. $\bullet$ Deterministic Graph Algorithms. These algorithms focus on providing exact and reliable solutions to graph-related problems, ensuring correctness and precision. They prove especially effective for tasks requiring strict guarantees, such as subgraph isomorphism detection, shortest path computation, and graph traversals (e.g., Dijkstra’s algorithm (Dijkstra, 2022), Bellman-Ford algorithm (Bellman, 1958)). Their main advantage lies in their accuracy, but they may incur high computational costs for large-scale graphs. Figure 5 illustrates an example of the exact path computed between two nodes, highlighting how deterministic algorithms ensure optimality in path finding. $\bullet$ Probabilistic Graph Algorithms. These methods leverage probabilistic and statistical techniques to approximate graph properties or retrieve relevant information about entities and relationships. Instead of guaranteeing exact results, they offer efficient solutions for tasks where full precision is unnecessary or infeasible. Personalized PageRank, Markov Chains, and Monte Carlo methods are common examples, excelling in applications such as node ranking, influence estimation, and recommendation systems. These methods trade off determinism for scalability and adaptability in dynamic or uncertain environments. Figure 5 illustrates an example of a probabilistic random walk, where transitions between nodes occur with a probability $p$ , demonstrating how these algorithms explore graph structures in a stochastic manner. $\bullet$ Heuristic-Based Graph Algorithms. This category mainly includes algorithms designed to provide efficient approximate solutions, especially in cases where exact methods are computationally prohibitive. By using heuristic-based strategies or domain-specific insights, they improve scalability while maintaining reasonable accuracy. Examples include greedy algorithms for approximate subgraph matching, K-hop sampling, and subgraph sampling techniques, which are widely applied in community detection, graph partitioning, and large-scale knowledge retrieval. These methods prioritize computational feasibility and speed over absolute correctness. Figure 5 presents a toy example of 1-hop sampling, demonstrating how nodes efficiently gather information from their immediate neighbors. By removing the trainable parameters in the algorithms, non-parameterized methods are relatively efficient in identifying informative subgraphs that provide valuable context for reasoning. These methods are particularly effective in scenarios where the structure of the knowledge graph is well understood, and simple traversal or ranking techniques can yield meaningful insights. Their simplicity and lack of model parameters make them easy to implement and computationally lightweight, allowing for quick and direct retrieval of relevant graph-based information to support the reasoning of LLMs. <details> <summary>x3.png Details</summary>  ### Visual Description ## Diagram Type: Model Architectures Comparison ### Overview The image presents five distinct computational model architectures arranged horizontally, each labeled with its approach: Deterministic, Probabilistic, Heuristic-Based, Convolutional-based, and Attention-Based. Each diagram uses nodes (circles) and directional connections (arrows) to represent relationships, with varying annotations for probabilities, weights, and contextual elements. ### Components/Axes - **Labels**: - Deterministic - Probabilistic - Heuristic-Based - Convolutional-based - Attention-Based - **Annotations**: - Probabilistic diagram includes edge probabilities: `p = 0.2`, `p = 0.3`, `p = 0.5`. - Convolutional-based diagram labels connections as `C_ux` (input-to-hidden) and `C_u3` (hidden-to-output). - Attention-Based diagram labels attention weights as `A_ux1`, `A_ux2`, `A_ux3`. - **Visual Elements**: - Dashed green lines in Deterministic and Heuristic-Based diagrams suggest contextual or indirect relationships. - Solid arrows indicate direct connections. - Central nodes (blue) in all diagrams represent core processing units. ### Detailed Analysis 1. **Deterministic**: - Central node connects to four peripheral nodes via solid arrows. - Dashed green line loops around the central node and two peripheral nodes, implying a contextual or feedback mechanism. 2. **Probabilistic**: - Central node (blue) connects to two peripheral nodes (white and purple) with labeled probabilities: - `p = 0.2` (white → blue) - `p = 0.3` (blue → white) - `p = 0.5` (blue → purple) - Probabilities suggest stochastic transitions between states. 3. **Heuristic-Based**: - Central node connects to three peripheral nodes via solid arrows. - Dashed green line encloses the central node and two peripheral nodes, indicating heuristic rules or contextual grouping. 4. **Convolutional-based**: - Central node (`x_u`) connects to three peripheral nodes (`x1`, `x2`, `x3`) via labeled connections: - `C_ux` (input-to-hidden) - `C_u3` (hidden-to-output) - Arrows emphasize hierarchical feature processing. 5. **Attention-Based**: - Central node (`x_u`) connects to three peripheral nodes (`x1`, `x2`, `x3`) with attention weights: - `A_ux1`, `A_ux2`, `A_ux3` - Bidirectional arrows (e.g., `A_ux1` and `A_ux2`) suggest dynamic weighting of inputs. ### Key Observations - **Progressive Complexity**: Models evolve from simple deterministic flows (Deterministic) to probabilistic transitions (Probabilistic), heuristic rules (Heuristic-Based), feature hierarchies (Convolutional-based), and dynamic attention mechanisms (Attention-Based). - **Central Node Role**: All architectures share a central node as the primary processing unit, but its connectivity and contextual relationships vary. - **Probabilistic Uncertainty**: The Probabilistic diagram explicitly quantifies uncertainty via edge probabilities, contrasting with deterministic certainty. - **Attention Dynamics**: The Attention-Based model introduces asymmetric, weighted connections, reflecting adaptive focus on inputs. ### Interpretation The diagrams illustrate a spectrum of computational strategies: - **Deterministic** models rely on fixed, rule-based logic. - **Probabilistic** models incorporate uncertainty, useful for stochastic environments. - **Heuristic-Based** models use simplified rules for complex problems. - **Convolutional-based** models prioritize hierarchical feature extraction (e.g., in neural networks). - **Attention-Based** models dynamically allocate computational resources to relevant inputs, enhancing efficiency and adaptability. The progression highlights increasing sophistication in handling uncertainty, context, and resource allocation. The Attention-Based model’s bidirectional arrows suggest a feedback loop, enabling iterative refinement—a key advantage in tasks like natural language processing or image recognition. </details> Figure 5. Non-parameterized Algorithms and learning-based for Graph Retrieval 5.1.2. Methods Using Deterministic Graph Algorithms GLBK (Delile et al., 2024) recognizes entities in the question, computes shortest paths that connect them in the knowledge graph, and retrieves text linked to the nodes and incident edges along these paths, which reveals indirect relations that support answer synthesis. HyKGE (Jiang et al., 2024c) aligns query and hypothesis entities to a medical graph and then collects three deterministic reasoning chains between anchor entities, namely simple paths, co-ancestor chains, and co-occurrence chains, followed by pruning that keeps the most relevant connections as retrieval evidence. GNN-RAG (Mavromatis and Karypis, 2024) first scores candidate answers with graph neural reasoning and then extracts the shortest paths from the question entities to these candidates so that the retrieved paths provide faithful and efficient multi hop evidence. KG-GPT (Kim et al., 2023a) segments the question into triple level units, matches entities to the graph through semantic alignment, identifies the relations that connect them, and assembles a subgraph that serves as compact retrieval support for downstream reasoning. 5.1.3. Methods Using Probabilistic Graph Algorithms Within the taxonomy in Table 2, probabilistic graph algorithms cast retrieval as stochastic evidence flow on the graph. Walklm (Tan et al., 2024b) introduces targeted random walk sampling from query anchored nodes, which turns the visited trails into textual sequences and aggregates path evidence to rank candidate neighborhoods and passages. HippoRAG (Gutiérrez et al., 2024) illustrates diffusion-driven retrieval, where detected query entities are aligned to knowledge graph nodes and used as seeds for Personalized PageRank, which spreads probability through nearby structure and then applies specificity-based reweighting to select high probability neighborhoods and surface the associated passages for multi-hop reasoning. 5.1.4. Methods Using Heuristic-Based Graph Algorithms HybridRAG (Sarmah et al., 2024) pairs vector search over the document store with a one-hop neighborhood walk from entities detected in the question, which enriches the retrieved context and improves answer quality while keeping the search path simple. GRAG (Hu et al., 2024) illustrates structured candidate selection, where K-hop ego graphs are ranked by embedding similarity to the query. Then a learnable pruner suppresses low-value nodes and edges, and the top candidates are merged into a compact subgraph for generation, which preserves semantic focus and graph topology under tight runtime budgets. 5.2. Retrieval with Learning-based Algorithms 5.2.1. A Unified View In contrast to non-parameterized approaches, the learning-based retrieval method can be formulated as a parameterized function $f_{\theta}(·)$ that learns to retrieve relevant knowledge by optimizing an objective function: $R_{q}=f_{\theta}(\mathcal{G},e(q)),$ where $\theta$ represents the learnable parameters within $f_{\theta}(·)$ . These parameters are trained on a dataset using an optimization objective, enabling the retrieval mechanism to adapt dynamically to different data distributions and retrieval tasks. Unlike traditional rule-based retrieval, which relies on predefined heuristics, learning-based approaches can generalize across domains, capture complex semantic relationships, and improve retrieval relevance over time. Depending on how the system processes and extracts information from the knowledge graph, learning-based retrieval algorithms can be categorized into the following two major classes: $\bullet$ Convolutional-based Algorithms. Convolutional-based algorithms are designed to exploit the structural properties of graphs by performing neighborhood aggregation through trainable network layers. These methods are inspired by convolutional operations in image processing but are adapted to graphs to capture local connectivity patterns and facilitate information propagation across nodes. A representative example is the Graph Convolutional Network (GCN) (Kipf and Welling, 2017), which employs a layer-wise message-passing mechanism to iteratively aggregate features from neighboring nodes. As shown on the LHS of Figure 5, given the central node $x_{u}$ and its neighbors $x_{1}\sim x_{3}$ , different weights $C_{uu}$ and $C_{u1}\sim C_{u3}$ are applied to aggregate the features from each neighbor. This weighted aggregation allows the central node to incorporate diverse contextual information, leading to richer feature representations. By stacking multiple layers, these algorithms allow retrieval models to learn hierarchical representations, enhancing their ability to extract relevant knowledge from graph-structured data. $\bullet$ Attention-Based Algorithms. Attention-based algorithms in graph retrieval have gained significant interest due to their ability to dynamically prioritize the most relevant parts of the graph during learning. Unlike traditional approaches that treat all neighboring nodes equally, these methods employ attention mechanisms to assign varying importance weights ( $A_{uu}$ and $A_{u1}\sim A_{u3}$ in RHS of Figure 5) to nodes and edges, enabling the model to focus on key relationships that are most critical for a given task. A prominent example is the Graph Attention Network (GAT) (Veličković et al., 2018), which introduces node-wise attention coefficients to aggregate information more selectively. This adaptive weighting mechanism allows attention-based retrieval models to capture long-range dependencies, refine contextual understanding, and improve retrieval precision. 5.2.2. Methods Using Convolutional-based Algorithms REANO (Fang et al., 2024) is a framework designed to leverage structural information in knowledge graphs to enhance the retrieval of triple features by employing an L-hop graph convolution. Before the retrieval step, REANO uses the TAGME model (Ferragina and Scaiella, 2010) to extract entities and then builds relationships among these entities through both intra-context and inter-context manners. Subsequently, an $L$ -hop GNN is applied to update entity representations, allowing REANO to select the top- $k$ triple candidates based on their similarity to the given question. By incorporating the topological structure over multiple hops within the knowledge graph, REANO gathers critical information needed to answer questions, effectively alleviating the reasoning burden of the answer predictor and improving the overall efficiency of the QA process. SURGE (Kang et al., 2022) is a framework designed to extract context-relevant knowledge from knowledge graphs while enforcing consistency across facts to ensure that the generated responses faithfully reflect the retrieved knowledge corresponding to the question. To achieve this, SURGE leverages an existing edge message passing framework (Jo et al., 2021), which transforms the edges of the original graph into nodes within a dual hypergraph (Scheinerman and Ullman, 2011). This transformation enables the use of existing node-level GNNs to represent the relationships within the original graph. Subsequently, SURGE samples negative subgraphs and employs contrastive learning, encouraging the model to generate responses that are consistent with the positive subgraph, thereby improving the fidelity of the generated content. By enforcing consistency through contrastive learning, SURGE ensures that its responses remain faithful to the retrieved knowledge, enhancing the reliability of the generated answers. 5.2.3. Methods Using Attention-Based Algorithms HSGE (Sun et al., 2023) studies the complex interactions between the entities of the knowledge base by reasoning in the history semantic graph, which is built by employing a pre-trained language model BERT (Devlin et al., 2019) on conversation contexts. A fundamental method of HSGE to retrieve the structure information in the history semantic graph is to update the node embeddings with the attention-based module such as TransformerConv (Shi et al., 2020). In order to further adapt to the change triggered by the new-coming conversations, HSGE encodes the position of each historical interaction and utilizes the attention mechanism again to aggregate the most relevant information for each mentioned entity in the query. Compared with concatenating all previous conversation turns, this paradigm is more computationally efficient, while still retaining key context information needed for effective question answering. HamQA (Dong et al., 2023b) intends to identify the importance of different relationships between the entities by emphasizing the information most relevant for addressing the question. Specifically, HamQA formulates the weights of each edge in the KG utilizing a learnable function combined with the entity representations. Moreover, considering the geometrically hierarchical features between different entities, HamQA further employs Hyperbolic distances (Balazevic et al., 2019) to measure the importance of different neighbors. These two strategies enable HamQA to effectively calculate the attention scores when incorporating the adjacent information and serve as the constraints to guide graph propagation toward more significant messages. Table 3. Summary of prompting methods. | Prompting Method | Pros | Cons | Reference | | --- | --- | --- | --- | | Topology-Aware Prompting | Preserves structure, enables multi-hop reasoning | Complex formatting, hard to understand for LLMs | LLaGA (Chen et al., 2024b), GNN-RAG (Mavromatis and Karypis, 2024), G-Retriever (He et al., 2024), GRAPH-COT (Jin et al., 2024b), ROG (Luo et al., 2024b), RRA (Wu et al., 2023b), MVP-Tuning (Huang et al., 2023b), Keqing (Wang et al., 2023a), GRAG (Hu et al., 2024), ETD (Liu et al., 2024a), DALK (Li et al., 2024a), MindMap (Wen et al., 2024), Lark (Choudhary and Reddy, 2024), REALM (Zhu et al., 2024b), HybridRAG (Sarmah et al., 2024), EWEK-QA (Dehghan et al., 2024), TOG2 (Ma et al., 2024), SURGE (Kang et al., 2022), GraphGPT (Tang et al., 2024), NLGraph (Wang et al., 2024a), HyKGE (Jiang et al., 2024c), HamQA (Dong et al., 2023b), GraphEval2000 (Wu et al., 2024c), MuseGraph (Tan et al., 2024a), GOT (Besta et al., 2024a) | | Text Prompting | Simple implementation, compatible with LLM input format | Loses structured information, limited reasoning ability | Talk (Fatemi et al., 2023), GPT4Graph (Guo et al., 2023), UniOQA (Li et al., 2024e), FABULA (Ranade and Joshi, 2023), ODA (Sun et al., 2024b), KnowGPT (Zhang et al., 2024c), Kaping (Baek et al., 2023b), MedGraphRAG (Wu et al., 2024b), Golden-Retriever (An et al., 2024), KALMV (Baek et al., 2023c), Fact (Opsahl, 2024), DepsRAG (Alhanahnah et al., 2024), KELP (Liu et al., 2024b), KGQA (Sen et al., 2023), Graph-LLM (Chen et al., 2024c), GLBench (Li et al., 2024d), Walklm (Tan et al., 2024b) | 5.3. Comparison between Different Retrieval Methods Non-parameterized and learning-based retrieval methods offer distinct advantages and trade-offs in graph-driven retrieval. Non-parameterized approaches rely on explicit graph traversal, ranking, or sampling techniques, making them computationally efficient and interpretable. They work particularly well in structured knowledge graphs, where exact or approximate methods can effectively retrieve relevant subgraphs without requiring training data. However, their performance is often limited by predefined heuristics and an inability to adapt dynamically to different retrieval contexts. In contrast, learning-based retrieval methods leverage trainable models to capture richer graph representations through message passing or attention mechanisms, allowing for more adaptive and context-aware retrieval. These methods excel at modeling complex dependencies but require substantial computational resources and labeled data for training. Moving forward, future research may focus on hybrid retrieval strategies that integrate the efficiency of non-parameterized methods with the adaptability of learning-based approaches. Additionally, improving self-supervised learning for graph retrieval can reduce dependence on labeled data, enhancing the availability in real-world applications. 5.4. Topology-Aware Prompting 5.4.1. A Unified View Topology-aware prompting captures the topological essence of a knowledge graph by explicitly encoding nodes, edges, and their relationships into structured prompts that models capable of directly processing structured inputs can readily interpret. These featured prompting methods can be divided into following categories: (1) path prompting that encodes relation chains or shortest paths to expose composition (GRAPH-COT (Jin et al., 2024b), ROG (Luo et al., 2024b), TOG2 (Ma et al., 2024), MindMap (Wen et al., 2024)); (2) subgraph prompting that linearizes compact subgraphs for information summarization (G-Retriever (He et al., 2024), DALK (Li et al., 2024a), GRAG (Hu et al., 2024), LLaGA (Chen et al., 2024b)); (3) rule-aware prompting that introduces relation types and declarative constraints to guide reasoning (HyKGE (Jiang et al., 2024c), HamQA (Dong et al., 2023b)). Instead of providing simple textual descriptions, these prompts explicitly represent relationships using structured formats such as triple statements (e.g., ” (Albert Einstein, born in, Ulm) ”) or relational paths (e.g., ” Paris → capital of → France → located in → Europe ”). A key motivation behind graph-structure prompting is to allow the model to reason more effectively about multi-hop and complex relational patterns (Ma et al., 2024). For instance, rather than treating the graph as a mere collection of facts, the model can examine path-based relationships to draw inferences. This is especially valuable in scenarios where the desired answer depends on understanding connections among multiple entities or interpreting intricate graph substructures. Moreover, this approach can boost explainability by making the model’s reasoning process more transparent. When each relationship or path is clearly defined, users can trace how the system derived its conclusions, thereby increasing trust and clarity in the model’s output. In summary, graph-structure prompting offers a powerful mechanism for harnessing a graph’s rich relational data, allowing RAG-based systems to deliver more reliable answers and deeper insights. 5.4.2. Methods Applying the Topology-Aware Prompting Keqing (Wang et al., 2023a) utilizes graph-structured prompting to enhance KGQA tasks by guiding LLMs through complex multi-hop questions. It achieves this by employing two distinct prompting strategies to help LLMs understand and reason about the logical relationships in retrieved triplets from a KG. The first strategy explicitly explains the triplet structure in the prompt, detailing the format as (subject, relation, object), and ensuring that responses align strictly with the entities within the triplets. The second strategy converts triplets into plain text by serializing their components, making them more digestible for fine-tuned models like LLaMA 2 (Touvron et al., 2023). These graph-structured prompts act as a chain-of-thought (COT) mentor for LLMs, enabling them to decompose complicated queries into simpler sub-questions, retrieve relevant entities through logical chains, and generate accurate and interpretable responses. By leveraging the structured nature of KGs and tailored prompting techniques, Keqing demonstrates improved reliability and interpretability in KGQA tasks, highlighting its potential as a scalable solution for knowledge-intensive reasoning. Both RoG (Luo et al., 2024b) and ToG (Sun et al., 2024a) generate reasoning paths from KGs to enhance RAG prompting by structuring retrieval and reasoning. RoG focuses on generating relation-grounded paths, ensuring faithful and structured reasoning by retrieving valid reasoning paths from KGs. This approach enhances accuracy and consistency while providing interpretable results that improve trust and understanding. Similarly, ToG identifies relevant entities from a query and explores the KG by searching for meaningful triples. During reasoning, the LLM evaluates the retrieved triples and selects the most valuable ones to construct a reasoning path. This method offers explicit and editable reasoning paths, improving explainability and allowing users to trace and correct model outputs when necessary. Both approaches strengthen RAG by integrating structured retrieval with reasoning, ensuring more coherent, interpretable, and contextually relevant responses. 5.5. Text Prompting 5.5.1. A Unified View In the cases where the machine struggles to understand the graph-structure prompt, text prompting involves transforming the structured graph knowledge with language models into a linear, textual representation that the LLMs can easily understand. After graph retrieval, the nodes, relationships, and properties are converted into descriptive sentences or paragraphs that capture the essence of the original graph while making it compatible with natural language input formats. The focus of text prompting is on providing a human-like narrative that conveys the important details of the graph. Text prompting ensures that even complex graph structures are expressed in a format that a typical LLMs can understand, thus allowing the LLMs to utilize the graph-based information without requiring specialized graph-processing capabilities. 5.5.2. Methods Applying the Text Prompting MedGraphRAG (Wu et al., 2024b) follows a general, reusable pattern of graph-based RAG: graph building, graph retrieval, prompting, and refinement. In the build step, an LLM reads medical documents, extracts entities, links relations, and keeps short evidence or definition snippets so that each node and edge is grounded in source text. At query time, the system embeds the user question and graph entities, selects the top-k most similar entities, and expands a small working subgraph around them for focused context. The model then generates a draft answer from this subgraph and iteratively improves it by feeding the draft back into the LLM for a few rounds, stopping at a preset threshold and summarizing low-value regions to remain efficient. This simple loop—evidence-linked graph construction, embedding-based top-k retrieval with local expansion, and short refinement—captures a general template for reliable, source-aware graph-grounded QA in high-stakes domains. KCQA (Sen et al., 2023) builds a framework that efficiently retrieves and prompts knowledge from a KG to answer questions. Initially, KCQA estimates the distribution of relations in the KG using the Rigel model (Oliya et al., 2021) and subsequently predicts relevant triples within two hops. To focus on the most relevant information, it retains only the top-K triples of the estimated relations, achieved through a Hadamard (element-wise) product to extract a weighted vector representation of the triples. These selected triples are then converted into natural language for prompting. In the prompting process, KCQA also considers inverse triples and utilizes a unified template to compose the prompt, which is then fed as the input into a language model generating an answer. 5.6. Comparison between Different Prompting Methods The strengths and limitations of topology-Aware prompting and text prompting map naturally to task demands. For multi-hop question answering (Tang and Yang, 2024), personalized recommendation (Deldjoo et al., 2024), temporal reasoning (Zhu et al., 2025), and program synthesis over graphs (Gu and Su, 2022), topology-aware prompting is preferable because it preserves edge types and path structure, enforces constraints, and yields auditable chains. However, as shown in Table 3, it brings heavy formatting overhead, larger token budgets, schema canonicalization burdens, and can be harder for general LLMs to parse, making it brittle under noisy graphs (Hadi et al., 2023). By contrast, for abstractive summarization over documents attached to the graph (Edge et al., 2024), for broad retrieval where lexical recall dominates (Choi et al., 2023), and for user-facing explanations that prioritize fluency (Wang et al., 2023c), text prompting is the better choice. It is simple to implement, aligns with standard LLM inputs, remains robust under schema drift, and achieves lower latency, but it sacrifices explicit path identity, weakens constraint satisfaction, and limits compositional reasoning depth (Mao et al., 2024). 6. Graph-Structured Pipelines 6.1. A Unified View <details> <summary>figures/pipeline_graph.jpg Details</summary>  ### Visual Description ## Flowchart: Information Retrieval and Response Generation System ### Overview The diagram illustrates a multi-stage process for handling user queries through information retrieval, prompting, and language model (LLM) interaction. It includes three pipeline types (sequential, loop, tree) represented by different arrow styles. ### Components/Axes **Nodes (Process Stages):** 1. **User Query** (Green circle with person icon) - Top-left 2. **Retrieval** (Blue circle with search icon) - Center-top 3. **Prompting** (Gray circle with document icon) - Right-center 4. **LLM** (Yellow circle with robot icon) - Bottom-right 5. **Output Response** (Purple circle with robot icon) - Bottom-center **Arrows (Pipeline Types):** - **Sequential Pipeline** (Solid gray arrows) - **Loop Pipeline** (Dashed red arrows) - **Tree Pipeline** (Dotted blue arrows) **Legend Position:** Bottom of diagram, clearly labeling arrow types ### Detailed Analysis 1. **User Query** (Green) initiates the process, connected via: - Sequential pipeline (solid gray) to **Retrieval** - Loop pipeline (dashed red) back to itself (self-loop) 2. **Retrieval** (Blue) connects to: - Sequential pipeline to **Prompting** - Loop pipeline back to **User Query** 3. **Prompting** (Gray) connects to: - Tree pipeline (dotted blue) to **LLM** - Sequential pipeline to **Output Response** 4. **LLM** (Yellow) connects via: - Sequential pipeline to **Output Response** 5. **Output Response** (Purple) has no outgoing connections ### Key Observations - The system features three distinct pipeline architectures: - **Sequential Flow**: Main path from User Query → Retrieval → Prompting → LLM → Output Response - **Loop Mechanism**: Allows iterative refinement between User Query ↔ Retrieval and Retrieval ↔ User Query - **Branching Path**: Prompting → LLM represents a decision point with potential parallel processing - No numerical data points present; the diagram focuses on process flow rather than quantitative metrics ### Interpretation This flowchart represents a hybrid information processing system combining: 1. **Iterative Refinement**: The loop pipeline between User Query and Retrieval suggests a feedback mechanism for query optimization 2. **Parallel Processing**: The tree pipeline from Prompting to LLM indicates potential for concurrent model interactions 3. **Linear Execution**: The sequential pipeline forms the primary execution path through the system The architecture implies a balance between: - **Efficiency** (through sequential processing) - **Adaptability** (via loop-based query refinement) - **Complexity Handling** (through tree-based model interactions) The absence of quantitative metrics suggests this is a conceptual workflow rather than a performance benchmark. The system design prioritizes flexibility in handling different query types and processing requirements. </details> Figure 6. Pipeline formulation with graph structure Based on the pipeline of RAG, we can visualize the different steps in the pipeline as nodes, and the processes or transitions between them as edges. Each component in the RAG system can be represented in this graph-based structure, and the relationships between components can be defined based on their interactions. Particularly, we formulate each specific step as a node in the path of the directed graph as: (i) $e_{1}$ : user query; (ii) $e_{2}$ : graph retrieval; (iii) $e_{3}$ : graph prompting; (iv) $e_{4}$ : large language model generation; (v) $e_{5}$ : output response. Moreover, we define the the edge $r_{i,j}$ (from node $e_{i}$ to node $e_{j}$ ) as: (i) $r_{1,2}$ : passing the user query to the graph retrieval step; (ii) $r_{2,3}$ : processing the graph retrieval output for prompting; (iii) $r_{3,4}$ : passing the graph-structured or textual prompt to the LLMs; (iv) $r_{4,5}$ : generating the final output response from LLMs. In this configuration, the various steps and their transitional passages form a natural topological structure. As illustrated in Figure 6, we categorize the existing pipelines into three main types. (i) Sequential Pipeline. In reference to the figure, the sequential pipeline follows the path $e_{1}\overset{r_{1,2}}{\longrightarrow}e_{2}\overset{r_{2,3}}{\longrightarrow}e_{3}\overset{r_{3,4}}{\longrightarrow}e_{4}\overset{r_{4,5}}{\longrightarrow}e_{5}$ , mirroring the standard RAG paradigm shown in Figure 1. This approach represents a linear flow, wherein each step immediately succeeds the previous one, and no node is revisited. (ii) Loop Pipeline. In this variant, certain stages incorporate feedback mechanisms or iterative processes. For instance, the graph retrieval step may be repeated if the initial retrieval fails to supply adequate context, or the LLMs might require additional prompts to refine their responses. We capture this behavior with a cyclic graph. In Figure 6, the final output can also be routed back to earlier steps, either merging with the user query ( $e_{5}\overset{r_{5,1}}{\longrightarrow}e_{1}$ ) or informing another retrieval phase ( $e_{5}\overset{r_{5,2}}{\longrightarrow}e_{2}$ ). (iii) Tree Pipeline. This paradigm is designed for scenarios in which multiple components of the pipeline can execute in parallel. The graph splits into multiple branches, allowing different retrieval or prompting strategies to occur simultaneously. In the following sections, we introduce representative methods employing these different pipeline designs and summarize a widely investigated classification in Table 4. Table 4. Summary of graph-structured pipelines. | Pipelines | Pros | Cons | Reference | | --- | --- | --- | --- | | Sequential Pipeline | Simple and efficient execution | Fixed order, no refinement or error correction | GNN-RAG (Mavromatis and Karypis, 2024), GrapeQA (Taunk et al., 2023), SR (Zhang et al., 2022), RRA (Wu et al., 2023b), KG-GPT (Kim et al., 2023a), QA-GNN (Yasunaga et al., 2021), GLBK (Delile et al., 2024), ATLANTIC (Munikoti et al., 2023), UniOQA (Li et al., 2024e), GRAG (Hu et al., 2024), HyKGE, (Jiang et al., 2024c), KG-Rank (Yang et al., 2024a), KnowledGPT (Wang et al., 2023c), ETD (Liu et al., 2024a), HippoRAG (Gutiérrez et al., 2024), DALK (Li et al., 2024a), KnowGPT (Zhang et al., 2024c), Engine (Zhu et al., 2024c), GraphBridge (Wang et al., 2024c), ChatKBQA (Luo et al., 2024a), NuTrea (Choi et al., 2023), etc | | Loop Pipeline | Iterative refinement, error correction | Higher computational cost, low efficiency | GraphRAG (Edge et al., 2024), GRAPH-COT (Jin et al., 2024b), KnowledgeNavigator (Guo et al., 2024), KD-COT (Wang et al., 2023b), KG-Agent (Jiang et al., 2024a), GNN-Ret (Li et al., 2024f), Knowledge Solver (Feng et al., 2023), KGP (Wang et al., 2024d), FABULA (Ranade and Joshi, 2023), ODA (Sun et al., 2024b), QR-LLM (Ma et al., 2023), MufassirQAS (Alan et al., 2024), ArcaneQA (Gu and Su, 2022), MedGraphRAG (Wu et al., 2024b), KALMV (Baek et al., 2023c) | | Tree Pipeline | Parallel exploration, multi-path reasoning | Higher computational cost | EWEK-QA (Dehghan et al., 2024), HamQA (Dong et al., 2023b), SURGE (Kang et al., 2022), TOG2 (Ma et al., 2024), GOT (Besta et al., 2024a), Beyond-CoT (Yao et al., 2023) | 6.2. Controllers of Pipeline Implementation The controller is the decision-making mechanism that formalizes our pipeline topologies at inference time, which chooses when to retrieve, whether to iterate or branch, when to verify and revise, and when to stop. Different controller families emphasize sequential, iterative loop, and branching tree pipelines, each with distinct tradeoffs among accuracy, latency, and computational cost. In what follows, we present four widely adopted controller families in RAG systems and show how representative methods instantiate each family within our pipeline design. Reinforcement-Learning Controllers. A reinforcement-learning (RL) controller is a learned policy that, given the current reasoning state, selects the next pipeline action (retrieve, expand, verify, revise, or stop) to maximize a reward that trades off answer quality and computational cost. For example, RAG-RL (Huang et al., 2025) post-trains the generator with RL so it learns to select and cite useful contexts from larger retrieved sets, which tightens the coupling between retrieval and generation and effectively learns the loop and tree branching rules for exploration and pruning. Multi-module RL has also been used to coordinate several RAG components jointly (query rewriter, retriever, selector, generator), as in MMOA-RAG (Chen et al., 2025), which frames the pipeline as cooperative agents optimized toward a unified reward and thus implements a learned loop controller and branch selection over candidate contexts. System-2 Slow-Thinking Controllers. A system-2 controller is an inference-time search policy that allocates more computation than a single forward pass, which naturally realizes loop and tree pipelines without updating model weights. For instance, Self-Ask (Press et al., 2022) decomposes a question into sub-questions, interleaves retrieval during the reasoning process, and thus implements a loop that alternates “ask–retrieve–answer” over a constrained subgraph. FLARE (Jiang et al., 2023c) generalizes this idea to long-form generation by forecasting upcoming content, triggering retrieval only when predicted segments are low confidence, and regenerating with fresh evidence. Verifier-Guided Controllers. A verifier-guided controller attaches an external evaluator that scores partial solutions or retrieved sets and feeds decisions back to the pipeline, thereby governing revision and selection. Self-RAG (Asai et al., 2024) trains a model to retrieve, generate, and critique using reflection tokens. During inference, it adaptively decides whether to retrieve again, accept or revise a segment, and produce citations, which instantiates a loop controller with verify–revise cycles over the current subgraph. CRAG (Yan et al., 2024) adds a lightweight retrieval evaluator that estimates the quality of retrieved documents and triggers corrective actions, such as re-retrieval or web fallback, effectively steering the loop and pruning poor branches in a tree of candidate evidence. Budget-/Cost-Aware Controllers. A budget-aware controller explicitly optimizes an accuracy–latency objective by selecting per-query computation and retrieval budgets, thereby choosing whether a sequence suffices or whether a loop or tree is warranted.Adaptive-RAG (Jeong et al., 2024) predicts query complexity and selects among no-retrieval, single-step retrieval, or iterative retrieval strategies, implementing a controller that upgrades a sequence pipeline into a loop only when complexity demands. Conversational gating methods such as RAGate (Wang et al., 2024b) estimate, at each turn, whether external augmentation is needed and hence decide to skip or invoke retrieval, providing a cost-aware switch between sequence and loop behavior under compute or latency constraints. 6.3. Methods Applying the Sequential Pipeline Keqing (Wang et al., 2023a) adopts a sequential pipeline to address complex multi-hop question answering. It decomposes complex queries into simpler sub-questions using predefined templates. These sub-questions are then aligned with logical chains on a knowledge graph (KG), guiding the retrieval of candidate entities through multi-hop reasoning. A candidate reasoning module evaluates and selects the correct entities from the KG. Finally, the system generates responses by aggregating reasoning paths and answers to provide an interpretable and precise answer to multi-hop questions. QA-GNN (Yasunaga et al., 2021) integrates LMs and KGs for question answering with a sequential pipeline. The process begins by encoding the QA context including the question and answer options with an LM and retrieving a KG subgraph relevant to the question. A joint ”working graph” is constructed by connecting the QA context node to entities within the KG subgraph. To enhance reasoning, the KG nodes are scored for relevance based on their alignment with the QA context using LM-based scoring. An attention-based GNN performs iterative message passing on this working graph, updating both KG entities and QA context representations. Finally, predictions are made by aggregating the LM output, updated node features, and the working graph representation, achieving accurate and explainable results. DALK (Li et al., 2024a) makes LLMs and KGs benefit from each other through an elaborate sequential pipeline design. At the beginning of the pipeline, LLMs process the scientific corpus and construct two domain-specific KGs from the related literature via pair-wised relation extraction and generative relation extraction, respectively. This part of the pipeline can be seen as LLMs benefit KGs. Then, DALK utilizes a coarse-to-fine sampling method and a retrieval approach to select knowledge from KGs. The selected knowledge as well as the domain questions are sent to LLMs to generate accurate answers, therefore realizing that KGs benefit LLMs. 6.4. Methods Applying the Loop Pipeline RGNN-Ret (Li et al., 2024f) tackles multi-hop reasoning questions by iteratively decomposing the reasoning process into manageable steps. It incorporates a self-critique mechanism that prompts LLMs to generate sub-questions for each reasoning step. After answering a sub-question with retrieved passages, the LLM evaluates whether the accumulated evidence is sufficient to generate a final answer or if further reasoning steps are required. This iterative process enables the system to dynamically adapt to the complexity of the query. To enhance the retrieval quality across reasoning steps, a Recurrent Graph Neural Network (RGNN) is used. The RGNN integrates the graphs of passages from previous reasoning steps, establishing connections between retrieved passages to ensure that relationships between sub-questions and evidence are maintained. By combining semantic distances and contextual information across steps, the RGNN reduces the impact of incorrect sub-questions and improves retrieval accuracy for supporting passages. The iterative loop continues until the self-critique determines that enough evidence has been collected to answer the question. GRAPH-COT (Jin et al., 2024b) provides a benchmark to enhance LLM reasoning on graphs through iterative steps of reasoning, interaction, and execution. At each iteration, the LLM identifies the required information and formulates graph interactions to retrieve relevant data, such as nodes or relationships. These interactions are executed on the graph, and the results are fed back into the reasoning process. This cycle repeats until the LLM concludes the reasoning task and produces the final answer. By iteratively refining its understanding of graph structures, GRAPH-COT effectively handles multi-hop reasoning tasks, ensuring accurate and explainable results in graph-based question answering. 6.5. Methods Applying the Tree Pipeline SURGE (Kang et al., 2022) effectively leverages a tree pipeline by structuring its retrieval and generation processes into multiple parallel branches, allowing different retrieval and prompting strategies to execute simultaneously. Instead of sequentially retrieving a single set of facts, SURGE retrieves multiple context-relevant subgraphs in parallel, each representing different aspects of the dialogue history. These subgraphs are processed simultaneously through distinct embedding pathways, ensuring diverse knowledge representations. The model then applies its invariant graph encoding technique to each retrieved subgraph independently, maintaining permutation and relation-inversion invariance across multiple branches. Additionally, during response generation, SURGE integrates these parallel knowledge streams using graph-text contrastive learning, ensuring that the generated response remains consistent across all retrieved knowledge sources. This parallel execution not only enhances efficiency but also enables the model to synthesize information from multiple perspectives, improving response accuracy and informativeness in knowledge-grounded dialogue systems. GoT (Besta et al., 2024a) effectively leverages a tree pipeline by expanding an initial prompt into multiple parallel branches of “thoughts,” each branch pursuing a distinct partial solution under a predefined Graph of Operations. Rather than committing to a single chain, GoT repeatedly applies a generate–score–select routine: candidate branches are spawned with a controllable branching factor, locally validated and scored, and then pruned via a keep-best policy to preserve only the most promising subtrees. This staged branching decomposes the task into tractable subproblems that can be solved concurrently within separate branches, while refinement loops allow individual branches to self-correct without restarting the whole pipeline. Crucially, GoT then fuses the surviving branches using explicit aggregation operators, turning the pure tree into a directed acyclic workflow that consolidates partial results into a coherent final output. By combining parallel expansion, principled pruning, iterative refinement, and late-stage aggregation, GoT maintains the efficiency benefits of a tree pipeline across the entire process while achieving higher solution quality and better cost–latency trade-offs than sequential methods. Beyond-CoT (Yao et al., 2023) leverages a tree-style pipeline by expanding a single query into multiple concurrent branches that persist across two stages—rationale generation and answer inference. First, an Extract–Cluster–Coreference (ECC) routine builds a branched thought graph from the input and optional image caption, effectively fanning out the problem into many node-level sub-traces. Each branch is then encoded in parallel: text via a Transformer encoder, the thought graph via a graph attention network, and vision via a dedicated encoder. Cross-attention aligns tokens with graph nodes, and a gated-fusion layer aggregates these parallel branches into a unified representation that the decoder uses to produce rationales. In stage two, the pipeline re-branches by concatenating the predicted rationales back to the input, reconstructing the thought graph, and repeating the same parallel encoders, alignment, and gated fusion before decoding the final answer. This end-to-end design preserves the efficiency and diversity benefits of a tree pipeline—branching for exploration, parallel encoding for coverage, and late fusion for consolidation—while generalizing the tree into a graph to capture non-linear dependencies. 6.6. Comparison between Different Pipelines Different pipelines offer varying degrees of flexibility and efficiency, influencing how information flows through RAG systems. Sequential pipelines provide a straightforward, stepwise execution, ensuring clarity and simplicity but limiting adaptability when intermediate refinements are needed. In contrast, loop pipelines introduce feedback mechanisms, allowing iterative refinement of retrieved knowledge or prompt modifications, making them well-suited for tasks requiring multiple rounds of reasoning. However, they can introduce computational overhead due to repeated processing. Tree pipelines, on the other hand, enable parallel execution of different retrieval or prompting strategies, improving efficiency in handling diverse queries but potentially increasing system complexity. A future trend of pipelines is moving toward adaptive pipeline selection, where the system dynamically determines the optimal pipeline structure based on the query’s complexity and reasoning needs (Jiang et al., 2025). This strategy can enable more efficient, context-aware retrieval and generation while minimizing unnecessary computational overhead. 7. Graph-Oriented Tasks 7.1. KGQA Tasks 7.1.1. A Unified View Knowledge Graph Question Answering (KGQA) targets answering natural language queries by leveraging the structured information encoded in KGs. Unlike traditional question-answering methods that rely solely on text-based retrieval, KGQA focuses on grounding its reasoning in the relationships and entities represented within a knowledge graph. By utilizing the rich semantic structure of KGs, which capture entities, their attributes, and the interrelationships between them, KGQA systems aim to provide precise and contextually accurate answers. This task not only requires understanding the natural language query but also necessitates navigating the graph’s structure to extract relevant facts, perform logical inferences, and deliver a well-supported response. In the ensuing section, we offer in-depth discussions of select noteworthy works, and a more extensive overview of additional works can be found in Figure 5. To better summarize these methods, we leave the brief introduction of them in the Appendix. 7.1.2. Methods Targeting the KGQA Tasks G-Retriever (He et al., 2024) is designed to improve KGQA by addressing challenges in reasoning, scalability, and accuracy for real-world textual graphs. It introduces a new benchmark of graph-based question answering, enabling models to handle complex, multi-domain questions beyond basic graph reasoning tasks. A key innovation of this benchmark is formulating the retrieval of subgraph as a Prize-Collecting Steiner Tree optimization (Bienstock et al., 1993) problem, ensuring relevant information is extracted while maintaining explainability. G-Retriever integrates GNNs and LLMs for fine-tuned reasoning and achieves superior performance across domains such as knowledge graphs, scene graphs, and common-sense reasoning. By offering a conversational interface for graph queries, it advances the explainability, efficiency, and usability of KGQA tasks. This makes it a benchmark in advancing graph-based question-answering systems. GraphRAG (Edge et al., 2024) enhances KGQA by integrating traditional RAG with graph-based summarization, addressing global queries that traditional RAG struggles to resolve. It builds an LLM-derived knowledge graph that uses community detection algorithms, such as Leiden, to group related entities and relationships into modular communities. These communities are pre-summarized, allowing for efficient multi-hop reasoning and improved retrieval accuracy. For answering queries, partial responses from community summaries are combined into a global answer through a map-reduce approach. This method improves comprehensiveness, diversity, and scalability, enabling precise, contextually rich responses for large-scale datasets with significantly reduced computational overhead. Table 5. Summary of different tasks. The brief introduction of the listed methods can be found in the appendix. | Tasks | Reference | | --- | --- | | KGQA Tasks | HSGE (Sun et al., 2023), ReTraCk (Chen et al., 2021), RNG-KBQA (Ye et al., 2021), ArcaneQA (Gu and Su, 2022), EWEK-QA (Dehghan et al., 2024), HamQA (Dong et al., 2023b), KG-FiD (Yu et al., 2022), Golden-Retriever (An et al., 2024), SURGE (Kang et al., 2022),KELP (Liu et al., 2024b), Fact (Opsahl, 2024), TOG2 (Ma et al., 2024), MINERVA (Das et al., 2018), KALMV (Baek et al., 2023c),REANO (Fang et al., 2024),LLaGA (Chen et al., 2024b), GNN-RAG (Mavromatis and Karypis, 2024), G-Retriever (He et al., 2024), GraphRAG (Edge et al., 2024), ROG (Luo et al., 2024b), SR (Zhang et al., 2022), RRA (Wu et al., 2023b), KnowledgeNavigator (Guo et al., 2024), KG-GPT (Kim et al., 2023a), KD-COT (Wang et al., 2023b), QA-GNN (Yasunaga et al., 2021), MVP-Tuning (Huang et al., 2023b), StructGPT (Jiang et al., 2023a), MHKG (Feng et al., 2020), MHGRN (Chakraborty, 2024), Temple-MQA (Cheng et al., 2024), KagNet (Lin et al., 2019), KG-Agent (Jiang et al., 2024a), GenKGQA (Gao et al., 2024b), GLBK (Delile et al., 2024), Keqing (Wang et al., 2023a), UniKGQA (Jiang et al., 2023b), GRAG (Hu et al., 2024), Knowledge Solver (Feng et al., 2023), WebQSQ (Yih et al., 2016), MetaQA (Zhang et al., 2018), PullNet (Sun et al., 2019), KnowledGPT (Wang et al., 2023c), ETD (Liu et al., 2024a), DecAF (Yu et al., 2023),HippoRAG (Gutiérrez et al., 2024), GGE (Gao et al., 2022), NSM (He et al., 2021), SKP (Dong et al., 2023a), KnowGPT (Zhang et al., 2024c), Difar (Baek et al., 2023a), Kaping (Baek et al., 2023b), NuTrea (Choi et al., 2023), OreoLM (Hu et al., 2022) | | Graph Tasks | GPT4Graph (Guo et al., 2023), GraphText (Zhao et al., 2023a), LLaGA (Chen et al., 2024b), GRAPH-COT (Jin et al., 2024b), Engine (Zhu et al., 2024c), GraphBridge (Wang et al., 2024c), MMGCN (Wei et al., 2019),ConvE (Dettmers et al., 2018), Graph-LLM (Chen et al., 2024c), GraphGPT (Tang et al., 2024), NLGraph (Wang et al., 2024a), GLBench (Li et al., 2024d), GraphEval2000 (Wu et al., 2024c), MuseGraph (Tan et al., 2024a), Walklm (Tan et al., 2024b) | | Domain-specific Tasks | GLBK (Delile et al., 2024), ATLANTIC (Munikoti et al., 2023), HyKGE (Jiang et al., 2024c), KG-Rank (Yang et al., 2024a), DALK (Li et al., 2024a), KGP (Wang et al., 2024d), MindMap (Wen et al., 2024), FABULA (Ranade and Joshi, 2023), FoodGPT (Qi et al., 2023), KAM-CoT (Mondal et al., 2024), MufassirQAS (Alan et al., 2024), REALM (Zhu et al., 2024b), MEDQA (Jin et al., 2020), HybridRAG (Sarmah et al., 2024), MedGraphRAG (Wu et al., 2024b), DepsRAG (Alhanahnah et al., 2024) | 7.2. Graph Tasks 7.2.1. A Unified View By incorporating LLMs into graph task execution, the retrieval of relevant subgraphs or neighboring nodes can improve the accuracy of tasks like node classification, link prediction, and graph classification. These tasks benefit from the reasoning capabilities of LLMs, which can understand complex relationships within the graph. Specifically, by utilizing the LLM’s ability to process and reason over graph structures, predictions become more accurate, allowing for better insights and performance across a variety of graph-based applications. 7.2.2. Methods Targeting the Graph-centric Tasks RAGRAPH (Jiang et al., 2024b) performs node classification, link prediction, and graph classification tasks. By retrieving and leveraging the most relevant subgraphs, RAGRAPH passes information within subgraphs to enhance the representation of the center node, facilitating various graph learning tasks. LitFM (Zhang et al., 2024a) considers link prediction (citation link prediction, paper recommendation) and text generation (title generation, abstract completion, citation sentence generation) tasks. GraphGPT (Tang et al., 2024) aims to align LLMs with graph learning tasks through a graph instruction tuning paradigm. The framework focuses on integrating graph structural knowledge with LLMs to enhance generalization in both zero-shot and supervised settings. GraphGPT includes a text-graph grounding component that aligns graph structures with natural language, allowing LLMs to comprehend complex graph structures. This alignment is achieved through a dual-stage instruction tuning process: first, self-supervised instruction tuning is employed to provide structural knowledge, and second, task-specific instruction tuning is used to enhance adaptability across various graph tasks, including node classification and link prediction. GraphGPT also leverages COT distillation to improve step-by-step reasoning abilities, making it effective at addressing the challenges of zero-shot learning in graph-based tasks. 7.3. Domain-specific Tasks GraphRAG can be effectively employed in diverse domains such as academia, e-commerce, scientific literature, healthcare, and legislation due to its ability to integrate powerful language generation models with retrieval-based knowledge. Healthcare. Existing literature leverages RAG to efficiently summarize long-form medical documents, enhancing LLMs’ ability to generate accurate and evidence-based responses. For instance, MedGraphRAG (Wu et al., 2024b) serves as a dedicated framework for utilizing graph-based RAG in healthcare applications. Its retrieval process involves generating a tag-summary on the user query, identifying the most relevant graph through similarity-based selection, and refining responses. HyKGE (Jiang et al., 2024c) is a framework leveraging RAG based on knowledge graphs in LLM-empowered medical applications. To improve medical consultation quality, HyKGE integrates a granularity-aware reranking module to eliminate noise while preserving diversity-relevance balance in retrieved knowledge. REALM (Zhu et al., 2024b) enhances the clinical predictive capabilities by integrating RAG with multimodal Electronic Health Records (EHR). It enhances the utilization of EHR data in healthcare and reconciles it with the intricate medical context necessary for educated clinical forecasts. Scientific Literature. By harnessing the capabilities of GraphRAG, it becomes possible to navigate the vast expanse of scientific literature, extracting key insights and condensing complex ideas and high-quality knowledge. For example, DALK (Li et al., 2024a) constructs a specific knowledge graph sourced from Alzheimer’s Disease scientific literature. The essential knowledge is filtered through a coarse-to-fine sampling algorithm so that it can offer valuable insights. ATLANTIC (Munikoti et al., 2023) introduces a retrieval-augmented language model that incorporates structural awareness for interdisciplinary scientific tasks, integrating heterogeneous document graphs to capture structural relationships among scientific documents. By fusing textual and structural embeddings, ATLANTIC enhances the retrieval of coherent and faithful passages. KGP (Wang et al., 2024d) creates a KG over multiple kinds of literature and employs an LLM-guided traversal agent to retrieve and synthesize relevant information for answering complex queries. Code Completion. In code completion tasks, using a graph with the RAG system helps capture and represent the complex relationships between code elements, such as control flow, data dependencies, and function calls. For example, GraphCoder (Liu et al., 2024e) is an RAG framework designed to improve code completion by integrating repository-specific knowledge into code LLMs. Unlike traditional methods, it uses a Code Context Graph (CCG), which captures relationships like control flow and data dependence. The framework employs a two-step retrieval process: coarse-grained filtering to find candidate code snippets, followed by fine-grained re-ranking to prioritize those with aligned dependencies. This combination of structural and lexical context leads to more relevant code snippets, improving the accuracy of code generation. GraphCoder outperforms baseline methods, achieving higher code and identifier exact matches with lower time and storage costs. Its language-agnostic design, tested on Python and Java, proves its versatility across different programming environments. Biomedical Question Answering. Biomedical knowledge is specialized and constantly changing, making it hard for traditional language models to provide accurate, up-to-date answers without access to external, structured information. GLBK (Delile et al., 2024) aims to tackle the information overload problem in biomedical question answering. The primary objective is to retrieve and prioritize relevant biomedical documents from a vast corpus, such as PubMed, while addressing complex, open-ended queries like identifying drug targets for diseases. Unlike traditional methods relying solely on embedding similarity, this approach integrates knowledge graph structures to mitigate biases from overrepresented topics and improve access to the long tail of biomedical knowledge. By leveraging entity recognition, relationship extraction, and graph-based rebalancing, the method enhances retrieval precision and ensures more contextually relevant information is surfaced. 7.4. Evaluation of Tasks Open-source benchmark datasets. Following the taxonomy above, we also group the open-source benchmark datasets into three representative families (summarized in Table 6). For KGQA, WebQ (Berant et al., 2013) and WebQSP (Yih et al., 2016) focus on entity linking and compositional queries over knowledge graphs, while HotpotQA (Yang et al., 2018) emphasizes multi-hop aggregation and GrailQA (Gu et al., 2021) targets schema generalization across unseen relations. Recent temporal KGQA corpora—CronQuestion (Saxena et al., 2021), Forecast (Ding et al., 2023), and MultiTQ (Chen et al., 2023) —introduce explicit time constraints to test era-aware retrieval and timestamp-valid answers. For graph-centric tasks, RelBench (Robinson et al., 2024) provides standardized node, edge, and graph prediction suites. TUDataset (Morris et al., 2020) offers diverse graph classification collections, and GLBench (Li et al., 2024d) together with GraphEval2000 (Wu et al., 2024c) concentrate on LLM-oriented graph reasoning and program execution. In domain-specific settings, MedQA (Jin et al., 2020) and PubMedQA (Jin et al., 2019) address clinically grounded QA with domain terminology and long-context evidence. BioRED (Luo et al., 2022) targets biomedical entity and relation extraction, and AlphaFin (Li et al., 2024b) represents financial analysis and forecasting under realistic market dynamics. Table 6. Evaluation of different tasks. | Tasks | Metrics | Benchmark datasets | | --- | --- | --- | | KGQA Tasks | Exact Match, F1 Score, SCR, Accuracy, ROUGE, BLEU, METEOR, TTFT | SimpleQuestion (Bordes et al., 2015), WebQ (Berant et al., 2013), CWQ (Talmor and Berant, 2018), MetaQA (Zhang et al., 2018), CronQuestion (Saxena et al., 2021), Forecast (Ding et al., 2023), MultiTQ (Chen et al., 2023), GraphQA (He et al., 2024), GRBENCH (Jin et al., 2024b), CRAG (Yang et al., 2024c), NQ (Kwiatkowski et al., 2019), KILT (Petroni et al., 2021), MultiHop (Tang and Yang, 2024), CRUD-RAG (Lyu et al., 2024), CREAK (Onoe et al., 2021), HotpotQA (Yang et al., 2018), TriviaQA (Joshi et al., 2017), FACTKG (Kim et al., 2023b), WebQSP (Yih et al., 2016), GrailQA (Gu et al., 2021), | | Graph Tasks | Accuracy, F1 Score, MRR, NDCG, Hit Ratio, MAE, RMSE | RelBench (Robinson et al., 2024), TabGraphs (Bazhenov et al., 2024), HiTab (Cheng et al., 2021), TUDataset (Morris et al., 2020), GLBench (Li et al., 2024d), GPT4Graph (Guo et al., 2023), CS-TAG (Yan et al., 2023), GraphEval2000 (Wu et al., 2024c) | | Domain-specific Tasks | Accuracy, F1 Score, MRR, ROUGE, RMSD, Cumulative Return | STARK (Wu et al., 2024e), AlphaFin (Li et al., 2024b), MedQA (Jin et al., 2020), BioRED (Luo et al., 2022), QALD (Perevalov et al., 2022), Mintaka (Sen et al., 2022), TutorQA (Yang et al., 2024b), UltraDomain (Qian et al., 2024), PubMedQA (Jin et al., 2019) | Evaluation metrics. We summarize widely used metrics across the three task families and note that Accuracy and F1 score are ubiquitous core measures for them. For answer quality in, Exact Match (EM), token-level F1 score, and Accuracy are standard, with Recall-Oriented Understudy for Gisting Evaluation (ROUGE), Bilingual Evaluation Understudy (BLEU), and Metric for Evaluation of Translation with Explicit ORdering (METEOR) applied when systems produce long-form justifications. For ranking and retrieval in graph reasoning, Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (NDCG), and Hits@k are additionally reported. In domain-specific tasks, biomedical information extraction typically uses Accuracy and micro or macro F1; biology-oriented structure comparison relies on Root Mean Square Deviation (RMSD); and finance-oriented studies commonly track Cumulative Return, Sharpe Ratio, Maximum Drawdown and so on. Efficiency and stability are often captured with latency measures such as Time to First Token (TTFT) and robustness measures such as Self-Consistency Rate (SCR). The complete introduction of each metric is detailed in the appendix. 7.5. Importance of Graph in Different Task Families We finally examine the advantages of graph-based RAG over non-graph retrieval across three task families. (a) For simple factoid KGQA, a non-graph retriever often suffices because the answer typically resides in a single passage with little need for disambiguation. For complex KGQA, graphs add clear value by grounding entities and relations, composing multi-hop paths before generation, enforcing type and temporal constraints, constraining search to relevant neighborhoods, and preserving provenance for verification. These capabilities are instantiated by the Prize Collecting Steiner Tree module in G-Retriever, which selects an evidence subgraph aligned to the question, and the Personalized PageRank diffusion module in HippoRAG, which prioritizes neighborhoods seeded by detected entities to guide ranking and multi-hop reasoning. (b) For graph-centric tasks such as node classification and link prediction, performance hinges on local topology, relation types, and neighborhood signals. Graph-based RAG contributes by retrieving a compact subgraph anchored on the target nodes or node pairs, preserving adjacency and type constraints so the generator reasons over dense, topology-aware evidence within a small token budget. This reduces distraction from long passages, improves label and edge consistency, and supports clear provenance. These advantages are exemplified by subgraph message passing modules in RAGRAPH, which strengthen center node representations for node and link prediction, and the feature&text level enhancement modules in LLMs- EP, which inject two-hop structural signals to improve node classification. (c) Domain-specific tasks demand high accuracy and transparent explanations. Graph-based RAG strengthens accuracy by supporting loop and tree pipelines that let the model revisit, branch, and refine retrieval and generation across multiple passes. It also advances explainability by representing structured knowledge beyond classic knowledge graphs, including document, citation, clinical, and code graphs, so the system returns subgraphs with explicit entities, relations, and paths as verifiable evidence. These advantages are illustrated by the traversal agent in KGP, which iteratively expands and consolidates evidence on literature graphs, and the shortest-path retrieval in GLBK, which surfaces indirect biomedical relations with clear provenance. 8. Graph-based RAG in dynamic scenarios Graph-based RAG in dynamic scenarios increasingly couples time-aware indexing with retrieval that respects event order and temporal constraints. T-GRAG (Li et al., 2025a) constructs a time-stamped knowledge graph and performs layered retrieval that first isolates time-relevant subgraphs, then filters candidate nodes and finally selects fine-grained facts for generation, thereby avoiding cross-time conflicts. DyG-RAG (Sun et al., 2025) defines dynamic event units with explicit temporal anchors and links them into an event graph, retrieving coherent timelines and guiding generation with time-aware chain-of-thought. STAR-RAG (Zhu et al., 2025) compresses temporal regularities into a rule graph and applies seeded personalized PageRank to prioritize a time-aligned neighborhood that matches a query’s temporal intent. TimeRAG (Yang et al., 2025) builds a time series knowledge base and retrieves reference sequences via Dynamic Time Warping before prompting an LLM forecaster with the retrieved patterns. TS-Retriever (Wu et al., 2024a) improves retrieval for time-sensitive questions by training a contrastive, temporally aware retriever with query-side fine-tuning and routing, so that queries with implicit or interval constraints are matched to correctly dated evidence. DRAGIN (Su et al., 2024) triggers retrieval adaptively during generation and formulates queries to reflect the model’s immediate information needs, inserting external knowledge only when beneficial. RAG4DyG (Wu et al., 2024f) retrieves temporally and contextually similar demonstrations from across a dynamic graph and fuses them with the query sequence to adapt to evolving interaction patterns. DynaGRAG (Thakrar, 2024) enhances subgraph representations through de-duplication, two-step pooling, query-aware retrieval, and dynamic similarity–aware traversal to prioritize diverse yet relevant subgraphs as evidence. GenTKG (Gao et al., 2024a) retrieves ordered historical events from a temporal knowledge graph as instruction-style context to support generative forecasting of future links. Collectively, these methods show a convergent design: encode temporal structure explicitly, retrieve evidence that is locally time-consistent, and compose temporally ordered contexts so that generation remains faithful to evolving knowledge. 9. Practical Applications in Industry Industrial applications of graph-based RAG can be grouped into two tiers: production RAG systems and the workflow–platform stacks. The production tier includes Microsoft’s GraphRAG https://github.com/microsoft/graphrag, which builds entity-based knowledge graphs and pre-generated community summaries to improve query-focused summarization; NebulaGraph’s GraphRAG https://www.nebula-graph.io/posts/graph-RAG, which integrates large-language-model reasoning with the NebulaGraph database; Eosphoros’s GraphRAG https://github.com/eosphoros-ai/DB-GPT, which combines DB-GPT (Zhou et al., 2024a), OpenSPG https://github.com/OpenSPG/openspg, and TuGraph https://tugraph.tech/ for triple extraction, subgraph location, and traversal; Neo4j’s NaLLM https://github.com/neo4j/NaLLM, which enables natural-language interfaces to knowledge graphs, construction from unstructured data, and report generation; Writer https://writer.com/, which utilizes graph-based RAG to enhance enterprise content generation and Q&A systems; and Leroy Merlin https://www.lettria.com/case-study/leroy-merlin-knowledge-graph-product-recommendations, which uses product knowledge graphs to refine recommendation systems, providing personalized customer experiences. The workflow–platform tier comprises LangChain https://www.langchain.com/ and LangGraph https://langchain-ai.github.io/langgraph for graph-structured pipelines; LlamaIndex’s Agentic Document Workflows https://www.llamaindex.ai/blog/agentic-rag-with-llamaindex-2721b8a49ff6 for end-to-end document processing and meta-agent coordination; multi-agent frameworks such as CrewAI https://github.com/crewAIInc/crewAI and AG2 AutoGen https://github.com/ag2ai/ag2; and cloud platforms including Vertex AI https://cloud.google.com/vertex-ai, Amazon Bedrock https://aws.amazon.com/cn/bedrock/, and IBM watsonx https://www.ibm.com/products/watsonx. The Usage of Graph-based RAG in Frontier LLMs. Frontier LLMs such as GPT-5.2 https://openai.com/index/introducing-gpt-5-2/ and Gemini 3 Pro https://ai.google.dev/gemini-api/docs/gemini-3 make graph-based RAG more practical because they are stronger at long-context understanding, tool selection, and multi-step planning. Therefore they can reliably act as controllers and verifiers in retrieval pipelines. In particular, both ecosystems now provide effective retrieval tools named as File Search, which ingest enterprise documents, build searchable indexes, and retrieve only the most relevant evidence for generation https://platform.openai.com/docs/guides/migrate-to-responses https://ai.google.dev/gemini-api/docs/file-search. Under our broad definition, this enables graph-based RAG beyond classic knowledge-graph traversal, where document-level structures and hierarchical graphs can be distilled from text and metadata, and retrieval can return compact subgraphs that enforce entity constraints before generation while reducing token and latency costs. Moreover, Gemini offers explicit controls for extended reasoning, enabling loop and tree style pipelines that iteratively retrieve evidence, verify intermediate results, and revise outputs, which matches the needs of graph-structured pipeline execution in practice https://ai.google.dev/gemini-api/docs/thinking. Compared with earlier LLMs, frontier models mainly address two practical bottlenecks in RAG: they reduce controller brittleness for iterative retrieve–verify–revise pipelines, and they make RAG scalable to large private corpora by relying on managed retrieval interfaces instead of manual prompt stuffing (Li et al., 2025b). Efficiency Discussion. The efficiency of industrial graph-based RAG can be examined through three views. First, graph construction and precomputation determine the baseline latency and storage footprint: Microsoft’s GraphRAG constructs entity-centric knowledge graphs and pre-generates community summaries to speed query-focused summarization at the expense of build time and storage, while Neo4j’s NaLLM automates the extraction of nodes, relations, and properties from unstructured data to reduce manual engineering effort in the indexing stage. Second, the retrieval strategy will control most online computation: AntGroup’s GraphRAG narrows work by locating keywords, mapping them to graph nodes, and traversing only the necessary subgraph with breadth-first or depth-first search, and NebulaGraph’s GraphRAG integrates large language models with the graph database to raise precision and avoid wasteful candidate expansion. Third, pipeline design governs coordination overhead and response quality. Sequential chains centralize control and keep latency low, exemplified by LangChain’s modular RAG chains and CrewAI’s support for sequential processes. Loop pipelines add iterative refinement and self-correction when increasing the cost. For example, LangGraph provides explicit loop capability with state persistence so a retriever and generator can revisit intermediate results when needed. Tree pipelines branch sub-queries or specialized agents for parallel search, which raises coordination cost but can improve answer quality. For instance, AG2 AutoGen together and OpenAI Swarm framework https://github.com/openai/swarm fans out sub-queries before synthesis, which improves coverage at the cost of additional coordination. 10. Future Research Opportunities Adaptive Prompts for LLMs. Future research could delve into the design and evaluation of adaptive prompts to better align with the pretrained knowledge of LLMs. By tailoring the structure, terminology, and hierarchy of information drawn from graphs, system designers could harness improved synergy between graph-based data and the LLM’s reasoning capabilities. One could also investigate methods to prioritize the most critical entities, relationships, or paths in a query (Li et al., 2024f; Yang et al., 2024a), ensuring that the prompt remains precise and contextually rich without overwhelming the model. Such selective attention could enhance both the relevance and interpretability of LLM outputs. Furthermore, exploring feedback mechanisms that dynamically refine the prompt based on the LLM’s responses might be an interesting direction, allowing systems to continuously optimize for coherence and clarity. Ultimately, adaptive prompt strategies could pave the way for more accurate, context-aware question answering, harnessing the potential of complex graph structures in tandem with cutting-edge language models. Enhanced Understanding for Graph Problems. Future research could delve into approaches that enable deeper structural comprehension of graphs within LLMs. Since many current models rely on sequential token representations, they struggle to handle larger or more complex graph structures—especially when a sequence involves a large number of nodes (Chen et al., 2024a; Guo et al., 2023). This limitation hinders their ability to solve typical graph-centric problems, such as path finding or community detection, which often require more specialized, non-linear reasoning. One direction might be to integrate graph-specific modules or adopt novel representations that faithfully capture relational information without overwhelming the model. Moreover, another avenue of exploration could involve combining LLMs with algorithmic components adopted at handling extensive, node-rich graphs. Multi-Modal Graphs for Cross-Domain RAG. Future advancements in Multi-Modal Graphs for Cross-Domain RAG should focus on the development of adaptable retrieval algorithms that accommodate the diversity of data types encountered in real-world applications. Presently, graph-based RAG systems are largely confined to single-domain use cases, often employing retrieval methods tailored to specific types of textual information. However, as cross-domain scenarios become increasingly common, there is a pressing need for retrieval techniques that can seamlessly handle multi-modal graph structures, including images (Han et al., 2022), audio (Nie et al., 2020), and numerical data. These enhanced retrieval algorithms must not only integrate and interpret a wide range of data formats but also dynamically adjust their retrieval strategies to match the unique characteristics of each domain. By enabling a more flexible, multi-modal approach, future research in this field can overcome the limitations of current systems, paving the way for more robust and versatile knowledge acquisition across diverse domains. Improved Graph Construction Techniques. The future of RAG systems can center around overcoming the limitations of traditional triple-based knowledge graphs, which are often too simplistic for complex, real-world data (Lan and Jiang, 2020). While the subject-predicate-object (Zhang et al., 2017) format has served as a foundation, it does not fully capture intricate relationships and evolving patterns within knowledge. To address this, future graph construction methods should explore hypergraphs, which allow for multi-node relationships (Li and Horvath, 2007), and semantic embeddings (Wang et al., 2018) that can represent context, enhancing the depth and accuracy of graph structures. Additionally, GNNs can be used to learn graph representations that dynamically evolve as new information is introduced, allowing for more precise and adaptive knowledge storage. Another important direction will be the development of hierarchical graphs such as the approaches utilized in (Ying et al., 2018) that capture different levels of abstraction, providing a more flexible and structured way to organize knowledge. Furthermore, adaptive graph construction techniques, which adjust the structure and organization of the graph based on the type and volume of data, will be essential for handling real-time updates without sacrificing retrieval speed. In summary, these advanced techniques will allow RAG systems to store knowledge more efficiently, retrieve information faster, and support more complex, multi-modal data interactions. Incorporating User Interaction. Future work in RAG systems can be significantly improved by integrating user interaction to improve the efficiency, accuracy, and user satisfaction of knowledge retrieval. By integrating Human-Computer Interaction (HCI) principles, RAG systems can offer more intuitive and adaptive interfaces, allowing users to actively shape the retrieval process. For example, adaptive query refinement (Deshpande et al., 2007) could enable users to iteratively clarify or expand their queries, with the system responding to these modifications in real-time. Additionally, interactive visualizations of the knowledge graph could allow users to explore relationships between data points, enhancing understanding and control over the retrieval process. Incorporating user feedback loops (Mansoury et al., 2020) — where users can rate the relevance of results or suggest corrections—would also help the system learn and adapt over time, making it more responsive to evolving user needs. Moreover, adopting context-aware systems (Hong et al., 2009) could allow RAG systems to interpret not only the user’s query but also the broader context of the interaction, further refining results based on user preferences and past behaviors. By blending these HCI concepts into RAG design, future systems can offer more personalized, efficient, and transparent experiences, ultimately enhancing user trust and satisfaction. 11. Conclusion This paper provides a comprehensive survey of the integration of graph-based techniques into RAG systems, offering a detailed review of their applications, advancements, and challenges. By categorizing existing methods and proposing a novel taxonomy, it provides valuable insights into how graphs can enhance the reasoning capabilities and accuracy of LLMs. The survey also identifies key challenges, such as dynamic graph integration, scalability, and explainability, while outlining future research directions to address these issues. Overall, this work contributes to a deeper understanding of the role of graphs in RAG systems and lays a foundation for future research aimed at unlocking their potential in improving LLM performance across a wide range of tasks. References - G. Agrawal, T. Kumarage, Z. Alghamdi, and H. Liu (2023) Can knowledge graphs reduce hallucinations in llms?: a survey. arXiv preprint arXiv:2311.07914. Cited by: §1. - A. Y. Alan, E. Karaarslan, and O. Aydin (2024) A rag-based question answering system proposal for understanding islam: mufassirqas llm. arXiv preprint arXiv:2401.15378. Cited by: §12.1, Table 4, Table 5. - M. Alhanahnah, Y. Boshmaf, and B. Baudry (2024) DepsRAG: towards managing software dependencies using large language models. External Links: 2405.20455, Link Cited by: §12.1, Table 2, Table 3, Table 5. - Z. An, X. Ding, Y. Fu, C. Chu, Y. Li, and W. Du (2024) Golden-retriever: high-fidelity agentic retrieval augmented generation for industrial knowledge base. External Links: 2408.00798, Link Cited by: §12.1, Table 3, Table 5. - G. Angeli, M. J. J. Premkumar, and C. D. Manning (2015) Leveraging linguistic structure for open domain information extraction. In ACL (1), pp. 344–354. Cited by: §4.2.1. - A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi (2024) Self-rag: learning to retrieve, generate, and critique through self-reflection. Cited by: §6.2. - S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. G. Ives (2007a) DBpedia: A nucleus for a web of open data. In ISWC/ASWC, Lecture Notes in Computer Science, Vol. 4825, pp. 722–735. Cited by: Table 1. - S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives (2007b) Dbpedia: a nucleus for a web of open data. In international semantic web conference, pp. 722–735. Cited by: §1. - J. Baek, A. F. Aji, J. Lehmann, and S. J. Hwang (2023a) Direct fact retrieval from knowledge graphs without entity linking. In ACL (1), pp. 10038–10055. Cited by: §12.1, Table 5. - J. Baek, A. F. Aji, and A. Saffari (2023b) Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. External Links: 2306.04136, Link Cited by: §12.1, §4.1.1, Table 3, Table 5. - J. Baek, S. Jeong, M. Kang, J. C. Park, and S. J. Hwang (2023c) Knowledge-augmented language model verification. In EMNLP, pp. 1720–1736. Cited by: §12.1, Table 3, Table 4, Table 5. - I. Balazevic, C. Allen, and T. Hospedales (2019) Multi-relational poincaré graph embeddings. Advances in Neural Information Processing Systems 32. Cited by: §5.2.3. - G. Bazhenov, O. Platonov, and L. Prokhorenkova (2024) TabGraphs: a benchmark and strong baselines for learning on graphs with tabular node features. arXiv preprint arXiv:2409.14500. Cited by: Table 6. - R. Bellman (1958) On a routing problem. Quarterly of applied mathematics 16 (1), pp. 87–90. Cited by: §5.1.1. - J. Berant, A. Chou, R. Frostig, and P. Liang (2013) Semantic parsing on freebase from question-answer pairs. In EMNLP, pp. 1533–1544. Cited by: §7.4, Table 6. - M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk, and T. Hoefler (2024a) Graph of thoughts: solving elaborate problems with large language models. In AAAI, pp. 17682–17690. Cited by: Table 3, §6.5, Table 4. - M. Besta, F. Memedi, Z. Zhang, R. Gerstenberger, N. Blach, P. Nyczyk, M. Copik, G. Kwasniewski, J. Müller, L. Gianinazzi, A. Kubicek, H. Niewiadomski, O. Mutlu, and T. Hoefler (2024b) Topologies of reasoning: demystifying chains, trees, and graphs of thoughts. CoRR abs/2401.14295. Cited by: §2. - D. Bienstock, M. X. Goemans, D. Simchi-Levi, and D. Williamson (1993) A note on the prize collecting traveling salesman problem. Mathematical programming 59 (1), pp. 413–420. Cited by: §7.1.2. - Y. Bisk, R. Zellers, R. L. Bras, J. Gao, and Y. Choi (2020) PIQA: reasoning about physical commonsense in natural language. In AAAI, pp. 7432–7439. Cited by: Table 1. - K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp. 1247–1250. Cited by: §1, §4.1.1, §4.1.2. - A. Bordes, N. Usunier, S. Chopra, and J. Weston (2015) Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075. Cited by: Table 6. - A. Bridge (2001) Wikipedia, the free encyclopedia. San Francisco (CA): Wikimedia Foundation. Cited by: §4.1.2. - T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020) Language models are few-shot learners. Advances in neural information processing systems 33, pp. 1877–1901. Cited by: §2. - S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. (2023) Sparks of artificial general intelligence: early experiments with gpt-4. arXiv preprint arXiv:2303.12712. Cited by: §1. - A. Chakraborty (2024) Multi-hop question answering over knowledge graphs using large language models. External Links: 2404.19234, Link Cited by: Table 5. - H. Chen (2024) Large knowledge model: perspectives and challenges. External Links: 2312.02706, Link Cited by: §2. - N. Chen, Y. Li, J. Tang, and J. Li (2024a) GraphWiz: an instruction-following language model for graph problems. arXiv preprint arXiv:2402.16029. Cited by: §10. - R. Chen, T. Zhao, A. Jaiswal, N. Shah, and Z. Wang (2024b) LLaGA: large language and graph assistant. External Links: 2402.08170, Link Cited by: §12.1, §5.4.1, Table 3, Table 5, Table 5. - S. Chen, Q. Liu, Z. Yu, C. Lin, J. Lou, and F. Jiang (2021) ReTraCk: A flexible and efficient framework for knowledge base question answering. In ACL (demo), pp. 325–336. Cited by: §12.1, §4.2.1, Table 1, Table 5. - Y. Chen, L. Yan, W. Sun, X. Ma, Y. Zhang, S. Wang, D. Yin, Y. Yang, and J. Mao (2025) Improving retrieval-augmented generation through multi-agent reinforcement learning. arXiv preprint arXiv:2501.15228. Cited by: §6.2. - Z. Chen, H. Mao, H. Li, W. Jin, H. Wen, X. Wei, S. Wang, D. Yin, W. Fan, H. Liu, et al. (2024c) Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter 25 (2), pp. 42–61. Cited by: §1, §12.1, §12.2, Table 2, Table 3, Table 5. - Z. Chen, J. Liao, and X. Zhao (2023) Multi-granularity temporal question answering over knowledge graphs. In ACL (1), pp. 11378–11392. Cited by: §7.4, Table 6. - K. Cheng, G. Lin, H. Fei, Y. zhai, L. Yu, M. A. Ali, L. Hu, and D. Wang (2024) Multi-hop question answering under temporal knowledge editing. External Links: 2404.00492, Link Cited by: Table 5. - Z. Cheng, H. Dong, Z. Wang, R. Jia, J. Guo, Y. Gao, S. Han, J. Lou, and D. Zhang (2021) Hitab: a hierarchical table dataset for question answering and natural language generation. arXiv preprint arXiv:2108.06712. Cited by: Table 6. - H. K. Choi, S. Lee, J. Chu, and H. J. Kim (2023) NuTrea: neural tree search for context-guided multi-hop KGQA. In NeurIPS, Cited by: §12.1, §5.6, Table 2, Table 2, Table 4, Table 5. - N. Choudhary and C. K. Reddy (2024) Complex logical reasoning over knowledge graphs using large language models. External Links: 2305.01157, Link Cited by: Table 3. - R. Das, S. Dhuliawala, M. Zaheer, L. Vilnis, I. Durugkar, A. Krishnamurthy, A. Smola, and A. McCallum (2018) Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In ICLR (Poster), Cited by: §12.1, Table 1, Table 2, Table 5. - DeepSeek-AI, D. Guo, and et al. (2025) DeepSeek-r1: incentivizing reasoning capability in llms via reinforcement learning. External Links: 2501.12948, Link Cited by: §1. - M. Dehghan, M. A. Alomrani, S. Bagga, D. Alfonso-Hermelo, K. Bibi, A. Ghaddar, Y. Zhang, X. Li, J. Hao, Q. Liu, J. Lin, B. Chen, P. Parthasarathi, M. Biparva, and M. Rezagholizadeh (2024) EWEK-QA : enhanced web and efficient knowledge graph retrieval for citation-based question answering systems. In ACL (1), pp. 14169–14187. Cited by: §1, §12.2, Table 1, Table 3, Table 4, Table 5. - Y. Deldjoo, Z. He, J. McAuley, A. Korikov, S. Sanner, A. Ramisa, R. Vidal, M. Sathiamoorthy, A. Kasirzadeh, and S. Milano (2024) A review of modern recommender systems using generative models (gen-recsys). External Links: 2404.00579, Link Cited by: §5.6. - J. Delile, S. Mukherjee, A. V. Pamel, and L. Zhukov (2024) Graph-based retriever captures the long tail of biomedical knowledge. External Links: 2402.12352, Link Cited by: §12.2, Table 1, §5.1.2, Table 2, Table 4, §7.3, Table 5, Table 5. - A. Deshpande, Z. Ives, V. Raman, et al. (2007) Adaptive query processing. Foundations and Trends® in Databases 1 (1), pp. 1–140. Cited by: §10. - T. Dettmers, P. Minervini, P. Stenetorp, and S. Riedel (2018) Convolutional 2d knowledge graph embeddings. In AAAI, pp. 1811–1818. Cited by: §1, §12.1, Table 2, Table 5. - J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), pp. 4171–4186. Cited by: §1, §2, §5.2.3. - E. W. Dijkstra (2022) A note on two problems in connexion with graphs. In Edsger Wybe Dijkstra: his life, work, and legacy, pp. 287–290. Cited by: §5.1.1. - J. Ding, H. Nguyen, and H. Chen (2024) Evaluation of question-answering based text summarization using llm invited paper. In 2024 IEEE International Conference on Artificial Intelligence Testing (AITest), pp. 142–149. Cited by: §1. - Z. Ding, Z. Li, R. Qi, J. Wu, B. He, Y. Ma, Z. Meng, S. Chen, R. Liao, Z. Han, et al. (2023) Forecasttkgquestions: a benchmark for temporal question answering and forecasting over temporal knowledge graphs. In International Semantic Web Conference, pp. 541–560. Cited by: §7.4, Table 6. - G. Dong, R. Li, S. Wang, Y. Zhang, Y. Xian, and W. Xu (2023a) Bridging the kb-text gap: leveraging structured knowledge-aware pre-training for KBQA. In CIKM, pp. 3854–3859. Cited by: §12.1, Table 2, Table 5. - J. Dong, Q. Zhang, X. Huang, K. Duan, Q. Tan, and Z. Jiang (2023b) Hierarchy-aware multi-hop question answering over knowledge graphs. In Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 - 4 May 2023, pp. 2519–2527. Cited by: §5.2.3, §5.4.1, Table 2, Table 3, Table 4, Table 5. - D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson (2024) From local to global: a graph rag approach to query-focused summarization. External Links: 2404.16130, Link Cited by: Table 1, §5.6, Table 4, §7.1.2, Table 5. - H. ElSahar, P. Vougiouklis, A. Remaci, C. Gravier, J. S. Hare, F. Laforest, and E. Simperl (2018) T-rex: A large scale alignment of natural language with knowledge base triples. In LREC, Cited by: §4.1.1, §4.1.2, Table 1. - O. Etzioni, M. Banko, S. Soderland, and D. S. Weld (2008) Open information extraction from the web. Communications of the ACM 51 (12), pp. 68–74. Cited by: §4.2.1. - W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T. Chua, and Q. Li (2024a) A survey on rag meeting llms: towards retrieval-augmented large language models. External Links: 2405.06211, Link Cited by: §1, §2. - W. Fan, S. Wang, J. Huang, Z. Chen, Y. Song, W. Tang, H. Mao, H. Liu, X. Liu, D. Yin, and Q. Li (2024b) Graph machine learning in the era of large language models (llms). External Links: 2404.14928, Link Cited by: §2. - J. Fang, Z. Meng, and C. MacDonald (2024) REANO: optimising retrieval-augmented reader models through knowledge graph generation. In ACL (1), pp. 2094–2112. Cited by: Table 1, §5.2.2, Table 2, Table 5. - B. Fatemi, J. Halcrow, and B. Perozzi (2023) Talk like a graph: encoding graphs for large language models. External Links: 2310.04560, Link Cited by: §1, Table 3. - C. Feng, X. Zhang, and Z. Fei (2023) Knowledge solver: teaching llms to search for domain knowledge from knowledge graphs. External Links: 2309.03118, Link Cited by: §12.1, Table 2, Table 4, Table 5. - Y. Feng, X. Chen, B. Y. Lin, P. Wang, J. Yan, and X. Ren (2020) Scalable multi-hop relational reasoning for knowledge-aware question answering. In EMNLP (1), pp. 1295–1309. Cited by: Table 2, Table 5. - Z. Feng, Y. Zhang, H. Li, W. Liu, J. Lang, Y. Feng, J. Wu, and Z. Liu (2024) Improving llm-based machine translation with systematic self-correction. arXiv preprint arXiv:2402.16379. Cited by: §1. - P. Ferragina and U. Scaiella (2010) Tagme: on-the-fly annotation of short text fragments (by wikipedia entities). In Proceedings of the 19th ACM international conference on Information and knowledge management, pp. 1625–1628. Cited by: §5.2.2. - D. Flanagan (2007) Developing metaweb-enabled web applications. Metaweb Technologies. Cited by: §4.1.2. - H. Gao, L. Wu, P. Hu, Z. Wei, F. Xu, and B. Long (2022) Graph-augmented learning to rank for querying large-scale knowledge graph. In AACL/IJCNLP (1), pp. 82–92. Cited by: §12.1, Table 2, Table 2, Table 5. - Y. Gao, L. Qiao, Z. Kan, Z. Wen, Y. He, and D. Li (2024a) Two-stage generative question answering on temporal knowledge graph using large language models. arXiv preprint arXiv:2402.16568. Cited by: §8. - Y. Gao, L. Qiao, Z. Kan, Z. Wen, Y. He, and D. Li (2024b) Two-stage generative question answering on temporal knowledge graph using large language models. External Links: 2402.16568, Link Cited by: §12.1, Table 2, Table 5. - Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang (2024c) Retrieval-augmented generation for large language models: a survey. External Links: 2312.10997, Link Cited by: §1, §1, §2. - Y. Gu, S. Kase, M. Vanni, B. M. Sadler, P. Liang, X. Yan, and Y. Su (2021) Beyond I.I.D.: three levels of generalization for question answering on knowledge bases. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pp. 3477–3488. Cited by: Table 1, §7.4, Table 6. - Y. Gu and Y. Su (2022) ArcaneQA: dynamic program induction and contextualized encoding for knowledge base question answering. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 1718–1731. Cited by: §12.2, Table 1, §5.6, Table 2, Table 4, Table 5. - Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon (2022) Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Heal. 3 (1), pp. 2:1–2:23. Cited by: §12.2. - J. Guo, L. Du, H. Liu, M. Zhou, X. He, and S. Han (2023) GPT4Graph: can large language models understand graph structured data ? an empirical evaluation and benchmarking. External Links: 2305.15066, Link Cited by: §1, §10, §12.1, Table 3, Table 5, Table 6. - T. Guo, Q. Yang, C. Wang, Y. Liu, P. Li, J. Tang, D. Li, and Y. Wen (2024) KnowledgeNavigator: leveraging large language models for enhanced reasoning over knowledge graph. External Links: 2312.15880, Link Cited by: §12.1, Table 4, Table 5. - B. J. Gutiérrez, Y. Shu, Y. Gu, M. Yasunaga, and Y. Su (2024) HippoRAG: neurobiologically inspired long-term memory for large language models. External Links: 2405.14831, Link Cited by: §1, §4.2.1, Table 1, §5.1.3, Table 2, Table 4, Table 5. - M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili, et al. (2023) A survey on large language models: applications, challenges, limitations, and practical usage. Authorea Preprints. Cited by: §2, §5.6. - H. Han, Y. Wang, H. Shomer, K. Guo, J. Ding, Y. Lei, M. Halappanavar, R. A. Rossi, S. Mukherjee, X. Tang, et al. (2024) Retrieval-augmented generation with graphs (graphrag). arXiv preprint arXiv:2501.00309. Cited by: §2. - K. Han, Y. Wang, J. Guo, Y. Tang, and E. Wu (2022) Vision gnn: an image is worth graph of nodes. Advances in neural information processing systems 35, pp. 8291–8303. Cited by: §10. - Z. Han, Y. Feng, and M. Sun (2023) A graph-guided reasoning approach for open-ended commonsense question answering. External Links: 2303.10395, Link Cited by: §4.2.1, Table 1, Table 2. - G. He, Y. Lan, J. Jiang, W. X. Zhao, and J. Wen (2021) Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In WSDM, pp. 553–561. Cited by: §12.1, Table 5. - X. He, Y. Tian, Y. Sun, N. V. Chawla, T. Laurent, Y. LeCun, X. Bresson, and B. Hooi (2024) G-retriever: retrieval-augmented generation for textual graph understanding and question answering. External Links: 2402.07630, Link Cited by: §4.1.4, §5.4.1, Table 3, §7.1.2, Table 5, Table 6. - J. Hoffart, M. A. Yosef, I. Bordino, H. Fürstenau, M. Pinkal, M. Spaniol, B. Taneva, S. Thater, and G. Weikum (2011) Robust disambiguation of named entities in text. In EMNLP, pp. 782–792. Cited by: Table 2. - J. Hong, E. Suh, and S. Kim (2009) Context-aware systems: a literature review and classification. Expert Systems with applications 36 (4), pp. 8509–8522. Cited by: §10. - Y. Hu and Y. Lu (2024) RAG and rau: a survey on retrieval-augmented language model in natural language processing. External Links: 2404.19543, Link Cited by: §1, §2. - Y. Hu, Z. Lei, Z. Zhang, B. Pan, C. Ling, and L. Zhao (2024) GRAG: graph retrieval-augmented generation. External Links: 2405.16506, Link Cited by: §12.2, §4.1.1, §5.1.4, §5.4.1, Table 2, Table 2, Table 3, Table 4, Table 5. - Z. Hu, Y. Dong, K. Wang, and Y. Sun (2020) Heterogeneous graph transformer. In Proceedings of the web conference 2020, pp. 2704–2710. Cited by: §12.2. - Z. Hu, Y. Xu, W. Yu, S. Wang, Z. Yang, C. Zhu, K. Chang, and Y. Sun (2022) Empowering language models with knowledge graph reasoning for open-domain question answering. In EMNLP, pp. 9562–9581. Cited by: §12.1, Table 2, Table 5. - J. Huang, S. Madala, R. Sidhu, C. Niu, H. Peng, J. Hockenmaier, and T. Zhang (2025) Rag-rl: advancing retrieval-augmented generation via rl and curriculum learning. arXiv preprint arXiv:2503.12759. Cited by: §6.2. - J. Huang and K. C. Chang (2022) Towards reasoning in large language models: a survey. arXiv preprint arXiv:2212.10403. Cited by: §2. - L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu (2023a) A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. External Links: 2311.05232, Link Cited by: §2. - Y. Huang and J. Huang (2024) A survey on retrieval-augmented text generation for large language models. External Links: 2404.10981, Link Cited by: §1, §2. - Y. Huang, Y. Li, Y. Xu, L. Zhang, R. Gan, J. Zhang, and L. Wang (2023b) MVP-tuning: multi-view knowledge retrieval with prompt tuning for commonsense reasoning. In ACL (1), pp. 13417–13432. Cited by: §12.1, Table 3, Table 5. - S. Jeong, J. Baek, S. Cho, S. J. Hwang, and J. C. Park (2024) Adaptive-rag: learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:2403.14403. Cited by: §6.2. - J. Jiang, K. Zhou, Z. Dong, K. Ye, X. Zhao, and J. Wen (2023a) StructGPT: A general framework for large language model to reason over structured data. In EMNLP, pp. 9237–9251. Cited by: Table 5. - J. Jiang, K. Zhou, W. X. Zhao, Y. Song, C. Zhu, H. Zhu, and J. Wen (2024a) KG-agent: an efficient autonomous agent framework for complex reasoning over knowledge graph. External Links: 2402.11163, Link Cited by: §12.1, Table 4, Table 5. - J. Jiang, K. Zhou, X. Zhao, and J. Wen (2023b) UniKGQA: unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, Cited by: §12.1, §4.1.1, Table 5. - K. Jiang, D. Wu, and H. Jiang (2019) FreebaseQA: A new factoid QA data set matching trivia-style question-answer pairs with freebase. In NAACL-HLT (1), pp. 318–323. Cited by: Table 1. - X. Jiang, R. Qiu, Y. Xu, W. Zhang, Y. Zhu, R. Zhang, Y. Fang, X. Chu, J. Zhao, and Y. Wang (2024b) RAGraph: a general retrieval-augmented graph learning framework. arXiv preprint arXiv:2410.23855. Cited by: §7.2.2. - X. Jiang, R. Zhang, Y. Xu, R. Qiu, Y. Fang, Z. Wang, J. Tang, H. Ding, X. Chu, J. Zhao, and Y. Wang (2024c) HyKGE: a hypothesis knowledge graph enhanced framework for accurate and reliable medical llms responses. External Links: 2312.15883, Link Cited by: §5.1.2, §5.4.1, Table 2, Table 3, Table 4, §7.3, Table 5. - Y. Jiang, H. Wang, L. Xie, H. Zhao, H. Qian, J. Lui, et al. (2025) D-llm: a token adaptive computing resource allocation strategy for large language models. Advances in Neural Information Processing Systems 37, pp. 1725–1749. Cited by: §6.6. - Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y. Yang, J. Callan, and G. Neubig (2023c) Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7969–7992. Cited by: §6.2. - B. Jin, G. Liu, C. Han, M. Jiang, H. Ji, and J. Han (2024a) Large language models on graphs: a comprehensive survey. External Links: 2312.02783, Link Cited by: §2, §3.2. - B. Jin, C. Xie, J. Zhang, K. K. Roy, Y. Zhang, Z. Li, R. Li, X. Tang, S. Wang, Y. Meng, and J. Han (2024b) Graph chain-of-thought: augmenting large language models by reasoning on graphs. External Links: 2404.07103, Link Cited by: §4.1.4, §5.4.1, Table 3, §6.4, Table 4, Table 5, Table 6. - D. Jin, E. Pan, N. Oufattole, W. Weng, H. Fang, and P. Szolovits (2020) What disease does this patient have? a large-scale open domain question answering dataset from medical exams. External Links: 2009.13081, Link Cited by: §12.1, Table 1, §7.4, Table 5, Table 6. - Q. Jin, B. Dhingra, Z. Liu, W. W. Cohen, and X. Lu (2019) Pubmedqa: a dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146. Cited by: §7.4, Table 6. - J. Jo, J. Baek, S. Lee, D. Kim, M. Kang, and S. J. Hwang (2021) Edge representation learning with hypergraphs. Advances in Neural Information Processing Systems 34, pp. 7534–7546. Cited by: §5.2.2. - M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer (2017) TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pp. 1601–1611. Cited by: Table 1, Table 6. - M. Kang, J. M. Kwak, J. Baek, and S. J. Hwang (2022) Knowledge-consistent dialogue generation with knowledge graphs. In ICML 2022 Workshop on Knowledge Retrieval and Language Models, Cited by: §5.2.2, Table 2, Table 3, §6.5, Table 4, Table 5. - J. Kim, Y. Kwon, Y. Jo, and E. Choi (2023a) KG-GPT: A general framework for reasoning on knowledge graphs using large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pp. 9410–9421. Cited by: §5.1.2, Table 2, Table 4, Table 5. - J. Kim, S. Park, Y. Kwon, Y. Jo, J. Thorne, and E. Choi (2023b) FactKG: fact verification via reasoning on knowledge graphs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 16190–16206. Cited by: Table 1, Table 6. - T. N. Kipf and M. Welling (2017) Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, Cited by: §5.2.1. - T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. P. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, K. Toutanova, L. Jones, M. Kelcey, M. Chang, A. M. Dai, J. Uszkoreit, Q. Le, and S. Petrov (2019) Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics 7, pp. 452–466. Cited by: Table 6. - Y. Lan and J. Jiang (2020) Query graph generation for answering multi-hop complex questions from knowledge bases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 969–974. Cited by: §10, §4.2.2, Table 1, Table 2. - E. Lavrinovics, R. Biswas, J. Bjerva, and K. Hose (2025) Knowledge graphs, large language models, and hallucinations: an nlp perspective. Journal of Web Semantics 85, pp. 100844. Cited by: §1. - A. Li and S. Horvath (2007) Network neighborhood analysis with the multi-node topological overlap measure. Bioinformatics 23 (2), pp. 222–231. Cited by: §10. - D. Li, S. Yang, Z. Tan, J. Y. Baik, S. Yun, J. Lee, A. Chacko, B. Hou, D. Duong-Tran, Y. Ding, H. Liu, L. Shen, and T. Chen (2024a) DALK: dynamic co-augmentation of llms and kg to answer alzheimer’s disease questions with scientific literature. External Links: 2405.04819, Link Cited by: §1, §4.2.1, Table 1, §5.4.1, Table 3, §6.3, Table 4, §7.3, Table 5. - D. Li, Y. Niu, Y. Ai, X. Zou, B. Qi, and J. Liu (2025a) T-grag: a dynamic graphrag framework for resolving temporal conflicts and redundancy in knowledge retrieval. arXiv preprint arXiv:2508.01680. Cited by: §8. - X. Li, Z. Li, C. Shi, Y. Xu, Q. Du, M. Tan, J. Huang, and W. Lin (2024b) AlphaFin: benchmarking financial analysis with retrieval-augmented stock-chain framework. arXiv preprint arXiv:2403.12582. Cited by: §7.4, Table 6. - X. Li, Y. Bai, B. Jin, F. Zhu, L. Pan, and Y. Cao (2025b) Long context vs. rag: strategies for processing long documents in llms. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 4110–4113. Cited by: §9. - Y. Li, Z. Li, P. Wang, J. Li, X. Sun, H. Cheng, and J. X. Yu (2024c) A survey of graph meets large language model: progress and future directions. External Links: 2311.12399, Link Cited by: §2. - Y. Li, P. Wang, X. Zhu, A. Chen, H. Jiang, D. Cai, V. W. K. Chan, and J. Li (2024d) GLBench: a comprehensive benchmark for graph with large language models. ArXiv abs/2407.07457. External Links: Link Cited by: §12.1, Table 2, Table 3, §7.4, Table 5, Table 6. - Z. Li, L. Deng, H. Liu, Q. Liu, and J. Du (2024e) UniOQA: a unified framework for knowledge graph question answering with large language models. External Links: 2406.02110, Link Cited by: Table 3, Table 4. - Z. Li, Q. Guo, J. Shao, L. Song, J. Bian, J. Zhang, and R. Wang (2024f) Graph neural network enhanced retrieval for question answering of llms. External Links: 2406.06572, Link Cited by: §10, §12.2, §4.2.1, Table 1, Table 2, §6.4, Table 4. - B. Y. Lin, X. Chen, J. Chen, and X. Ren (2019) KagNet: knowledge-aware graph networks for commonsense reasoning. In EMNLP/IJCNLP (1), pp. 2829–2839. Cited by: §12.1, Table 5. - B. Y. Lin, Z. Wu, Y. Yang, D. Lee, and X. Ren (2021) RiddleSense: reasoning about riddle questions featuring linguistic creativity and commonsense knowledge. In ACL/IJCNLP (Findings), Findings of ACL, Vol. ACL/IJCNLP 2021, pp. 1504–1515. Cited by: Table 1. - G. Liu, Y. Zhang, Y. Li, and Q. Yao (2024a) Explore then determine: a gnn-llm synergy framework for reasoning over knowledge graph. External Links: 2406.01145, Link Cited by: §12.2, Table 2, Table 3, Table 4, Table 5. - H. Liu, S. Wang, Y. Zhu, Y. Dong, and J. Li (2024b) Knowledge graph-enhanced large language models via path selection. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pp. 6311–6321. Cited by: Table 2, Table 3, Table 5. - J. Liu, C. S. Xia, Y. Wang, and L. Zhang (2024c) Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36. Cited by: §1. - J. Liu, C. Yang, Z. Lu, J. Chen, Y. Li, M. Zhang, T. Bai, Y. Fang, L. Sun, P. S. Yu, and C. Shi (2024d) Towards graph foundation models: a survey and beyond. External Links: 2310.11829, Link Cited by: §2. - W. Liu, A. Yu, D. Zan, B. Shen, W. Zhang, H. Zhao, Z. Jin, and Q. Wang (2024e) GraphCoder: enhancing repository-level code completion via code context graph-based retrieval and language model. arXiv preprint arXiv:2406.07003. Cited by: §7.3. - Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019) RoBERTa: a robustly optimized bert pretraining approach. External Links: 1907.11692, Link Cited by: §1. - L. Logeswaran, M. Chang, K. Lee, K. Toutanova, J. Devlin, and H. Lee (2019) Zero-shot entity linking by reading entity descriptions. In ACL (1), pp. 3449–3460. Cited by: Table 2. - D. Luo, J. Sheng, H. Xu, L. Wang, and B. Wang (2023) Improving complex knowledge base question answering with relation-aware subgraph retrieval and reasoning network. In International Joint Conference on Neural Networks, IJCNN 2023, Gold Coast, Australia, June 18-23, 2023, pp. 1–8. Cited by: §1. - H. Luo, H. E, Z. Tang, S. Peng, Y. Guo, W. Zhang, C. Ma, G. Dong, M. Song, W. Lin, Y. Zhu, and L. A. Tuan (2024a) ChatKBQA: a generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. External Links: 2310.08975, Link Cited by: Table 1, Table 4. - L. Luo, P. Lai, C. Wei, C. N. Arighi, and Z. Lu (2022) BioRED: a rich biomedical relation extraction dataset. Briefings Bioinform. 23 (5). External Links: Document, Link Cited by: Table 1, §7.4, Table 6. - L. Luo, Y. Li, G. Haffari, and S. Pan (2024b) Reasoning on graphs: faithful and interpretable large language model reasoning. External Links: 2310.01061, Link Cited by: §12.1, §5.4.1, §5.4.2, Table 3, Table 5. - Y. Lyu, Z. Li, S. Niu, F. Xiong, B. Tang, W. Wang, H. Wu, H. Liu, T. Xu, and E. Chen (2024) CRUD-rag: a comprehensive chinese benchmark for retrieval-augmented generation of large language models. arXiv preprint arXiv:2401.17043. Cited by: Table 6. - S. Ma, C. Xu, X. Jiang, M. Li, H. Qu, and J. Guo (2024) Think-on-graph 2.0: deep and interpretable large language model reasoning with knowledge graph-guided retrieval. External Links: 2407.10805, Link Cited by: §12.1, §5.4.1, §5.4.1, Table 2, Table 3, Table 4, Table 5. - X. Ma, Y. Gong, P. He, H. Zhao, and N. Duan (2023) Query rewriting for retrieval-augmented large language models. External Links: 2305.14283, Link Cited by: Table 4. - M. Mansoury, H. Abdollahpouri, M. Pechenizkiy, B. Mobasher, and R. Burke (2020) Feedback loop and bias amplification in recommender systems. In Proceedings of the 29th ACM international conference on information & knowledge management, pp. 2145–2148. Cited by: §10. - Q. Mao, Z. Liu, C. Liu, Z. Li, and J. Sun (2024) Advancing graph representation learning with large language models: a comprehensive survey of techniques. External Links: 2402.05952, Link Cited by: §2, §5.6. - C. Mavromatis and G. Karypis (2024) GNN-rag: graph neural retrieval for large language model reasoning. External Links: 2405.20139, Link Cited by: §5.1.2, Table 2, Table 3, Table 4, Table 5. - T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal (2018) Can a suit of armor conduct electricity? A new dataset for open book question answering. In EMNLP, pp. 2381–2391. Cited by: Table 1. - S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao (2024) Large language models: a survey. arXiv preprint arXiv:2402.06196. Cited by: §2. - D. Mondal, S. Modi, S. Panda, R. Singh, and G. S. Rao (2024) KAM-cot: knowledge augmented multimodal chain-of-thoughts reasoning. In AAAI, pp. 18798–18806. Cited by: §12.1, Table 2, Table 5. - C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann (2020) TUDataset: a collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020), Cited by: Table 1, §7.4, Table 6. - S. Munikoti, A. Acharya, S. Wagle, and S. Horawalavithana (2023) ATLANTIC: structure-aware retrieval-augmented language model for interdisciplinary science. External Links: 2311.12289, Link Cited by: §12.2, §4.2.1, Table 1, Table 2, Table 4, §7.3, Table 5. - D. Myers, R. Mohawesh, V. I. Chellaboina, A. L. Sathvik, P. Venkatesh, Y. Ho, H. Henshaw, M. Alhawawreh, D. Berdik, and Y. Jararweh (2024) Foundation and large language models: fundamentals, challenges, opportunities, and social impacts. Cluster Computing 27 (1), pp. 1–26. Cited by: §2. - H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian (2023) A comprehensive overview of large language models. arXiv preprint arXiv:2307.06435. Cited by: §2. - W. Nie, M. Ren, J. Nie, and S. Zhao (2020) C-gcn: correlation based graph convolutional network for audio-video emotion recognition. IEEE Transactions on Multimedia 23, pp. 3793–3804. Cited by: §10. - A. Oliya, A. Saffari, P. Sen, and T. Ayoola (2021) End-to-end entity resolution and question answering using differentiable knowledge graphs. arXiv preprint arXiv:2109.05817. Cited by: §5.5.2. - Y. Onoe, M. J. Q. Zhang, E. Choi, and G. Durrett (2021) CREAK: A dataset for commonsense reasoning over entity knowledge. In NeurIPS Datasets and Benchmarks, Cited by: Table 1, Table 6. - OpenAI (2024) GPT-4 technical report. External Links: 2303.08774, Link Cited by: §1. - T. A. Opsahl (2024) Fact or fiction? improving fact verification with knowledge graphs through simplified subgraph retrievals. External Links: 2408.07453, Link Cited by: §12.1, Table 2, Table 3, Table 5. - V. Pahuja, B. Wang, H. Latapie, J. Srinivasa, and Y. Su (2023) A retrieve-and-read framework for knowledge graph link prediction. In CIKM, pp. 1992–2002. Cited by: §1. - J. Z. Pan, S. Razniewski, J. Kalo, S. Singhania, J. Chen, S. Dietze, H. Jabeen, J. Omeliyanenko, W. Zhang, M. Lissandrini, R. Biswas, G. de Melo, A. Bonifati, E. Vakaj, M. Dragoni, and D. Graux (2023) Large language models and knowledge graphs: opportunities and challenges. TGDK 1 (1), pp. 2:1–2:38. Cited by: §2. - S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu (2024) Unifying large language models and knowledge graphs: A roadmap. IEEE Trans. Knowl. Data Eng. 36 (7), pp. 3580–3599. Cited by: §2. - B. Peng, Y. Zhu, Y. Liu, X. Bo, H. Shi, C. Hong, Y. Zhang, and S. Tang (2024) Graph retrieval-augmented generation: a survey. arXiv preprint arXiv:2408.08921. Cited by: §1, §2, §2, §3.2. - Z. Peng and Y. Yang (2024) Connecting the dots: inferring patent phrase similarity with retrieved phrase graphs. External Links: 2403.16265, Link Cited by: Table 2, Table 2. - A. Perevalov, D. Diefenbach, R. Usbeck, and A. Both (2022) QALD-9-plus: A multilingual dataset for question answering over dbpedia and wikidata translated by native speakers. In ICSC, pp. 229–234. Cited by: Table 1, Table 6. - F. Petroni, A. Piktus, A. Fan, P. Lewis, M. Yazdani, N. D. Cao, J. Thorne, Y. Jernite, V. Karpukhin, J. Maillard, V. Plachouras, T. Rocktäschel, and S. Riedel (2021) KILT: a benchmark for knowledge intensive language tasks. In NAACL-HLT, pp. 2523–2544. Cited by: Table 6. - O. Press, M. Zhang, S. Min, L. Schmidt, N. A. Smith, and M. Lewis (2022) Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350. Cited by: §6.2. - Z. Qi, Y. Yu, M. Tu, J. Tan, and Y. Huang (2023) FoodGPT: a large language model in food testing domain with incremental pre-training and knowledge graph prompt. External Links: 2308.10173, Link Cited by: §12.1, Table 1, Table 5. - H. Qian, P. Zhang, Z. Liu, K. Mao, and Z. Dou (2024) Memorag: moving towards next-gen rag via memory-inspired knowledge discovery. arXiv preprint arXiv:2409.05591 1. Cited by: Table 6. - P. Ranade and A. Joshi (2023) FABULA: intelligence report generation using retrieval-augmented narrative construction. In Proceedings of the International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2023, Kusadasi, Turkey, November 6-9, 2023, pp. 603–610. Cited by: §12.1, §4.2.1, Table 1, Table 3, Table 4, Table 5. - V. Rawte, A. Sheth, and A. Das (2023) A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922. Cited by: §2. - J. Robinson, R. Ranjan, W. Hu, K. Huang, J. Han, A. Dobles, M. Fey, J. E. Lenssen, Y. Yuan, Z. Zhang, et al. (2024) Relbench: a benchmark for deep learning on relational databases. Advances in Neural Information Processing Systems 37, pp. 21330–21341. Cited by: §7.4, Table 6. - M. Sap, R. L. Bras, E. Allaway, C. Bhagavatula, N. Lourie, H. Rashkin, B. Roof, N. A. Smith, and Y. Choi (2019a) ATOMIC: an atlas of machine commonsense for if-then reasoning. In AAAI, pp. 3027–3035. Cited by: Table 1. - M. Sap, H. Rashkin, D. Chen, R. LeBras, and Y. Choi (2019b) SocialIQA: commonsense reasoning about social interactions. External Links: 1904.09728, Link Cited by: Table 1. - B. Sarmah, B. Hall, R. Rao, S. Patel, S. Pasquali, and D. Mehta (2024) HybridRAG: integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. External Links: 2408.04948, Link Cited by: Table 1, §5.1.4, Table 2, Table 3, Table 5. - A. Saxena, S. Chakrabarti, and P. Talukdar (2021) Question answering over temporal knowledge graphs. arXiv preprint arXiv:2106.01515. Cited by: §7.4, Table 6. - E. R. Scheinerman and D. H. Ullman (2011) Fractional graph theory: a rational approach to the theory of graphs. Courier Corporation. Cited by: §5.2.2. - P. Sen, A. F. Aji, and A. Saffari (2022) Mintaka: A complex, natural, and multilingual dataset for end-to-end question answering. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pp. 1604–1619. Cited by: Table 1, Table 6. - P. Sen, S. Mavadia, and A. Saffari (2023) Knowledge graph-augmented language models for complex question answering. In Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE), pp. 1–8. Cited by: §5.5.2, Table 2, Table 3. - T. Shen, R. Jin, Y. Huang, C. Liu, W. Dong, Z. Guo, X. Wu, Y. Liu, and D. Xiong (2023) Large language model alignment: a survey. arXiv preprint arXiv:2309.15025. Cited by: §2. - Y. Shi, Z. Huang, S. Feng, H. Zhong, W. Wang, and Y. Sun (2020) Masked label prediction: unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509. Cited by: §5.2.3. - K. Shuster, S. Poff, M. Chen, D. Kiela, and J. Weston (2021) Retrieval augmentation reduces hallucination in conversation. In EMNLP (Findings), pp. 3784–3803. Cited by: §1. - W. Su, Y. Tang, Q. Ai, Z. Wu, and Y. Liu (2024) DRAGIN: dynamic retrieval augmented generation based on the information needs of large language models. arXiv preprint arXiv:2403.10081. Cited by: §8. - F. M. Suchanek, G. Kasneci, and G. Weikum (2007) Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pp. 697–706. Cited by: Table 1. - H. Sun, T. Bedrax-Weiss, and W. W. Cohen (2019) PullNet: open domain question answering with iterative retrieval on knowledge bases and text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pp. 2380–2390. Cited by: §12.1, Table 5. - H. Sun, Y. Li, L. Deng, B. Li, B. Hui, B. Li, Y. Lan, Y. Zhang, and Y. Li (2023) History semantic graph enhanced conversational KBQA with temporal information modeling. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 3521–3533. Cited by: Table 1, §5.2.3, Table 2, Table 5. - J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y. Gong, L. M. Ni, H. Shum, and J. Guo (2024a) Think-on-graph: deep and responsible reasoning of large language model on knowledge graph. External Links: 2307.07697, Link Cited by: §12.2, §5.4.2. - L. Sun, Z. Tao, Y. Li, and H. Arakawa (2024b) ODA: observation-driven agent for integrating llms and knowledge graphs. External Links: 2404.07677, Link Cited by: Table 3, Table 4. - Q. Sun, J. Yuan, S. He, X. Guan, H. Yuan, X. Fu, J. Li, and P. S. Yu (2025) DyG-rag: dynamic graph retrieval-augmented generation with event-centric reasoning. arXiv preprint arXiv:2507.13396. Cited by: §8. - A. Talmor and J. Berant (2018) The web as a knowledge-base for answering complex questions. In NAACL-HLT, pp. 641–651. Cited by: Table 1, Table 6. - A. Talmor, J. Herzig, N. Lourie, and J. Berant (2019) CommonsenseQA: A question answering challenge targeting commonsense knowledge. In NAACL-HLT (1), pp. 4149–4158. Cited by: Table 1. - Y. Tan, H. Lv, X. Huang, J. Zhang, S. Wang, and C. Yang (2024a) MuseGraph: graph-oriented instruction tuning of large language models for generic graph mining. arXiv preprint arXiv:2403.04780. Cited by: §12.1, Table 2, Table 3, Table 5. - Y. Tan, Z. Zhou, H. Lv, W. Liu, and C. Yang (2024b) Walklm: a uniform language model fine-tuning framework for attributed graph embedding. Advances in Neural Information Processing Systems 36. Cited by: §5.1.3, Table 2, Table 3, Table 5. - J. Tang, Y. Yang, W. Wei, L. Shi, L. Su, S. Cheng, D. Yin, and C. Huang (2024) Graphgpt: graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 491–500. Cited by: Table 2, Table 3, §7.2.2, Table 5. - Y. Tang and Y. Yang (2024) MultiHop-rag: benchmarking retrieval-augmented generation for multi-hop queries. arXiv preprint arXiv:2401.15391. Cited by: Table 1, §5.6, Table 6. - D. Taunk, L. Khanna, S. V. P. K. Kandru, V. Varma, C. Sharma, and M. Tapaswi (2023) GrapeQA: graph augmentation and pruning to enhance question-answering. In WWW (Companion Volume), pp. 1138–1144. Cited by: Table 4. - K. Thakrar (2024) DynaGRAG: improving language understanding and generation through dynamic subgraph representation in graph retrieval-augmented generation. arXiv preprint arXiv:2412.18644. Cited by: §8. - S. Tonmoy, S. Zaman, V. Jain, A. Rani, V. Rawte, A. Chadha, and A. Das (2024) A comprehensive survey of hallucination mitigation techniques in large language models. arXiv preprint arXiv:2401.01313. Cited by: §2. - H. Touvron, L. Martin, and et al (2023) Llama 2: open foundation and fine-tuned chat models. External Links: 2307.09288, Link Cited by: §1, §5.4.2. - P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio (2018) Graph attention networks. External Links: 1710.10903, Link Cited by: §5.2.1. - D. Vrandečić and M. Krötzsch (2014) Wikidata: a free collaborative knowledgebase. Communications of the ACM 57 (10), pp. 78–85. Cited by: §1. - C. Wang, Y. Xu, Z. Peng, C. Zhang, B. Chen, X. Wang, L. Feng, and B. An (2023a) Keqing: knowledge-based question answering is a nature chain-of-thought mentor of llm. External Links: 2401.00426, Link Cited by: §5.4.2, Table 3, §6.3, Table 5. - H. Wang, S. Feng, T. He, Z. Tan, X. Han, and Y. Tsvetkov (2024a) Can language models solve graph problems in natural language?. Advances in Neural Information Processing Systems 36. Cited by: §12.1, Table 2, Table 3, Table 5. - K. Wang, F. Duan, S. Wang, P. Li, Y. Xian, C. Yin, W. Rong, and Z. Xiong (2023b) Knowledge-driven cot: exploring faithful reasoning in llms for knowledge-intensive question answering. External Links: 2308.13259, Link Cited by: §12.1, Table 4, Table 5. - X. Wang, P. Sen, R. Li, and E. Yilmaz (2024b) Adaptive retrieval-augmented generation for conversational systems. arXiv preprint arXiv:2407.21712. Cited by: §6.2. - X. Wang, Y. Ye, and A. Gupta (2018) Zero-shot recognition via semantic embeddings and knowledge graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6857–6866. Cited by: §10. - X. Wang, Q. Yang, Y. Qiu, J. Liang, Q. He, Z. Gu, Y. Xiao, and W. Wang (2023c) KnowledGPT: enhancing large language models with retrieval and storage access on knowledge bases. External Links: 2308.11761, Link Cited by: §12.1, §5.6, Table 2, Table 4, Table 5. - Y. Wang, Y. Zhu, W. Zhang, Y. Zhuang, Y. Li, and S. Tang (2024c) Bridging local details and global context in text-attributed graphs. External Links: 2406.12608, Link Cited by: §12.1, Table 2, Table 4, Table 5. - Y. Wang, N. Lipka, R. A. Rossi, A. F. Siu, R. Zhang, and T. Derr (2024d) Knowledge graph prompting for multi-document question answering. In AAAI, pp. 19206–19214. Cited by: §12.2, Table 1, Table 4, §7.3, Table 5. - Y. Wei, X. Wang, L. Nie, X. He, R. Hong, and T. Chua (2019) MMGCN: multi-modal graph convolution network for personalized recommendation of micro-video. In Proceedings of the 27th ACM international conference on multimedia, pp. 1437–1445. Cited by: §12.1, Table 2, Table 5. - Y. Wen, Z. Wang, and J. Sun (2024) MindMap: knowledge graph prompting sparks graph of thoughts in large language models. External Links: 2308.09729, Link Cited by: §12.2, Table 1, §5.4.1, Table 2, Table 3, Table 5. - F. Wu, L. Liu, W. He, Z. Liu, Z. Zhang, H. Wang, and M. Wang (2024a) Time-sensitve retrieval-augmented generation for question answering. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 2544–2553. Cited by: §8. - J. Wu, J. Zhu, and Y. Qi (2024b) Medical graph rag: towards safe medical large language model via graph retrieval-augmented generation. External Links: 2408.04187, Link Cited by: §4.2.1, Table 1, §5.5.2, Table 2, Table 3, Table 4, §7.3, Table 5. - Q. Wu, Z. Chen, W. Corcoran, M. Sra, and A. K. Singh (2024c) GraphEval2000: benchmarking and improving large language models on graph datasets. arXiv preprint arXiv:2406.16176. Cited by: §12.1, Table 2, Table 3, §7.4, Table 5, Table 6. - S. Wu, Y. Xiong, Y. Cui, H. Wu, C. Chen, Y. Yuan, L. Huang, X. Liu, T. Kuo, N. Guan, and C. J. Xue (2024d) Retrieval-augmented generation for natural language processing: a survey. External Links: 2407.13193, Link Cited by: §2. - S. Wu, S. Zhao, M. Yasunaga, K. Huang, K. Cao, Q. Huang, V. N. Ioannidis, K. Subbian, J. Zou, and J. Leskovec (2024e) STaRK: benchmarking llm retrieval on textual and relational knowledge bases. External Links: 2404.13207, Link Cited by: Table 6. - T. Wu, X. Bai, W. Guo, W. Liu, S. Li, and Y. Yang (2023a) Modeling fine-grained information via knowledge-aware hierarchical graph for zero-shot entity retrieval. In WSDM, pp. 1021–1029. Cited by: §4.2.1, Table 1, Table 2. - Y. Wu, N. Hu, S. Bi, G. Qi, J. Ren, A. Xie, and W. Song (2023b) Retrieve-rewrite-answer: a kg-to-text enhanced llms framework for knowledge graph question answering. External Links: 2309.11206, Link Cited by: §12.1, Table 3, Table 4, Table 5. - Y. Wu, Y. Fang, and L. Liao (2024f) Retrieval augmented generation for dynamic graph modeling. External Links: 2408.14523, Link Cited by: §8. - H. Yan, C. Li, R. Long, C. Yan, J. Zhao, W. Zhuang, J. Yin, P. Zhang, W. Han, H. Sun, et al. (2023) A comprehensive study on text-attributed graphs: benchmarking and rethinking. Advances in Neural Information Processing Systems 36, pp. 17238–17264. Cited by: Table 6. - S. Yan, J. Gu, Y. Zhu, and Z. Ling (2024) Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884. Cited by: §6.2. - J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, B. Yin, and X. Hu (2023) Harnessing the power of llms in practice: a survey on chatgpt and beyond. External Links: 2304.13712 Cited by: §2. - R. Yang, H. Liu, E. Marrese-Taylor, Q. Zeng, Y. H. Ke, W. Li, L. Cheng, Q. Chen, J. Caverlee, Y. Matsuo, and I. Li (2024a) KG-rank: enhancing large language models for medical qa with knowledge graphs and ranking techniques. External Links: 2403.05881, Link Cited by: §10, §12.1, Table 2, Table 4, Table 5. - R. Yang, B. Yang, A. Feng, S. Ouyang, M. Blum, T. She, Y. Jiang, F. Lecue, J. Lu, and I. Li (2024b) Graphusion: a rag framework for knowledge graph construction with a global perspective. arXiv preprint arXiv:2410.17600. Cited by: Table 6. - S. Yang, D. Wang, H. Zheng, and R. Jin (2025) Timerag: boosting llm time series forecasting via retrieval-augmented generation. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. Cited by: §8. - X. Yang, K. Sun, H. Xin, Y. Sun, N. Bhalla, X. Chen, S. Choudhary, R. D. Gui, Z. W. Jiang, Z. Jiang, L. Kong, B. Moran, J. Wang, Y. E. Xu, A. Yan, C. Yang, E. Yuan, H. Zha, N. Tang, L. Chen, N. Scheffer, Y. Liu, N. Shah, R. Wanga, A. Kumar, W. Yih, and X. L. Dong (2024c) CRAG – comprehensive rag benchmark. External Links: 2406.04744, Link Cited by: Table 6. - Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning (2018) HotpotQA: A dataset for diverse, explainable multi-hop question answering. In EMNLP, pp. 2369–2380. Cited by: Table 1, §7.4, Table 6. - Y. Yao, Z. Li, and H. Zhao (2023) Beyond chain-of-thought, effective graph-of-thought reasoning in language models. arXiv preprint arXiv:2305.16582. Cited by: §6.5, Table 4. - M. Yasunaga, H. Ren, A. Bosselut, P. Liang, and J. Leskovec (2021) QA-GNN: reasoning with language models and knowledge graphs for question answering. In NAACL-HLT, pp. 535–546. Cited by: §6.3, Table 4, Table 5. - X. Ye, S. Yavuz, K. Hashimoto, Y. Zhou, and C. Xiong (2021) Rng-kbqa: generation augmented iterative ranking for knowledge base question answering. arXiv preprint arXiv:2109.08678. Cited by: §12.1, Table 1, Table 2, Table 5. - W. Yih, M. Richardson, C. Meek, M. Chang, and J. Suh (2016) The value of semantic parse labeling for knowledge base question answering. In ACL (2), Cited by: §12.1, §4.1.1, §7.4, Table 5, Table 6. - Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec (2018) Hierarchical graph representation learning with differentiable pooling. Advances in neural information processing systems 31. Cited by: §10. - D. Yu, S. Zhang, P. Ng, H. Zhu, A. H. Li, J. Wang, Y. Hu, W. Y. Wang, Z. Wang, and B. Xiang (2023) DecAF: joint decoding of answers and logical forms for question answering over knowledge bases. In ICLR, Cited by: §12.1, Table 5. - D. Yu, C. Zhu, Y. Fang, W. Yu, S. Wang, Y. Xu, X. Ren, Y. Yang, and M. Zeng (2022) KG-fid: infusing knowledge graph in fusion-in-decoder for open-domain question answering. In ACL (1), pp. 4961–4974. Cited by: §4.2.1, §4.2.2, Table 1, Table 2, Table 5. - H. Yu, A. Gan, K. Zhang, S. Tong, Q. Liu, and Z. Liu (2024) Evaluation of retrieval-augmented generation: a survey. External Links: 2405.07437, Link Cited by: §2. - H. Zhang, Z. Kyaw, S. Chang, and T. Chua (2017) Visual translation embedding network for visual relation detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5532–5540. Cited by: §10. - J. Zhang, J. Chen, A. Maatouk, N. Bui, Q. Xie, L. Tassiulas, J. Shao, H. Xu, and R. Ying (2024a) LitFM: a retrieval augmented structure-aware foundation model for citation graphs. arXiv preprint arXiv:2409.12177. Cited by: §7.2.2. - J. Zhang, X. Zhang, J. Yu, J. Tang, J. Tang, C. Li, and H. Chen (2022) Subgraph retrieval enhanced model for multi-hop knowledge base question answering. In ACL (1), pp. 5773–5784. Cited by: §12.2, Table 4, Table 5. - M. Zhang, M. Sun, P. Wang, S. Fan, Y. Mo, X. Xu, H. Liu, C. Yang, and C. Shi (2024b) GraphTranslator: aligning graph model to large language model for open-ended tasks. In Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, pp. 1003–1014. Cited by: §1. - Q. Zhang, S. Chen, Y. Bei, Z. Yuan, H. Zhou, Z. Hong, J. Dong, H. Chen, Y. Chang, and X. Huang (2025) A survey of graph retrieval-augmented generation for customized large language models. arXiv preprint arXiv:2501.13958. Cited by: §1. - Q. Zhang, J. Dong, H. Chen, D. Zha, Z. Yu, and X. Huang (2024c) KnowGPT: knowledge graph based prompting for large language models. External Links: 2312.06185, Link Cited by: §12.1, Table 3, Table 4, Table 5. - Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al. (2023) Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. Cited by: §2. - Y. Zhang, H. Dai, Z. Kozareva, A. Smola, and L. Song (2018) Variational reasoning for question answering with knowledge graph. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Cited by: §12.1, Table 5, Table 6. - J. Zhao, L. Zhuo, Y. Shen, M. Qu, K. Liu, M. Bronstein, Z. Zhu, and J. Tang (2023a) GraphText: graph reasoning in text space. External Links: 2310.01089, Link Cited by: §12.1, Table 5. - P. Zhao, H. Zhang, Q. Yu, Z. Wang, Y. Geng, F. Fu, L. Yang, W. Zhang, J. Jiang, and B. Cui (2024a) Retrieval-augmented generation for ai-generated content: a survey. External Links: 2402.19473, Link Cited by: §2. - S. Zhao, Y. Yang, Z. Wang, Z. He, L. K. Qiu, and L. Qiu (2024b) Retrieval augmented generation (rag) and beyond: a comprehensive survey on how to make your llms use external data more wisely. arXiv preprint arXiv:2409.14924. Cited by: §2. - W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al. (2023b) A survey of large language models. arXiv preprint arXiv:2303.18223. Cited by: §2. - S. Zhou, B. Yu, A. Sun, C. Long, J. Li, H. Yu, J. Sun, and Y. Li (2022) A survey on neural open information extraction: current status and future directions. arXiv preprint arXiv:2205.11725. Cited by: §4.2.1. - X. Zhou, Z. Sun, and G. Li (2024a) Db-gpt: large language model meets database. Data Science and Engineering 9 (1), pp. 102–111. Cited by: §9. - Y. Zhou, Y. Liu, X. Li, J. Jin, H. Qian, Z. Liu, C. Li, Z. Dou, T. Ho, and P. S. Yu (2024b) Trustworthiness in retrieval-augmented generation systems: a survey. arXiv preprint arXiv:2409.10102. Cited by: §2. - Y. Zhu, J. Xiao, Y. Wang, and J. Sang (2024a) KG-fpq: evaluating factuality hallucination in llms with knowledge graph-based false premise questions. arXiv preprint arXiv:2407.05868. Cited by: §1. - Y. Zhu, C. Ren, S. Xie, S. Liu, H. Ji, Z. Wang, T. Sun, L. He, Z. Li, X. Zhu, et al. (2024b) REALM: rag-driven enhancement of multimodal electronic health records analysis via large language models. arXiv preprint arXiv:2402.07016. Cited by: Table 3, §7.3, Table 5. - Y. Zhu, Y. Wang, H. Shi, and S. Tang (2024c) Efficient tuning and inference for large language models on textual graphs. External Links: 2401.15569, Link Cited by: §12.1, Table 2, Table 4, Table 5. - Y. Zhu, H. Yuan, S. Wang, J. Liu, W. Liu, C. Deng, Z. Dou, and J. Wen (2023) Large language models for information retrieval: a survey. arXiv preprint arXiv:2308.07107. Cited by: §2. - Z. Zhu, H. Liu, M. He, and S. Luo (2025) Right answer at the right time-temporal retrieval-augmented generation via graph summarization. arXiv preprint arXiv:2510.16715. Cited by: §4.2.1, §5.6, §8. 12. Appendix 12.1. Brief Introduction of Works in Table 5 ReTraCk (Chen et al., 2021) utilizes a retriever, a transducer, and a checker to parse questions into syntax guaranteed logical forms. It retrieves entities and schema, applies a grammar based decoder, and performs instance level and ontology level checks with execution, thus reaching top accuracy with interactive latency on GrailQA and WebQuestionsSP. Golden-Retriever (An et al., 2024) utilizes reflection based question augmentation before retrieval by identifying jargon, inferring context, and rewriting the query. It also enriches the document store through optical character recognition and language model summarization, thus improving retrieval fidelity in industrial knowledge bases. Fact (Opsahl, 2024) utilizes efficient subgraph retrieval for fact verification over knowledge graphs and compares textual fine tuning, a question answering graph neural network hybrid, and large language model prompting. It reports higher accuracy and faster training on the FACTKG dataset. RNG-KBQA (Ye et al., 2021) utilizes a rank then generate pipeline in which a BERT based bi encoder scores logical form candidates and a T5 based sequence to sequence generator composes the final logical form. It achieves new state of the art results on GrailQA and WebQSP. TOG2 (Ma et al., 2024) alternates between graph retrieval and context retrieval to discover in-depth clues, and it tightly couples knowledge graphs with text so that each side improves the other. It aims for deep and faithful reasoning, and it remains training free and plug and play while delivering state-of-the-art accuracy on knowledge-intensive tasks. MINERVA (Das et al., 2018) learns a policy to walk on a knowledge graph conditioned on a query, and it answers by reaching the correct node. It supports variable-length paths, yields interpretable reasoning traces, and improves efficiency by searching local neighborhoods rather than ranking all entities. KALMV (Baek et al., 2023c) introduces a small instruction-tuned verifier that checks whether retrieved knowledge is relevant and whether the generated answer is grounded, and it rectifies errors by re-retrieving or regenerating when needed. It works as a plug and play module and reduces hallucinations across open-domain and knowledge-graph question answering. LLaGA (Chen et al., 2024b) adapts graphs to large language models by reorganizing nodes into structure-aware sequences and mapping them into the token embedding space through a projector. It preserves the general-purpose nature of language models and shows strong performance and interpretability across datasets and tasks. ROG (Luo et al., 2024b) proposes a planning, retrieval, and reasoning framework. It first generates relation paths as faithful plans grounded in the knowledge graph, then retrieves reasoning paths so the model answers with interpretable explanations. RRA (Wu et al., 2023b) introduces a retrieve, rewrite, and answer pipeline. It rewrites retrieved subgraphs into answer sensitive knowledge text with a KG to Text module and uses the text to guide the question answering model, which improves KGQA. KnowledgeNavigator (Guo et al., 2024) builds a three stage KGQA process. It predicts hop count and generates similar questions to constrain search, then iteratively retrieves a focused subgraph and composes a reasoning prompt to produce the answer. KD CoT (Wang et al., 2023b) turns Chain of Thought into a structured multi round dialogue with an external QA system. It verifies and amends intermediate steps using retrieved answers and thus yields more faithful reasoning and stronger results on WebQSP and ComplexWebQuestions. MVP Tuning (Huang et al., 2023b) combines multi view knowledge retrieval with parameter efficient prompt tuning. It augments the input with self view and consensus view triplets and jointly models text and knowledge in a single prompt tuned language model, which delivers state of the art accuracy on several commonsense QA benchmarks. KagNet (Lin et al., 2019) constructs a schema graph from an external commonsense graph and models it with a knowledge aware graph network that combines graph convolutional networks and LSTMs with hierarchical path attention, and it yields transparent reasoning and state of the art results on CommonsenseQA. KG-Agent (Jiang et al., 2024a) builds an autonomous agent that integrates a language model with a toolbox, a knowledge graph executor, and a memory, and it iteratively selects tools and updates memory to complete multi hop reasoning while a small tuned model surpasses stronger baselines on in domain and out of domain benchmarks. GenKGQA (Gao et al., 2024b) proposes a two stage generative framework for temporal knowledge graph question answering that first retrieves a question relevant temporal subgraph using language model guidance and then generates answers with instruction tuned fusion of graph signals and text through virtual knowledge indicators. UniKGQA (Jiang et al., 2023b) unifies retrieval and reasoning with a semantic matching module and a matching information propagation module, and it introduces abstract subgraphs and shared pre training and fine tuning so that it achieves large improvements on WebQSP and ComplexWebQuestions. Knowledge Solver (Feng et al., 2023) teaches language models to search a knowledge graph by turning retrieval into a multi hop decision sequence and by producing complete reasoning paths, and it improves zero shot and fine tuned performance on CommonsenseQA, OpenbookQA, and MedQA while increasing explainability. WebQSP (Yih et al., 2016) provides SPARQL labeled semantic parses for questions from WebQuestions and shows that such labels can be collected efficiently, and it demonstrates that training with semantic parses yields about five points higher average F1 and more consistent and updatable answers. MetaQA (Zhang et al., 2018) proposes a variational reasoning network that jointly handles uncertain topic entities and multi hop reasoning through a probabilistic framework, and it introduces a propagation style architecture trained with variance reduced reinforcement learning. It also releases a large scale benchmark of more than four hundred thousand questions across one hop to three hop settings and shows strong results on text and audio queries. PullNet (Sun et al., 2019) performs iterative retrieval to build a question specific subgraph and then reasons over this compact graph with a graph convolutional module. It learns which nodes to expand at each step and achieves large gains on multi hop datasets. KnowledGPT (Wang et al., 2023c) integrates large language models with external knowledge bases by generating program of thought search code to call KB functions, then executing the retrieved results for answering. It supports both retrieval and storage over public and private KBs without task specific training, and it differs from classical KBQA systems by covering diverse knowledge types. DecAF (Yu et al., 2023) jointly decodes logical forms and direct answers, and then combines the two signals to produce final predictions. It also linearizes knowledge bases into text to use general purpose retrieval, which improves adaptability across datasets. GGE (Gao et al., 2022) partitions a retrieved knowledge subgraph into smaller subgraphs and applies a graph augmented learning to rank model that captures global and local interactions between the question and subgraphs. It reduces search space and benefits subsequent answer selection in information retrieval based KGQA. NSM (He et al., 2021) adopts a teacher student design that couples forward and backward reasoning to learn reliable intermediate supervision signals and improve multi hop KBQA. It aligns entity distributions from both directions to mitigate spurious paths and strengthen the student model. SKP (Dong et al., 2023a) introduces structured knowledge aware pre training to narrow the gap between text and subgraphs. It adds an interval attention mechanism and an efficient linearization strategy, which improve retrieval and robustness under incomplete knowledge bases. KnowGPT (Zhang et al., 2024c) builds a knowledge graph based prompting framework for large language models. It extracts informative knowledge and constructs context aware prompts to reduce hallucination and raise accuracy on knowledge intensive tasks. DiFaR (Baek et al., 2023a) performs direct fact retrieval by embedding facts and queries in one space and retrieving triplets without entity linking. It then reranks top candidates, which avoids error propagation and yields strong results while remaining simple. KAPING (Baek et al., 2023b) augments zero shot prompting by retrieving relevant triples, verbalizing them as text, and injecting them into the prompt so answers are grounded in factual knowledge. It filters to the most similar triples and shows consistent gains across many language models. NuTrea (Choi et al., 2023) adopts a neural tree search scheme for multi hop knowledge graph question answering. It uses expansion, backup, and node ranking with a relation frequency inverse entity frequency embedding to incorporate broader graph context and improve path selection. OREOLM (Hu et al., 2022) inserts knowledge interaction layers into a transformer to interact with a knowledge graph reasoning module through contextualized random walks. It strengthens closed book question answering and provides interpretable reasoning paths. GPT4Graph (Guo et al., 2023) evaluates how large language models understand graph structured data across structural and semantic tasks, and it builds a benchmark while identifying current limits and future directions. GraphText (Zhao et al., 2023a) introduces a graph syntax tree that converts graphs into natural language prompts so an LLM can reason in text without extra training, and it injects graph inductive biases for stronger reasoning and explanation. ENGINE (Zhu et al., 2024c) presents an efficient tuning and inference framework for textual graphs by linking frozen language model layers with graph ladders in a side structure, and it uses caching and dynamic early exit to reduce time and memory while keeping accuracy high. GraphBridge (Wang et al., 2024c) proposes a multi granularity integration that bridges local node text and global topology through contextual textual information, and it adds a graph aware token reduction module for scalable learning with state of the art results. MMGCN (Wei et al., 2019) constructs a user item bipartite graph for each modality and propagates signals to capture modal specific preferences, and it shows clear gains on Tiktok, Kwai, and Movielens. ConvE (Dettmers et al., 2018) applies two dimensional convolutions over embeddings for knowledge graph link prediction with a simple yet expressive architecture, and it reaches state of the art mean reciprocal rank with efficient one to many scoring. Graph-LLM (Chen et al., 2024c) explores two pipelines for text-attributed graphs, using large language models either to enhance node attributes alongside graph neural networks or to act as direct predictors for node classification; it finds enhancer pipelines effective while predictor pipelines show preliminary promise yet risk inaccurate outputs and evaluation leakage. NLGraph (Wang et al., 2024a) introduces a natural-language benchmark of 29,370 problems across eight graph reasoning tasks and tests whether prompting strategies improve model reasoning; it shows gains on simple tasks with chain-of-thought style prompting, yet the benefits diminish and can reverse on complex problems. GLBench (Li et al., 2024d) provides a comprehensive benchmark for GraphLLM methods in supervised and zero shot settings, unifying datasets and splits to compare enhancer, predictor, and aligner roles; it reports that enhancer methods are robust, predictor methods are less reliable, and a training-free baseline can be competitive. GraphEval2000 (Wu et al., 2024c) constructs a code-based evaluation suite with 40 graph problems and 2,000 test cases and adds structured symbolic decomposition to guide solutions; it returns failed cases for feedback and shows sizable gains for popular models under the proposed instruction design. MuseGraph (Tan et al., 2024a) proposes a graph-oriented instruction-tuning framework that builds compact graph descriptions, generates diverse chain-of-thought instructions, and applies graph-aware tuning; it achieves superior results across five tasks and ten datasets and supports dynamic allocation of instruction volume by task and dataset complexity. KG-Rank (Yang et al., 2024a) integrates a medical knowledge graph with ranking and re ranking to support long form medical question answering. It identifies entities, retrieves related triples, and orders them to improve factuality and relevance for the final answer. FABULA (Ranade and Joshi, 2023) constructs an event plot graph from open source reports and guides stepwise retrieval and reasoning for intelligence analysis. It then drafts structured situation reports with evidence tracing and explanation. FoodGPT (Qi et al., 2023) is a domain large language model for food testing that adds incremental pre training and instruction tuning. It integrates an external knowledge graph for retrieval to reduce hallucination and to support precise outputs. KAM CoT (Mondal et al., 2024) trains a light verifier to check retrieval relevance and grounding during knowledge augmented generation. It detects retrieval errors and grounding errors and it triggers corrective actions to improve reliability. MufassirQAS (Alan et al., 2024) curates Quranic sources and builds a retrieval augmented pipeline with knowledge driven summarization. It ranks and synthesizes evidence to produce faithful answers with clarity for non expert users. MEDQA (Jin et al., 2020) is an open domain multiple choice dataset drawn from medical board examinations in the United States, Mainland China, and Taiwan. It requires retrieval over medical textbooks and logical reasoning for professional decisions. DepsRAG (Alhanahnah et al., 2024) is a multi agent RAG system for software dependency analysis that builds a dependency graph and answers queries over it. It coordinates assistant, search, dependency, and critic agents to decompose tasks and to validate outputs. 12.2. Additional Related Works GLBK (Delile et al., 2024) introduces a novel retrieval method for biomedical knowledge that leverages a knowledge graph to address the limitations of embedding-based similarity retrieval. This method constructs a biomedical knowledge graph by utilizing entity recognition and relationship extraction techniques, specifically fine-tuning PubmedBERT (Gu et al., 2022) for the biomedical domain. This graph builds text chunks from the PubMed https://ftp.ncbi.nlm.nih.gov/pubmed/ corpus by associating them with nodes (entities like genes, diseases, or compounds) and edges (relationships). Unlike embedding-based retrieval methods, which tend to prioritize over-represented clusters in the data, this graph-based approach re-balances information by under-sampling dense clusters and ensuring access to long-tail, less frequent yet valuable knowledge. By combining the semantic strengths of embedding similarity with the structural insights from the knowledge graph, the proposed hybrid model demonstrates superior performance of accuracy, addressing the information overload problem prevalent in biomedical research retrieval. ATLANTIC (Munikoti et al., 2023) is a structure-aware retrieval-augmented language model that addresses limitations in current RAG systems by integrating document structural relationships. A heterogeneous document graph is constructed from scientific literature, capturing four types of relationships: co-citation, co-topic, co-venue, and co-institution, to connect documents across 15 scientific disciplines. This structure is used to encode the relationships with a pre-trained Heterogeneous Graph Transformer (HGT) (Hu et al., 2020), which generates structural embeddings. These embeddings are fused with semantic embeddings from text, enabling ATLANTIC to retrieve passages that are both semantically relevant and structurally coherent. The approach mitigates the shortcomings of text-only retrieval methods, particularly in interdisciplinary domains where relational context is crucial. The model demonstrates improved faithfulness and relevance in retrieving passages for scientific tasks like question answering and document classification. GNN-Ret (Li et al., 2024f) constructs a database tailored for improved passage retrieval in complex question-answering scenarios. The authors create a ”Graph of Passages” (GoPs) by linking textual passages based on two key principles: structural relationships and shared keywords. Structural connections are established by linking adjacent passages in documents, maintaining their order and context. Keyword connections are derived by extracting entities using LLMs, and then linking passages that share the same keywords to highlight semantic associations. This graph-based representation addresses challenges like retrieving semantically distant but related passages, enabling improved retrieval coverage and more accurate question answering. The innovative use of GoPs mitigates the limitation of traditional database, which often treat passages as isolated units, and introduces an efficient way to model and retrieve interconnected information. Explore-then-Determine (EtD) (Liu et al., 2024a) utilizes a graph convolutional method to enhance graph retrieval by employing adaptive propagation and message passing. In the Explore phase, the framework initializes with a topic entity and uses a lightweight GNN, integrated with semantic representations from LLMs, to expand the candidate set by propagating to relevant neighbors. Attention weights are computed to rank and prune irrelevant edges, focusing on the top- $K$ edges at each layer to refine candidate sets. In the Determine phase, the filtered graph is processed by a frozen LLM with a knowledge-enhanced multiple-choice prompt, merging explicit graph-based knowledge with the LLM’s implicit understanding to determine the final answer. This approach efficiently filters irrelevant KG information while enabling precise multi-hop reasoning. GRAG (Hu et al., 2024) introduces a dual approach to graph-structure prompting for integrating textual graphs into LLMs. In the Graph View, textual graphs are encoded as soft prompts using a Graph Neural Network (GNN) to preserve topological information. Retrieved subgraph embeddings are aligned with LLM token embeddings via an MLP, ensuring seamless integration of graph structure. In the Text View, textual graphs are transformed into hierarchical text descriptions as hard prompts. This involves splitting retrieved ego-graphs into a Breadth-First Search (BFS) tree and residual edge set, then organizing them into structured templates to retain narrative and topological context. The query, hierarchical text description, and graph embeddings are concatenated as input to the LLM, enabling context-aware generation that leverages both structural and semantic aspects of the graph. MindMap (Wen et al., 2024) creates prompts for LLMs by converting knowledge graph subgraphs into natural language descriptions. First, it mines evidence subgraphs using path-based and neighbor-based exploration techniques to gather relevant information. Then, it performs an evidence graph aggregation step, where each subgraph is formatted as an entity chain, such as $\mathcal{G}^{\prime}=\{(e_{h},r,e_{t})\}$ , which is translated into a natural sentence by a predefined template. These descriptions are merged to form a reasoning graph, which is included as part of the LLM’s prompt. This approach ensures that the LLM understands both the structural and semantic aspects of the graph, enabling robust and context-aware reasoning. KGP (Wang et al., 2024d) formulates the process of generating knowledge graph prompts for multi-document question answering scenarios into two steps. First, a general knowledge graph is constructed based on the passage similarity from multiple documents. During the process, the merits and limitations of different construction methods are examined, to build the mapping between the most appropriate methods and scenarios. Second, a language model agent is employed to traverse the knowledge graph and retrieve relevant context. The agent traverses over the KG through the new evidence given by the LLM iteratively according to the retrieved passages. The final structured passage queue from the KG services acts as the prompt for multi-document question answering. ArcaneQA (Gu and Su, 2022) is designed to address the combinatorial explosion problem commonly encountered in searching for matched subgraphs within large-scale knowledge bases. Unlike traditional approaches that involve an exhaustive search, ArcaneQA narrows its focus to a small set of admissible tokens derived from the relevant subgraph, rather than considering the entire vocabulary. This selective prediction significantly reduces computational complexity. Utilizing an encode-decode architecture built on top of the BERT model, ArcaneQA iteratively predicts additional sequences from the subgraph, allowing for efficient stepwise progression. By breaking down the task of program generation into a sequence of smaller, more manageable decision-making processes, ArcaneQA effectively transforms a daunting search challenge into a tractable series of localized predictions, streamlining the reasoning process while maintaining accuracy. EWEK-QA (Dehghan et al., 2024) triggers two pipelines simultaneously to extract informative knowledge from the web content and the knowledge base. For the retrieval from the web text, EWEK-QA first retrieves the relevant content utilizing the Bing search and then conducts a series of processes to filter and rerank the candidate texts from the web pages. Simultaneously, EWEK-QA avoids retrieval techniques relying on LLMs such as TOG (Sun et al., 2024a) and directly employs a match of representation similarity (cosine score) to obtain the relevant triples. To this end, EWEK-QA further utilizes a pre-trained LLMs to integrate KG triples and web quotes together and construct a unified prompt. By substituting the iterative retrieval in the knowledge base and enhancing it with web search, EWEK-QA significantly improves the quality of extracted knowledge in terms of relevance to the queries without hampering efficiency. SR (Zhang et al., 2022) aims to enhance KGQA by efficiently retrieving high-quality subgraphs closely aligned with the query. It achieves this by decoupling the retrieval process from the reasoning step, enabling a plug-and-play framework that can be integrated with various KGQA models. SR uses a dual-encoder architecture to expand and refine paths, automatically stopping when relevant subgraphs are constructed. This separation reduces reasoning bias caused by partial subgraphs and improves retrieval precision. By employing weakly supervised and end-to-end fine-tuning strategies, SR demonstrates significant improvements in reasoning accuracy, explainability, and retrieval efficiency, achieving state-of-the-art results in complex multi-hop KGQA tasks. LLMs-EP (Chen et al., 2024c) focuses on understanding how text-attribute GNNs can leverage LLMs to enhance node classification tasks. This paper presents an empirical study of two distinct pipelines for incorporating LLMs into graph tasks. In the first pipeline, LLMs are employed as enhancers targeting the text content associated with each node. Specifically, LLMs-EP utilizes LLMs either as embedding encoders to generate node representations or to incorporate additional textual content from related nodes. These approaches are termed feature-level and text-level enhancement, respectively, enabling more informative representations of nodes. In the second pipeline, LLMs-EP provides an in-depth analysis of two scenarios: first, exploring whether LLMs can directly predict node categories, and second, examining the impact of using 2-hop neighbors to incorporate structural information into the prediction process. By combining text and structural data, LLMs-EP aims to improve the accuracy and understanding of node classification in graph-based tasks.