2407.08516v5

# Converging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Empowered Autonomous Agents **Authors**: Haoyi Xiong, Zhiyuan Wang, Xuhong Li, Jiang Bian, Zeke Xie, Shahid Mumtaz, Anwer Al-Dulaimi, and Laura E. Barnes > H. Xiong and J. Bian are with Microsoft Corporation. Z. Wang and L. E. Barnes are with the University of Virginia. Xuhong Li is with Univiersité de technologie de Compiègne. Z. Xie is with Hong Kong University of Science and Technology (GZ). S. Mumtaz is with Nottingham Trent University, Nottingham, UK and Silesian University of Technology, Gliwice, Poland. Anwer Al-Dulaimi is with College of Technological Innovation, Zayed University, Abu Dhabi, UAE. ## Abstract This article explores the convergence of connectionist and symbolic artificial intelligence (AI), from historical debates to contemporary advancements. Traditionally considered distinct paradigms, connectionist AI focuses on neural networks, while symbolic AI emphasizes symbolic representation and logic. Recent advancements in large language models (LLMs), exemplified by ChatGPT and GPT-4, highlight the potential of connectionist architectures in handling human language as a form of symbols. The study argues that LLM-empowered Autonomous Agents (LAAs) embody this paradigm convergence. By utilizing LLMs for text-based knowledge modeling and representation, LAAs integrate neuro-symbolic AI principles, showcasing enhanced reasoning and decision-making capabilities. Comparing LAAs with Knowledge Graphs within the neuro-symbolic AI theme highlights the unique strengths of LAAs in mimicking human-like reasoning processes, scaling effectively with large datasets, and leveraging in-context samples without explicit re-training. The research underscores promising avenues in neuro-vector-symbolic integration, instructional encoding, and implicit reasoning, aimed at further enhancing LAA capabilities. By exploring the progression of neuro-symbolic AI and proposing future research trajectories, this work advances the understanding and development of AI technologies. Index Terms: Large Language Models (LLMs), LLM-Empowered Autonomous Agents (LAAs), Neuro-symbolic AI, Program-Proof-of-Thoughts (P 2 oT) prompting ## I Introduction Artificial Intelligence (AI) has historically navigated the fascinating duality of two foundational paradigms: connectionism and symbolism. Connectionism, deeply influenced by cognitive science and computational neuroscience, delves into neural networks and machine learning algorithms that echo the deep neural architecture and functions of the human brain [1]. Imagine a sprawling network of neurons firing in electric synchrony, mirroring how advanced AI systems identify patterns and glean insights from vast datasets. Conversely, symbolism is the epitome of conceptual and logical clarity. It anchors itself in the high-level abstractions and representations of knowledge, flourishing through rule-based systems that excel in reasoning and decision-making [2]. Picture a grand library where every book is a rule, and every chapter a pathway to logical deduction–symbolic AI analogising the thought processes of human reasoning. The dynamic interplay between these two paradigms has sculpted the continuous evolution of AI, like a grand philosophical debate, resulting in shifts in dominance and application across various research domains. Think of this dialectic as a dance through time—the elegant waltz of connectionism and symbolism, sometimes leading, sometimes following, yet always in a harmonious exchange that propels the boundaries of what AI can achieve. For instance, in the domain of image recognition, connectionist models driven by deep neural networks demonstrate their prowess by identifying subtle patterns in pixel data, akin to how our brains recognize faces in a crowd [3]. Meanwhile, in expert systems used for medical diagnostics, symbolism shines by methodically applying predefined rules to diagnose diseases, mimicking the logical flow of a doctor’s thought process [4]. This storied dance of paradigms has not just shaped, but revitalized AI, continuing to impact its trajectory as it ventures into increasingly sophisticated applications. The oscillation of dominance between these approaches resembles the ebb and flow of tides, each rise and retreat bringing new insights and innovations to the fore. In recent years, the advancements in Large Language Models (LLMs) and foundation models have catalyzed the integration of connectionist and symbolic AI paradigms, realizing new levels of computational intelligence and versatility [5]. These models, exemplified by systems such as OpenAI’s GPT-4, have demonstrated unprecedented capabilities in natural language understanding and generation, exhibiting robust performance across a range of complex tasks [6]. LLMs themselves are a triumph of connectionism, empowered by vast amounts of data and sophisticated neural architectures to produce coherent and contextually relevant texts. Moreover, the emergence of LLM-empowered Autonomous Agents (LAAs) signifies a pivotal juncture in the development of AI, embodying the convergence of symbolic and connectionist AI. As shown in Figure 1, LAAs combine a symbolic subsystem, utilizing language-based knowledge, rules, and workflows intrinsic to symbolic AI, with the generative capabilities of LLMs [7]. This symbolic subsystem works seamlessly with the neural subsystem and incorporates external tools for perceptions and actions [8]. LAAs demonstrate advanced reasoning, planning, and decision-making abilities, marking a new era in AI. The dual subsystems align with dual-process theories of reasoning [9] and Systems I and II proposed by Yoshua Bengio [10]. <details> <summary>x1.png Details</summary>  ### Visual Description ## Circular Diagram: LLM-Empowered Agent Architecture ### Overview The diagram illustrates a tripartite architecture for an LLM-Empowered Agent, divided into three concentric sections: 1. **External Tools** (blue) 2. **Large Language Models (Neural Sub-System)** (yellow) 3. **Agentic Workflows (Symbolic Sub-System)** (red) A central white circle labeled "LLM-Empowered Agent" contains core functionalities, surrounded by a legend featuring a robot icon. --- ### Components/Axes #### Legend - **Central Icon**: Robot figure in a suit with antenna, symbolizing the agent. - **Color Coding**: - Blue = External Tools - Yellow = Large Language Models (Neural Sub-System) - Red = Agentic Workflows (Symbolic Sub-System) #### Main Sections 1. **External Tools (Blue)** - Sub-components: - Database (Vector/Relational) - Sensors and Actuators - Local & Remote APIs/Web Services - Robotics and Embodies 2. **Large Language Models (Neural Sub-System) (Yellow)** - Sub-components: - Text Understanding & Generation - Question Answering (QA) - In-Context Learning (ICL) - Instruction Following 3. **Agentic Workflows (Symbolic Sub-System) (Red)** - Sub-components: - Pre-defined Rules & Logic - Chain/Tree-of-Thought - Self-reflection & other prompting tricks - Functionals and Procedures #### Core Agent Functions (White Circle) - Natural Language Interfacing - Decision Making and Planning - Task Decomposition and Actioning --- ### Detailed Analysis #### External Tools (Blue) - **Database**: Explicitly lists vector and relational databases. - **Sensors/Actuators**: Implies hardware integration for data collection and physical interaction. - **APIs/Web Services**: Distinguishes between local and remote connectivity. - **Robotics/Embodies**: Suggests physical deployment capabilities. #### Large Language Models (Yellow) - **Text Understanding/Generation**: Core NLP capabilities. - **QA/ICL**: Focus on dynamic, context-aware responses. - **Instruction Following**: Emphasizes adherence to user directives. #### Agentic Workflows (Red) - **Pre-defined Rules**: Structured logic for deterministic behavior. - **Chain/Tree-of-Thought**: Enables complex reasoning via sequential or branching logic. - **Self-reflection/Prompting**: Meta-cognitive capabilities for iterative improvement. - **Functionals/Procedures**: Modular execution of tasks. #### Core Agent Functions - **Natural Language Interfacing**: Bridges human language with system operations. - **Decision Making/Planning**: Strategic task allocation. - **Task Decomposition/Actioning**: Breaks down goals into executable steps. --- ### Key Observations 1. **Modular Design**: Each subsystem (neural, symbolic, external) operates independently but integrates through the central agent. 2. **Color Consistency**: Legend colors (blue/yellow/red) match their respective sections without overlap. 3. **Hierarchical Structure**: Core functions are nested within the agent, while sub-components radiate outward. 4. **Symbiotic Relationships**: Neural and symbolic subsystems complement each other (e.g., LLMs handle ambiguity, workflows enforce logic). --- ### Interpretation The diagram represents a hybrid AI architecture where: - **Neural Sub-Systems** (yellow) provide adaptability and contextual understanding. - **Symbolic Sub-Systems** (red) ensure reliability through rule-based logic. - **External Tools** (blue) supply real-world data and physical interaction capabilities. The central agent acts as a mediator, translating between subsystems and executing tasks. This design balances flexibility (via LLMs) with precision (via symbolic workflows), enabling robust, context-aware decision-making. The inclusion of robotics and APIs suggests applications in embodied AI systems requiring both digital and physical interaction. </details> Figure 1: Elements of LLM-empowered Autonomous Agents (LAAs): Large Language Models (Neural Sub-System), Agentic Workflows (Symbolic Sub-System), and External Tools In this work, we aim at examining the historical evolution and current state of AI by exploring the enduring debate between connectionism and symbolism and their convergence in modern technologies, particularly in the theme of neuro-symbolic approaches, including Knowledge Graphs, LLMs, and LAAs. This review aims to illustrate how the integration of these paradigms has led to groundbreaking advancements, offering new perspectives on the capabilities and future directions of AI. - Historical Context of Technology: This article provides an in-depth examination of the historical debate between connectionism and symbolism, contextualizing modern AI developments and highlighting the strengths of each approach. We present recent advancements in LLMs with Knowledge Graphs (KGs) [11] as references, discussing these techniques from the perspectives of symbolic, connectionist, and neuro-symbolic AI. The article also showcases the transformative impact of these techniques on knowledge modeling, acquisition, representation, and reasoning. - Convergence of Paradigms: This article highlights the convergence of symbolic and connectionist approaches in developing LAAs, emphasizing their enhanced reasoning, decision-making, and efficiency. By contrasting LAAs with Knowledge Graphs (KGs) within neuro-symbolic AI, we examine distinct patterns and functionalities. While both integrate symbolic and neural methodologies, LAAs demonstrate unique advantages over KGs: (1) analogizing human reasoning with agentic workflows and various prompting techniques [12, 13], (2) scaling effectively on large datasets, adapting to in-context samples, and leveraging the emergent abilities of LLMs. These strengths drive the surge of a new wave of neuro-symbolic AI [14]. - Future Directions: The article highlights the trend of converging paradigms and current limitations of LAAs, pointing to two promising future directions. First is the development of neuro-vector-symbolic architectures, which integrate vector manipulation to enhance the reasoning capabilities of agents. Second is the approach known as program-proof-of-thoughts (P 2 oT) prompting. This involves breaking down complex reasoning processes into verifiable propositions, utilizing program proof languages (such as Dafny) for structured verification. It aims to provide rigorous reasoning by modeling propositions, integrating with theorem provers, and focusing on applications in specific domains. These contributions are crucial as they provide a comprehensive understanding of the evolution of AI, highlight the significance of paradigm convergence, and offer insights into future research and application potentials in the rapidly evolving field of AI. ## II Preliminaries This section begins by summarizing the historical debate between connectionist AI and symbolic AI. We then explore knowledge graphs (KGs) as an early effort to synergize these two paradigms through neuro-symbolic AI. Lastly, we examine LLMs as the latest advancements in connectionist AI. <details> <summary>x2.png Details</summary>  ### Visual Description ## Timeline Diagram: Evolution of Artificial Intelligence Paradigms ### Overview This diagram illustrates the historical progression of AI research paradigms from 1956 to the present, highlighting key milestones, debates, and technological breakthroughs. It contrasts Symbolic AI and Connectionist AI approaches, culminating in modern Neuro-Symbolic Fusion frameworks. ### Components/Axes - **X-axis (Time)**: 1956 → 2022-Now, divided into decades (1950s, 1980s, 1990s, 2000s, 2010s) - **Y-axis (AI Paradigms)**: - Symbolic AI (represented by book icon) - Connectionist AI (represented by brain icon) - **Key Elements**: - Dotted line connecting paradigms (symbolizing integration) - Neuro-Symbolic Fusion section (2010s onward) - Technology icons (e.g., neural networks, transformers) ### Detailed Analysis 1. **Symbolic AI Era (1956-1980s)** - 1956: Logic Theorist (Introduction of symbolic AI) - 1958: Perceptron (Initiated connectionist AI) - 1980s: Backpropagation (Significant advancement in neural network training) 2. **1990s Developments** - RDF (Standardization of data interchange on the web) - Ontologies (Structuring and organizing knowledge) - Semantic Web (Enhancing data interoperability) 3. **2000s Machine Learning** - SVM, Kernels, Sparse Models, Boosting (Technical advancements) 4. **2010s Neuro-Symbolic Fusion** - AlexNet (2012): Revolutionized image recognition - Knowledge Graphs (Leveraging probabilistic Markov Logic and neural networks) 5. **2017-2018 Transformers** - BERT, GPT (Revolutionized NLP and language understanding) 6. **2022-Now Generative AI** - ChatGPT (Unprecedented capabilities in natural language processing) ### Key Observations - **Paradigm Shift**: Connectionist AI (neural networks) gained prominence after initial symbolic dominance - **Integration Trend**: Dotted line shows increasing convergence between symbolic and connectionist approaches - **Acceleration**: Breakthroughs became more frequent post-2010s (AlexNet → Transformers → Generative AI) - **Technical Depth**: Each era includes specific technical innovations (e.g., backpropagation, transformers) ### Interpretation This timeline demonstrates AI's evolution from rule-based systems (Symbolic AI) to data-driven neural networks (Connectionist AI), with modern research focusing on hybrid approaches. The 2010s marked a turning point with deep learning breakthroughs (AlexNet), followed by transformer architectures enabling advanced language models. The Neuro-Symbolic Fusion section suggests current efforts to combine probabilistic reasoning with neural networks for more robust AI systems. The diagram emphasizes both technical milestones and conceptual debates (e.g., "AI debates" label), reflecting the field's dynamic nature. The progression from specialized systems (Logic Theorist) to general-purpose models (ChatGPT) highlights AI's trajectory toward human-like cognitive capabilities. </details> Figure 2: Exploring the Evolution of Artificial Intelligence: A Timeline of Key Innovations and Milestones. It starts from the birth of symbolic and connectionist AI in the 1950s, through key milestones like the AI debates of the 1980s and the advancement in machine learning in the 1990s. This figure highlights significant developments such as the impact of AlexNet on image recognition, the transformation in NLP by models like BERT and GPT, and the rise of generative AI, culminating in the use of LLMs and Agents for autonomous decision-making in the 2020s. ### II-A Connectionism vs. Symbolism: a Historical Debate on AI As shown in Figure 2, the discourse of AI has long revolved around the dichotomy between connectionism and symbolism, two paradigms integral to the field. Connectionism models cognitive processes through artificial neural networks that emulate the brain’s neuron structures, emphasizing learning through algorithms and pattern recognition. This began with Frank Rosenblatt’s Perceptron in 1958 [15] and advanced significantly with the backpropagation algorithm developed by David Rumelhart, Geoffrey Hinton, and Ronald J. Williams in the 1980s [16], setting the stage for modern deep learning [1]. Conversely, symbolism focuses on high-level knowledge representations and symbolic manipulation to mimic human reasoning, gaining prominence with systems like the Logic Theorist by Allen Newell and Herbert A. Simon in 1956 [17]. Symbolic AI thrived with expert systems such as MYCIN [4] and DENDRAL [18] in the 1970s and 1980s, excelling in specific domains through predefined rules. In the 1980s, as Ashok Goel noted, debates often involved criticisms that attacked caricatures of the opposing methods [19]. Each approach has its limitations: connectionist AI is criticized for its black-box nature and lack of interpretability [20], while symbolic AI faced challenges with the labor-intensive knowledge acquisition process [21] and its limited adaptability [22]. Historical debates between figures, such as Yann LeCun, Yoshua Bengio, and Gary Marcus, have underscored these limitations [23]. However, the integration of both paradigms has led to robust hybrid models, combining neural networks’ pattern recognition with symbolic systems’ interpretability and logical reasoning [24]. Contemporary research exemplifies this convergence, seen in neuro-symbolic AI and large-scale pre-trained models like BERT [25], GPT [5], and hybrid reinforcement learning models [26], reflecting the ongoing evolution inspired by the historical debate. ### II-B Knowledge Graphs: An Early Neuro-symbolic Attempt Knowledge graphs have a foundation rooted in the evolution of semantic web technologies and the Resource Description Framework (RDF). Proposed by the W3C in the 1990s, RDF standardized data interchange on the web using triples (subject, predicate, object) for seamless data integration and interoperability [27]. This movement established the Semantic Web, aiming for a more intelligent and interconnected web [28]. Early adopters used RDF to build schemas and taxonomies, forming the basics of modern knowledge graphs [29]. As the field matured, the focus shifted towards capturing complex relationships and domain-specific knowledge. Ontologies, formal specifications of concepts and relationships, provided a framework for annotating and interlinking data, enabling semantic reasoning at a certain level [30]. Markov-logic networks introduced probabilistic reasoning to knowledge graphs, allowing for handling uncertainty and inconsistency in data [31]. The synergy of Ontologies and Markov-logic networks advanced the ability of symbolic AI to perform robust reasoning over large datasets [32]. In recent years, the use of graph neural networks (GNNs) has further revolutionized the landscape of knowledge graphs. GNNs adeptly leverage the graph structure for advanced pattern recognition and complex predictions. They excel in tasks such as node classification, link prediction, and the extraction of hidden patterns from graph-structured data [33]. This paradigm shift towards neural networks marks a convergence with modern machine learning techniques, enabling more nuanced and scalable interpretations of often massive and intricate datasets. The ability of GNNs to embed nodes and entire graphs numerically has significantly enhanced the computational handling of knowledge graphs [34]. In conclusion, the integration of graph neural networks with rule-based reasoning has positioned knowledge graphs at the core of the neuro-symbolic AI approach [11] prior to the surge of LLMs. ### II-C LLMs: Recent Connectionist AI Advancements The field of connectionist AI has undergone substantial evolution, beginning with the invention of the perceptron [15], kicking off the neural network research in the late 1950s. In the following decades, the development of Multi-Layer Perceptrons (MLPs) introduced hidden layers and non-linear activation functions, enabling the modeling of more complex functions [16]. In the 1990s, Long Short-Term Memory (LSTM) networks were developed to address the limitations of traditional recurrent neural networks (RNNs) by introducing gating mechanisms to handle long-term dependencies in sequential data [35]. Self-attention mechanisms and transformer architectures proposed in the late 2010s further revolutionized sequence modeling, such as texts for natural language processing, by allowing models to focus on different parts of the input sequence when generating each part of the output sequence [36]. The development of transformer-based pre-trained language models has significantly advanced natural language processing (NLP). These architectures include encoder-only models, e.g., BERT [25], which excel at understanding and classifying text; decoder-only models, e.g., GPT [6], which generate coherent and contextually relevant text; and encoder-decoder models, e.g., T5 [37], which are effective in tasks requiring both comprehension and generation. Transformer-based language models, such as OpenAI’s GPT-4 [38], Google’s Gemini [39] and PaLM [40], Microsoft’s Phi-3 [41], and Meta’s LLaMA [42], are termed Large Language Models (LLMs). These models, illustrated in Figure 3, are trained on large-scale transformers comprising billions of learnable parameters to support various abilities to enable agents, including perception, reasoning, planning, and action [12]. As the central component of an agent’s neural sub-system, the larger the model, the stronger the agent’s capability. <details> <summary>x3.png Details</summary>  ### Visual Description ## Scatter Plot: LLM Agent Benchmark Scores vs. Release Time ### Overview The image is a scatter plot visualizing the performance (LLM Agent Benchmark Score) of various large language models (LLMs) over time, with bubble size and color indicating parameter count. The x-axis represents release dates from March 2022 to December 2023, while the y-axis shows benchmark scores ranging from 0 to 4.0. Larger, yellower bubbles represent models with more parameters. ### Components/Axes - **X-Axis (Release Time)**: Dates from 2022-03 to 2023-12, spaced approximately every 3 months. - **Y-Axis (LLM Agent Benchmark Score)**: Scale from 0 to 4.0, with increments of 1.0. - **Color Scale (Parameter Number)**: Gradient from purple (0.2B parameters) to yellow (1.6T parameters), labeled "Parameter number (in billions)". - **Legend**: Positioned on the right, using a vertical color bar with parameter size labels (0.2B to 1.6T). ### Detailed Analysis 1. **Data Points**: - **OpenAI GPT-4***: 1.8T parameters (yellow), score 4.0 (top-right). - **Anthropic Claude***: 1.0T parameters (yellow-green), score 2.5 (mid-right). - **OpenAI GPT-3.5 Turbo***: 175B parameters (green), score 2.3 (lower-right). - **OpenAI Text Davinci 003**: 175B parameters (green), score 1.8 (mid-left). - **Meta CodeLlama 34B**: 34B parameters (blue), score 0.9 (lower-center). - **Meta Llama 2 70B**: 70B parameters (blue-green), score 0.8 (lower-center). - **Hugging Face Guanaco 65B**: 65B parameters (blue-green), score 0.7 (lower-center). - **LMSys Vicuna 33B**: 33B parameters (blue), score 0.6 (lower-center). - **OpenAI Text Davinci 002**: 12B parameters (purple), score 1.2 (bottom-left). 2. **Trends**: - **Parameter Size vs. Score**: Larger models (e.g., GPT-4, Claude*) generally achieve higher scores, but exceptions exist (e.g., GPT-3.5 Turbo outperforms smaller models like CodeLlama 34B). - **Temporal Progression**: Newer models (post-2023-06) show mixed performance. GPT-4 (2023-12) dominates, while older models like Davinci 002 (2022-03) lag. - **Color Consistency**: Bubble colors align with parameter sizes (e.g., GPT-4’s yellow matches its 1.8T parameter count, exceeding the color scale’s maximum of 1.6T). ### Key Observations - **Outliers**: GPT-4* is the clear outlier, with the highest score and parameter count. Older models like Davinci 002 underperform despite smaller parameter counts. - **Company Trends**: OpenAI dominates recent high-performing models, while Meta and Hugging Face models cluster in the lower score range. - **Color Scale Limitation**: GPT-4’s parameter size (1.8T) exceeds the color scale’s maximum (1.6T), suggesting the scale may not fully represent the largest models. ### Interpretation The data highlights a correlation between parameter count and performance, but architectural efficiency and training data quality also play critical roles. GPT-4’s exceptional score suggests breakthroughs beyond mere scale, possibly in model architecture or training techniques. The mixed performance of similarly sized models (e.g., GPT-3.5 Turbo vs. CodeLlama 34B) underscores the importance of optimization strategies. Temporal trends indicate rapid advancements, with newer models generally outperforming older ones, though exceptions persist. The color scale’s limitation for GPT-4 hints at the need for updated visualization tools to accommodate future models. </details> Figure 3: Large Language Models and Their Agentic Abilities. The X-axis shows the release dates, and the Y-axis represents the LLM Agent Benchmark Score [43]. Bubble size indicates the number of parameters (in billions). An asterisk (*) denotes estimated parameter counts when the official release is not available. In general, every LLM undergos a two-stage training process: pre-training and fine-tuning. Pre-training involves adjusting model parameters based on the statistical properties of a large text corpus, enabling an understanding of syntax, semantics, and linguistic nuances [25]. Fine-tuning then adapts the pre-trained model to specific tasks or domains using a smaller, task-specific dataset, optimizing performance for particular applications [44]. To ensure LLMs follow human’s instructions, align with human values and exhibit desired behaviors, instruction tuning and reinforcement learning from human feedback (RLHF) have been proposed on top of fine-tuning [45]. As the size of LLMs increases, they exhibit a range of emerging capabilities, such as writing computer code, playing chess, diagnosing medical conditions, and translating languages. These capabilities often develop suddenly and dramatically at certain scales due to scaling laws, which describe how task performance can surge unexpectedly when a model reaches a particular threshold size [46]. This phenomenon is particularly observable in tasks requiring multi-step reasoning, where success probabilities compound multiplicatively, leading to rapid performance jumps [47]. However, these advancements come with “hallucination” challenges [48], such as producing false or nonsensical information that appears convincing but is inaccurate or not based on reality. These issues underline the importance of continued research and engineering to harness the benefits of LLMs while mitigating their drawbacks. ## III LLM-empowered Autonomous Agents: The Convergence of Symbolism and Connectionism This section reviews the definition of both traditional and LLM-based agents, introduces core techniques for designing and implementing LAAs, and rethinks these innovations through the lens of symbolic AI. ### III-A Autonomous Agents: Classic and LLM-empowered An autonomous agent is an artificially intelligent entity designed to achieve specific goals independently, acquiring contextual factors to perceive the environmental state and undertaking context-relevant actions [49]. These agents, equipped with reasoning, learning, and adaptability, thrive in dynamic and complex contexts. Unlike traditional software programs that follow predetermined rules, autonomous agents operate with self-governing attributes, allowing them to function under varying conditions [50]. Leveraging these capabilities, they facilitate automation by performing tasks that typically require human intervention, enhancing efficiency, and reducing operational costs across fields such as robotics, communication, financial trading, and healthcare [50]. For instance, in robotic applications, autonomous agents can navigate tasks with minimal supervision, continuously monitor their surroundings, and adapt to new situations, making them robust solutions for long-term automation [51, 52]. The foundational techniques of autonomous agent design originate from classic AI approaches, such as Probabilistic Graphical Models [53], Reinforcement Learning [54], and Multi-Agent Systems [55], which manage uncertainty and learn optimal behaviors in dynamic environments or enable agents to interact and share information efficiently. However, the advent of LAAs marks a significant evolution beyond traditional AI for both symbolic and neural sub-systems. These agents use extensive pre-training on vast textual corpora to acquire broad knowledge, performing human reasoning tasks by generating contextually appropriate text [56]. This capability not only simulates understanding and decision-making but also allows the generation of code and other communicative texts, enhancing their practical utility [57]. By integrating pre-trained language models with natural language understanding, LAAs adapt flexibly to diverse scenarios, expanding AI’s potential in autonomous operations. ### III-B Design and Implementation of LAAs Central to the design of an agent is its neural sub-system–an LLM, which functions as the core controller or coordinator. The LLM orchestrates with the agent’s symbolic sub-system and external tools, including a planning and reasoning component for task decomposition and self-reflection, memory (both short-term and long-term), and a tool-use component that allows access to external information and functionalities. - Agentic Workflow: An agentic workflow combines planning, reasoning, memory management, tool integration, and user interfaces with LLMs. Frameworks, such as LangChain [58] and LlamaIndex [59], help design these workflows. - Planner and Reasoner: Advanced techniques such as chain-of-thought and tree-of-thought prompting [60] break down tasks into sub-tasks, with self-reflection allowing agents to critique and refine outputs [61]. - Memory Management: Incorporates short-term memory for context and long-term memory using external storage, such as vector databases, enabling efficient information retrieval and enhanced reasoning [62, 63]. - Tool-Use & Natural Language Interface (NLI) Integration: Agents can access external tools, APIs, and models, deciding when and how to utilize them based on task goals [64, 65]. In addition, An effective NLI interprets user requests and communicates actions [66]. Techniques, such as ReAct and MRKL, provide structured interaction steps (thought, action, action input, observation) [67, 68]. By integrating these components, LAAs can tackle complex tasks. However, challenges like limited context windows, long-term planning, and reliable interfaces remain, necessitating ongoing research and development. ### III-C Rethink LAAs from the Perspective of Neuro-symbolic AI Neuro-symbolic AI combines the strengths of neural networks and symbolic reasoning, producing decision-making processes that are both explicit and interpretable. In autonomous agents enhanced by LLMs, the latest advancements in deep neural networks are harnessed, while task decomposition and planning are guided by symbolic AI principles — breaking complex tasks into discrete, logical steps that can be systematically analyzed and reasoned through [69]. This fusion of symbolic structures and deep neural networks creates a powerful synergy, significantly boosting the capabilities of these agents. #### Symbolic Modeling and Neural Representation Classic symbolic AI represents knowledge using abstractions and symbols, utilizing explicit symbolic modeling such as rules and relationships to perform reasoning [70]. This approach typically involves well-defined logic and structured knowledge bases, enabling systems to behave based on pre-defined rules. In contrast, LAAs, driven by language models, represent knowledge in a more distributed and implicit manner. Instead of relying on explicit symbols and rules, these agents leverage vast amounts of corpus and self-supervised pre-training on language models to infer patterns and relationships from raw text [25]. The knowledge is embedded within the weights of LLMs, allowing for more flexible and context-driven reasoning. This advantage fundamentally contrasts with the rigidity of symbolic AI, providing LAAs with the ability to handle ambiguity and generate more human-like responses [5]. #### Search-based Decision Making by Generation Given a complex goal requiring multiple steps to achieve, existing agent technologies either harness symbolic AI to systematically explore the space of potential actions or employ reinforcement learning to optimize the trajectory of these actions, efficiently partitioning complex tasks into manageable subtasks [54]. Within a LLM-empowered agent, the Chain-of-Thought (CoT) method guides LLMs to generate texts about intermediate reasoning steps, enhancing their cognitive task performance [47]. By breaking tasks into logical sequences, CoT prompts encourage LLMs to structure their reasoning systematically. This method overcomes LLM limitations at the token level by enabling coherent, step-by-step elaboration of thought processes, improving problem-solving accuracy and reliability. More recently, Tree-of-Thought (ToT) prompting extends this approach by allowing LLMs to explore multiple reasoning paths simultaneously in a tree structure [71] and the proposal of functional search over program generation, leveraging large language models (LLMs), successfully facilitates mathematical discoveries [72]. These methods enhance LLM problem-solving abilities by promoting dynamic and reflective reasoning processes, closely mirroring symbolic reasoning techniques, on top of a neural basis. #### Case-based Reasoning through In-context Learning An agent must adapt to new situations, while traditional methods rely on either re-training neural networks or deducing examples of new situations into rules for better reasoning. Within a LLM-empowered agent, few-shot in-context learning (ICL) has been proposed to utilize given examples into a prompt to generate appropriate responses that solve problems without explicit re-training the LLM [73]. This approach mimics the case-based reasoning, a fundamental concept in symbolic AI, by leveraging explicit knowledge and experiences to tackle new problems. This enhances the model’s ability to generalize from specific examples, effectively creating a neuro-symbolic mapping from presented examples to desired outcomes. #### Neuro-symbolic Integration Driven by Emergent Abilities The emergent abilities of LLMs, such as contextual understanding, sequential reasoning, goal reformulation, and task decomposition, are surged by over-parameterized architectures and extensive pre-training corpora [46]. Combining well-designed rules with the emergent abilities of LLMs enables agents to create and follow complex workflows, known as agentic workflows. By prompting large language models with instructions like “let’s think step by step”, these models analogise human’s reasoning processes and can exhibit logical and mathematical reasoning, thereby enhancing their structured reasoning skills [12, 13]. This agentic approach allows LLMs to not only process but also proactively generate structured, logical, and adaptive reasoning pathways [56], significantly improving their problem-solving and decision-making capabilities, marking a pivotal evolution in neuro-symbolic AI technologies. ## IV Discussions and Future Directions In this section, we discuss the LLM-empowered autonomous agent by comparing it with an alternative neuro-symbolic approach—the Knowledge Graph—and then highlight future directions for this technology. ### IV-A Comparative Analysis: LAAs versus KGs Previous sections have presented LAAs and KGs, both of which exemplify neuro-symbolic approaches to AI. We here compare these two methodologies to highlight the superior positioning of LAAs in the current wave of AI advancements. KGs harness the power of symbolic AI, organizing domain-specific knowledge through explicit relationships and rules. This design makes them highly effective in static environments where precision, interpretability, and predefined schemas are crucial. Their logical reasoning capabilities ensure that outputs are consistent and verifiable, which is paramount for applications needing clarity and exactitude in knowledge modeling [74]. In addition, the scalability of KGs is inherently limited by their requirements of explicit schema definitions and manual updates [75]. As the volume of data grows, the complexity of managing and querying the graph increases significantly. The maintenance of a large-scale KG demands substantial computational resources and human expertise, affecting efficiency and agility in evolving environments. On the other hand, LAAs are designed with a more dynamic and flexible approach. By combining the language comprehension and generation abilities of neural networks with the structured reasoning of symbolic AI, these agents are equipped to tackle a wide range of complex tasks. The implicit knowledge stored in neural networks enables context-sensitive responses and seamless adaptation to changing environments [76]. Additionally, LLMs efficiently compress vast corpora into a learnable network, making these agents highly scalable. Once trained, the models can be fine-tuned with additional data at a fraction of the cost and effort required for updating knowledge graphs, and can even support in-context learning without fine-tuning. As a result, LLM-powered agents can handle larger datasets with ease and even process online data to respond to real-time changes effectively. Furthermore, the advanced reasoning mechanisms employed by LAAs, such as CoT [47] and ToT [71], enable them to break down and solve complex problems effectively through analogising human reasoning steps [12]. These methods mitigate the limitations of token-level constraints in LLMs, fostering a more robust and contextually aware decision-making process. As a result, LAAs are poised to drive future innovations in AI, offering more versatile and intelligent solutions than their knowledge graph counterparts. ### IV-B Future Directions Following prior discussions, we propose several future research directions aimed at enhancing the current landscape of LAAs. <details> <summary>x4.png Details</summary>  ### Visual Description ## Screenshot: Mathematical Proof Verification via Dafny Programming Language ### Overview The image depicts a structured interaction where a user requests an agent to verify a mathematical proof using the Dafny programming language. The prompt defines a theorem to prove and instructs decomposition into propositions, formalization in Dafny, and integration with theorem provers. The response provides a step-by-step breakdown of the problem and includes Dafny code snippets for verification. ### Components/Axes #### 1. **Prompt Section** - **Title**: `Prompt: Play a role as an agent to verify the correctness of a specific mathematical proof...` - **Key Text**: - Theorem statement: *"For any even integer (n), the sum of the first (n) positive integers is an even number."* - Instructions: Decompose into propositions, generate Dafny codes, and verify propositions. #### 2. **Response Section** - **Subsection 1: Decompose the Problem** - **Proposition 1**: Define evenness (e.g., `n % 2 == 0`). - **Proposition 2**: Formula for the sum of first `n` positive integers. - **Proposition 3**: Prove the sum is even when `n` is even. - **Subsection 2: Formalize Using Dafny** - **Code Blocks**: 1. `IsEven(n)` method to check evenness. 2. `SumOfFirstNIntegers(n)` method to compute the sum. 3. `ProveSumIsEvenForEvenN(n)` method to ensure the sum is even when `n` is even. - **Subsection 3: Integration with Theorem Provers** - Use Dafny to verify logical coherence and correctness. #### 3. **Code Blocks** - **Method Definitions**: - `IsEven(n)`: Returns `true` if `n % 2 == 0`. - `SumOfFirstNIntegers(n)`: Computes sum via formula `n * (n + 1) / 2`. - `ProveSumIsEvenForEvenN(n)`: Asserts `IsEven(SumOfFirstNIntegers(n))` when `IsEven(n)` holds. ### Content Details - **Dafny Code Snippets**: - `IsEven(n)`: </details> Figure 4: An Illustrative Example of Program-Proof-of-Thoughts (P 2 oT) for Mathematical Proof Verification ### IV-C Neuro-vector-symbolic Integrative Intelligence Current agentic reasoning approaches emulate human reasoning steps explicitly [12]. For instance, when an agent receives a user’s request, it retrieves similar cases and enhances its actions through in-context learning, and for ambiguous requests, the agent prompts the LLM to clarify and rewrite the request in various forms [77]. This process involve extracting vectors for each rewritten request and performing multi-vector retrieval, improving context understanding and generative performance but increasing computational load. A vector-centric perspective, utilizing encoder-decoder architectures such as GritLM [78] that prompt generative models for instructional text encoding/vectorization, implicit neural reasoners that extend transformers with causal relation graphs for enhanced long-range reasoning [79] with latent vectors and attention matrices, and vector-symbolic architectures (VSAs) [80], could significantly address this problem. Specifically, the VSA employs high-dimensional vectors to encode and manipulate information, allowing the representation of complex structures and relationships compactly and contextually [80]. It models the cognitive and reasoning processes as algebraic operations in the vector space. Combining VSAs with LLMs could enhance cognitive capabilities, enabling precise multi-step decision-making, with applications in scientific discovery, such as solving Raven’s progressive matrices [81], thus accelerating the convergence between connectionist and symbolic paradigms through computable vectorization. ### IV-D Program-Proof-of-Thoughts Reasoning We here illustrate the proposal of Program-Proof-of-Thoughts (P 2 oT) for agentic reasoning in a rigorous manner, building on the methodologies of CoT and ToT prompting. Specifically, P 2 oT decomposes complex reasoning processes into a series of propositions organized in linear or tree structures. It leverages the programming language for program proofs, such as Dafny [82] or Lean [83], to model and verify these propositions. Future research should focus on refining proposition modeling and verification by prompting LLMs for code generation [84], improving integration with external theorem provers and assertion verifiers (e.g., Dafny and Lean), and scaling P 2 oT to handle multi-modal data for advanced reasoning. Further, automating code generation, optimizing hybrid P 2 oT/CoT/ToT models, incorporating self-verification and self-correction, and adopting P 2 oT into domain-specific applications, including logical deduction and scientific discovery can significantly advance its capabilities [72]. Figure 4 demonstrates the use of the P 2 oT framework to verify a basic mathematical proof that for any even integer $n$ , the sum of the first $n$ positive integers is an even number. By decomposing the problem into distinct, verifiable propositions and using Dafny for formal verification, this example highlights the structured and rigorous approach of P 2 oT in logical reasoning and verification. ## V Conclusions In conclusion, the synthesis of connectionist and symbolic paradigms, particularly through the rise of LLM-empowered Autonomous Agents (LAAs), marks a pivotal evolution in the field of AI, especially the neuro-symbolic AI. This paper has highlighted the historical context and the ongoing convergence of symbolic reasoning and neural network-based methods, underscoring how LAAs leverage the text-based knowledge representation and generative capabilities of LLMs to achieve logical reasoning and decision-making. By contrasting LAAs with Knowledge Graphs (KGs), we have demonstrated the unique advantages of LAAs in mimicking human-like reasoning processes, scaling effectively with large datasets, and leveraging in-context learning without extensive re-training. Promising directions such as neuro-vector-symbolic architectures and program-proof-of-thoughts (P 2 oT) prompting are on the horizon, potentially enhancing the agentic reasoning capabilities of AI further. These insights not only encapsulate the transformative potential of current AI technologies but also provide a clear trajectory for future research, fostering a deeper understanding and more advanced applications of neuro-symbolic AI. ## References - [1] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, 2015. - [2] Allen Newell and Herbert A Simon. Computer science as empirical inquiry: Symbols and search. In ACM Turing award lectures, page 1975. 2007. - [3] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012. - [4] Edward Shortliffe. Computer-based medical consultations: MYCIN, volume 2. Elsevier, 2012. - [5] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. - [6] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. - [7] Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023. - [8] Haoyi Xiong, Jiang Bian, Sijia Yang, Xiaofei Zhang, Linghe Kong, and Daqing Zhang. Natural language based context modeling and reasoning with llms: A tutorial. arXiv preprint arXiv:2309.15074, 2023. - [9] Jonathan St BT Evans. In two minds: dual-process accounts of reasoning. Trends in cognitive sciences, 7(10):454–459, 2003. - [10] Yoshua Bengio. Deep learning for system 2 processing. AAAI 2020, 2020. - [11] Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 33(2):494–514, 2021. - [12] Taylor Webb, Keith J Holyoak, and Hongjing Lu. Emergent analogical reasoning in large language models. Nature Human Behaviour, 7(9):1526–1541, 2023. - [13] James WA Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, et al. Testing theory of mind in large language models and humans. Nature Human Behaviour, pages 1–11, 2024. - [14] Artur d’Avila Garcez and Luis C Lamb. Neurosymbolic ai: The 3 rd wave. Artificial Intelligence Review, 56(11):12387–12406, 2023. - [15] Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6):386, 1958. - [16] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature, 323(6088):533–536, 1986. - [17] Allen Newell and Herbert Simon. The logic theory machine–a complex information processing system. IRE Transactions on information theory, 2(3):61–79, 1956. - [18] Bruce G Buchanan and Edward A Feigenbaum. Dendral and meta-dendral: Their applications dimension. Artificial intelligence, 11(1-2):5–24, 1978. - [19] Ashok Kumar Goel. Integration of case-based reasoning and model-based reasoning for adaptive design problem-solving. The Ohio State University, 1989. - [20] Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57, 2018. - [21] Edward A Feigenbaum et al. The art of artificial intelligence: Themes and case studies of knowledge engineering. 1977. - [22] Charles Elkan and Russell Greiner. Building large knowledge-based systems: Representation and inference in the cyc project: Db lenat and rv guha, 1993. - [23] Yoshua Bengio, Yann Lecun, and Geoffrey Hinton. Deep learning for ai. Communications of the ACM, 64(7):58–65, 2021. - [24] Artur SD’Avila Garcez, Luis C Lamb, and Dov M Gabbay. Neural-symbolic cognitive reasoning. Springer Science & Business Media, 2008. - [25] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. - [26] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. Nature, 550(7676):354–359, 2017. - [27] World Wide Web Consortium (W3C). Resource description framework (rdf) model and syntax specification, 1999. Retrieved from https://www.w3.org/TR/1999/REC-rdf-syntax-19990222/. - [28] Tim Berners-Lee, James Hendler, and Ora Lassila. The semantic web: A new form of web content that is meaningful to computers will unleash a revolution of new possibilities. In Linking the World’s Information: Essays on Tim Berners-Lee’s Invention of the World Wide Web, pages 91–103. 2023. - [29] Grigoris Antoniou and Frank Van Harmelen. A semantic web primer. MIT press, 2004. - [30] Thomas R Gruber. A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2):199–220, 1993. - [31] Stanley Kok and Pedro Domingos. Learning the structure of markov logic networks. In Proceedings of the 22nd International Conference on Machine Learning, pages 441–448, 2005. - [32] Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104(1):11–33, 2015. - [33] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2022. - [34] Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 29(12):2724–2743, 2017. - [35] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. - [36] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. - [37] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. - [38] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. - [39] Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. - [40] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113, 2023. - [41] Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024. - [42] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. - [43] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688, 2023. - [44] Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220–235, 2023. - [45] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022. - [46] Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? Advances in Neural Information Processing Systems, 36, 2024. - [47] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. - [48] Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464, 2023. - [49] Pattie Maes. Modeling adaptive autonomous agents. Artificial life, 1(1_2):135–162, 1993. - [50] Stefano V Albrecht and Peter Stone. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence, 258:66–95, 2018. - [51] Ronald C Arkin. Behavior-based robotics. MIT press, 1998. - [52] Yara Rizk, Mariette Awad, and Edward W Tunstel. Cooperative heterogeneous multi-robot systems: A survey. ACM Computing Surveys (CSUR), 52(2):1–31, 2019. - [53] Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009. - [54] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018. - [55] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024. - [56] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864, 2023. - [57] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155, 2023. - [58] Oguzhan Topsakal and Tahir Cetin Akinci. Creating large language model applications utilizing langchain: A primer on developing llm apps fast. In International Conference on Applied Engineering and Natural Sciences, volume 1, pages 1050–1056, 2023. - [59] Jerry Liu. LlamaIndex, 11 2022. - [60] Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2998–3009, 2023. - [61] Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, and Xiang Ren. Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. Advances in Neural Information Processing Systems, 36, 2024. - [62] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547, 2019. - [63] Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR, 2020. - [64] Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth-maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. A generalist agent. Transactions on Machine Learning Research. - [65] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36, 2024. - [66] Jianfeng Gao, Michel Galley, and Lihong Li. Neural approaches to conversational ai. In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 1371–1374, 2018. - [67] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. - [68] Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, et al. Mrkl systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. arXiv preprint arXiv:2205.00445, 2022. - [69] Artur d’Avila Garcez, Tarek R Besold, Luc De Raedt, Peter Földiak, Pascal Hitzler, Thomas Icard, Kai-Uwe Kühnberger, Luis C Lamb, Risto Miikkulainen, and Daniel L Silver. Neural-symbolic learning and reasoning: contributions and challenges. In 2015 AAAI Spring Symposium Series, 2015. - [70] Ronald Brachman and Hector Levesque. Knowledge representation and reasoning. Morgan Kaufmann, 2004. - [71] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024. - [72] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625(7995):468–475, 2024. - [73] Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837, 2022. - [74] Lauren Nicole DeLong, Ramon Fernández Mir, and Jacques D Fleuriot. Neurosymbolic ai for reasoning over knowledge graphs: A survey. arXiv preprint arXiv:2302.07200, 2023. - [75] Ciyuan Peng, Feng Xia, Mehdi Naseriparsa, and Francesco Osborne. Knowledge graphs: Opportunities and challenges. Artificial Intelligence Review, 56(11):13071–13102, 2023. - [76] Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080, 2021. - [77] Arian Askari, Roxana Petcu, Chuan Meng, Mohammad Aliannejadi, Amin Abolghasemi, Evangelos Kanoulas, and Suzan Verberne. Self-seeding and multi-intent self-instructing llms for generating intent-aware information-seeking dialogs. arXiv preprint arXiv:2402.11633, 2024. - [78] Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning. arXiv preprint arXiv:2402.09906, 2024. - [79] Petar Veličković and Charles Blundell. Neural algorithmic reasoning. Patterns, 2(7), 2021. - [80] Pentti Kanerva. Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors. Cognitive computation, 1:139–159, 2009. - [81] Michael Hersche, Mustafa Zeqiri, Luca Benini, Abu Sebastian, and Abbas Rahimi. A neuro-vector-symbolic architecture for solving raven’s progressive matrices. Nature Machine Intelligence, 5(4):363–375, 2023. - [82] K Rustan M Leino. Dafny: An automatic program verifier for functional correctness. In International conference on logic for programming artificial intelligence and reasoning, pages 348–370. Springer, 2010. - [83] Leonardo de Moura and Sebastian Ullrich. The lean 4 theorem prover and programming language. In Automated Deduction–CADE 28: 28th International Conference on Automated Deduction, Virtual Event, July 12–15, 2021, Proceedings 28, pages 625–635. Springer, 2021. - [84] Eric Mugnier, Emmanuel Anaya Gonzalez, Ranjit Jhala, Nadia Polikarpova, and Yuanyuan Zhou. Laurel: Generating dafny assertions using large language models. arXiv preprint arXiv:2405.16792, 2024.