# Standard Neural Computation Alone Is Insufficient for Logical Intelligence
**Authors**: Youngsung Kim
## Abstract
Neural networks, as currently designed, fall short of achieving true logical intelligence. Modern AI models rely on standard neural computation âinner-product-based transformations and nonlinear activationsâto approximate patterns from data. While effective for inductive learning, this architecture lacks the structural guarantees necessary for deductive inference and logical consistency. As a result, deep networks struggle with rule-based reasoning, structured generalization, and interpretability without extensive post-hoc modifications. This position paper argues that standard neural layers must be fundamentally rethought to integrate logical reasoning. We advocate for Logical Neural Units (LNUs) âmodular components that embed differentiable approximations of logical operations (e.g., AND, OR, NOT) directly within neural architectures. We critique existing neurosymbolic approaches, highlight the limitations of standard neural computation for logical inference, and present LNUs as a necessary paradigm shift in AI. Finally, we outline a roadmap for implementation, discussing theoretical foundations, architectural integration, and key challenges for future research.
Machine Learning, ICML
## 1 Introduction
Artificial Intelligence (AI) aims to emulate human intelligence, yet its development has oscillated between two dominant paradigms: symbolic AI and connectionism. Since the 1956 Dartmouth workshop, symbolic AI has emphasized logical rules and structured inference, whereas connectionist models, such as neural networks, have focused on learning from data (McCulloch & Pitts, 1943; Kautz, 2022). Modern large-scale neural models trained on symbolic data, such as natural language, can appear to perform deductive reasoning, but they do not robustly replicate human-like logical inference (Tian et al., 2021). The challenge lies in their reliance on statistical approximations rather than explicit rule-based deduction (Hölldobler & Kalinke, 1994; Bader & Hitzler, 2005; Seda, 2006; Serafini & Garcez, 2016).
Symbolic AI, rooted in formal logic and computational rules, enables structured reasoning and systematic problem-solving. However, it faces the symbol grounding problem âsymbols remain abstract and lack intrinsic meaning unless anchored to real-world sensory experiences (Harnad, 1990; Barsalou, 1999). In contrast, connectionist models excel at pattern recognition from raw data but struggle with systematic reasoning, logical rule generalization, and interpretability (McClelland & Rumelhart, 1986; Rumelhart & McClelland, 1986). These complementary strengths and weaknesses suggest that neither approach alone is sufficient for achieving robust, human-like reasoning.
Neuro-symbolic AI seeks to bridge the gap between connectionist learning and symbolic computation by embedding structured reasoning within neural architectures (Hilario, 1995; Hatzilygeroudis & Prentzas, 2000; dâAvila Garcez & Lamb, 2023). While symbolic theories suggest that structured formal systems provide meaning (Dennett, 1969; Newell & Simon, 1972; Fodor, 1975; Haugeland, 1985), perceptual symbol theories argue for cognition grounded in sensory-motor experience (Barsalou, 1999). This distinction extends to AI methodologies, where symbolic systems provide formal inference but lack the adaptability of neural models, while neural networks learn representations flexibly but lack explicit symbolic reasoning (dâAvila Garcez et al., 2019).
Recent neuro-symbolic approaches embed symbolic inference within deep learning architectures or integrate learned representations into reasoning engines (Wang et al., 2019; Yang et al., 2020; Manhaeve et al., 2018; dâAvila Garcez et al., 2019; Besold et al., 2017; Yang et al., 2019; Bader & Hitzler, 2005), yet aligning continuous neural representations with discrete logic remains a challenge. Large language models (LLMs) employ structured reasoning techniques like chain-of-thought prompting (Wei et al., 2022), tree-based reasoning (Yao et al., 2023), and graph-based inference (Cai et al., 2023) to enhance multi-step reasoning (Sun et al., 2024). However, their probabilistic nature causes hallucinations, contradictions, and imprecise fact retrieval, limiting their reliability for strict logical inference (Bubeck et al., 2023; Kambhampati et al., 2024). Despite approximating logical reasoning, LLMs lack formal guarantees for symbolic inference, highlighting the need for architectures that embed explicit logical structures to bridge statistical learning and rule-based reasoning (Kambhampati et al., 2024; Rae et al., 2021; Creswell et al., 2022).
This paper argues that purely arithmetic-based neural layersâthose relying on inner products and nonlinear activationsâare insufficient for robust logical intelligence. While neural networks excel at function approximation through inductive learning, they lack inherent support for deductive reasoning and symbolic consistency. Building on early AI logic units and recent neuro-symbolic advances, we propose Logical Neural Units (LNUs), which embed differentiable approximations of logical operations (e.g., fuzzy AND/OR) directly within neural layers. Unlike external symbolic engines or loosely integrated reasoning modules, LNUs seamlessly incorporate logical inference into the sub-symbolic learning process, unifying structured reasoning with modern deep learning architectures.
Structure. We position our argument within existing research by first reviewing human intelligence and learning strategies (Section 2). We then examine classical, real-valued, and fuzzy logic approaches to illustrate how logical operations extend beyond Boolean domains (Section 3). Next, we revisit universal approximation theorems, highlighting why neural networks, despite their expressive power, do not inherently enable systematic logical reasoning (Section 4). A survey of neurosymbolic frameworks (Section 5) contextualizes the role of LNUs, which we introduce and illustrate through a toy example (Section 6). Finally, we discuss open challenges and future directions for integrating logical reasoning into neural networks (Section 7).
## 2 Foundations of Logical Reasoning in Neural Systems
Human Intelligence has been studied throughout history, yet no single, universally accepted definition exists, despite extensive efforts to characterize it through various intellectual frameworks (Minsky, 1961; Newell, 1965). Since Aristotle, logic has been regarded as a fundamental tool for scientific inquiry, often referred to as the Organon (Aristotle, 4th century BCE). Rooted in symbolic reasoning (McCarthy & Hayes, 1969), logic contrasts with perceptual learning, which does not require explicit symbolic knowledge and is acquired through experience (Newell & Simon, 1956; Whitehead & Russell, 1910; Tarski, 1944). Humans develop logical reasoning abilities even without formal training, often through structured learning and knowledge transfer.
Cognitive science distinguishes two complementary domains of intelligence (Cattell, 1943, 1963): fluid intelligence, which enables rapid pattern recognition and adaptability, and crystallized intelligence, which relies on accumulated knowledge and rule-based reasoning. Together, these support both flexible learning and structured problem-solving.
To achieve intelligence, modern AI systems primarily rely on inductive learning, which generalizes patterns from data, enabling adaptation to new scenarios. Neural networks exemplify this approach by identifying statistical associations without predefined rules. Techniques such as statistical machine learning and inductive logic programming (ILP) derive general rules from examples. In contrast, deductive reasoning applies established axioms to derive conclusions with certainty. Systems such as theorem provers ensure logical soundness but lack adaptability. While deduction does not inherently involve learning, it can simulate knowledge accumulation by deriving and storing new facts (McCarthy & Hayes, 1969; Newell & Simon, 1956).
Hybrid approaches in neurosymbolic AI (Bader & Hitzler, 2005; Besold et al., 2017; dâAvila Garcez & Lamb, 2023; Marra et al., 2024; Wang et al., 2025) integrate inductive learningâs adaptability with deductive reasoningâs precision. Induction discovers patterns, while deduction ensures their logical validity (Nilsson, 1980; Russell & Norvig, 2020). This fusion enhances flexibility through data-driven learning, soundness via rule-based inference, and interpretability by leveraging explicit logical rules.
Logical reasoning unites deduction and induction, enabling structured thought akin to human System 2 reasoning (Kahneman, 2011). Logical operations such as AND, OR, and NOT are fundamental to compositional and hierarchical inference. While neural networks can approximate these operations (McCulloch & Pitts, 1943; Newell & Simon, 1956; Besold et al., 2017), they lack explicit logical representations unless specifically designed to integrate them.
## 3 Extending Logical Operators Beyond Classical Rules
Classical symbolic AI relies on propositional and predicate logic for knowledge representation (Newell & Simon, 1956; McCarthy & Hayes, 1969; Bader & Hitzler, 2005). Propositional logic operates on Boolean variables ( $0 0$ or $1$ ) and applies logical operators such as AND, OR, NOT, and implication to derive conclusions. However, strict Boolean operations often fall short when dealing with real-world scenarios that involve uncertainty, continuous-valued information, or partial truths. Such rigid structures limit expressiveness, making it difficult to integrate symbolic logic with modern neural models that inherently operate over continuous feature spaces.
To address these limitations, several extensions to classical logic have been developed. Real-valued logic generalizes Boolean truth values by extending them to the interval $[0,1]$ , allowing for continuous operators such as t-norms and t-conorms (Menger, 1942; Schweizer & Sklar, 1961, 1983; Klement et al., 2000). These operators provide smooth approximations of AND and OR functions, enabling reasoning over continuous-valued domains (Tarski, 1944; Kleene, 1952). Similarly, fuzzy logic introduces the concept of partial membership, where truth values represent degrees of belonging rather than strict binary assignments. For instance, the concept of âhotâ may be assigned a membership value of $0.7$ rather than being forced into a rigid true or false classification. Logical operations in fuzzy logic, including min, max, and product t-norms, allow for approximate AND and OR computations, making the system more robust to vagueness and imprecise data (Zadeh, 1965; Dubois & Prade, 1980; Hajek, 1998).
Another critical extension is many-valued logic, such as Gödel and Ćukasiewicz logic, which allows discrete or continuous truth values beyond $\{0,1\}$ . These frameworks support reasoning over intermediate truth levels, making them particularly useful for modeling uncertainty and graded reasoning (Rescher, 1969; Smith, 1988; Ćukasiewicz, 1920; Cignoli et al., 2013). Probabilistic logic further extends classical reasoning by assigning probabilities to logical statements, allowing AI systems to quantify uncertainty rather than relying strictly on binary logic (Pearl, 1988; Nilsson, 1986; Fagin & Halpern, 1990). This probabilistic approach has been widely adopted in Statistical Relational Artificial Intelligence (StarAI), which integrates logic with probabilistic graphical models to manage structured and uncertain domains (Getoor & Taskar, 2007; Raedt et al., 2020).
The integration of these extended logical frameworks is central to our position that purely arithmetic-based neural networks are insufficient for robust logical intelligence. Neural activations operate over continuous domains, yet classical logic assumes discrete, well-defined truths. By incorporating differentiable approximations of logical operations, such as those found in real-valued, fuzzy, many-valued, and probabilistic logic, we can bridge the gap between sub-symbolic learning and symbolic reasoning. These extended operators form the basis for Logical Neural Units (LNUs), which embed logical computation within deep neural architectures. Rather than treating logic as an external rule-based system, LNUs integrate flexible logical reasoning directly within neural computations, addressing the expressiveness limitations of traditional neural models. The following sections further develop this argument by analyzing why universal approximation alone does not guarantee logical consistency and by positioning LNUs within the broader landscape of neurosymbolic AI.
## 4 Theoretical Limits of Standard Neural Computation for Logic
The previous section examined real-valued, fuzzy, and many-valued logic as extensions that generalize classical Boolean operators for handling uncertainty and graded reasoning. While these approaches introduce flexibility, their integration into neural networks remains an approximation rather than an exact logical representation. Neural networks, by design, operate over continuous spaces and optimize for statistical associations rather than strict logical inference. This fundamental difference raises the question: Can standard neural architectures inherently support logical reasoning, or are they fundamentally constrained in their ability to model symbolic computation?
### 4.1 Universal Approximation Theorem (UAT)
The Universal Approximation Theorem (UAT) states that a feedforward neural network with a single hidden layer and a sufficient number of neurons can approximate any continuous function on a compact subset of $\mathbb{R}^{n}$ to arbitrary precision, given a suitable activation function (Hornik et al., 1989). Formally, for a compact set $K\subseteq\mathbb{R}^{n}$ and a continuous function $f:K\to\mathbb{R}$ , for any $\epsilon>0$ , there exists a neural network $\phi(x)$ such that:
$$
\textstyle\sup_{x\in K}|f(x)-\phi(x)|<\epsilon. \tag{1}
$$
While UAT ensures that neural networks can approximate any continuous function, it does not guarantee efficient learning, interpretability, or logical consistency (Cybenko, 1989; Hornik et al., 1989). Moreover, it assumes that $f$ is continuous, which is a critical limitation when applied to discrete logical operations.
### 4.2 Limitations of UAT for Logical Reasoning
Logical reasoning tasks, such as theorem proving and symbolic inference, reveal fundamental shortcomings in applying UAT to logic-based computations. The key issue lies in the nature of logical rules, which require exact, discrete outputs. While neural networks can approximate logical operators such as AND, OR, and XOR using continuous activations, these approximations are computationally inefficient and lack the precision necessary for formal inference.
Moreover, logical reasoning demands deterministic conclusions. Standard neural networks, optimized via gradient-based learning, introduce approximations that may cause outputs to oscillate near correct logical values without strictly converging. This contrasts with formal symbolic systems, where rule-based computations guarantee correctness. The problem is further exacerbated by logical inference operating over unbounded domains, such as universal quantifiers ( $\forall x$ ), whereas UAT is constrained to compact subsets of $\mathbb{R}^{n}$ . This mismatch limits neural networksâ ability to ensure global consistency, which is crucial for systematic reasoning.
Another significant limitation is interpretability. Symbolic reasoning systems produce explicit inference chains that can be traced and verified, while neural networks trained under the UAT framework rely on distributed representations that are inherently opaque. The lack of explicit reasoning steps makes it difficult to validate logical consistency, posing challenges for domains requiring explainability.
### 4.3 How UAT Constraints Affect Logical Computation
Although UAT highlights the expressive power of neural networks, it also exposes fundamental challenges in logical computation. Logical reasoning requires exact, discrete, and symbolic representations, which are not naturally supported by continuous approximations. While inductive reasoning in neural networks can approximate logical functions, it does not guarantee soundness, completeness, or consistency in a formal sense. Consequently, additional mechanismsâsuch as neurosymbolic models incorporating explicit logical rules ( $\land$ , $\lor$ , $\neg$ ) or fuzzy t-normsâare needed to bridge the gap between statistical learning and symbolic reasoning.
### 4.4 Motivation for Logic-Based Architectures
The limitations of UAT for logical reasoning highlight the need for hybrid architectures that integrate symbolic constraints within neural networks. Standard neural models struggle with discrete logical computation, global consistency, and interpretabilityâkey elements of systematic reasoning. By embedding logical structures into neural computations, hybrid models improve consistency and explainability.
Existing neurosymbolic approaches attempt to bridge neural learning and symbolic inference using hard-coded rules or differentiable approximations of logical operators (Bader & Hitzler, 2005; Wang et al., 2025). However, many depend on external reasoning modules, reducing scalability and architectural cohesion. A more integrated solution requires embedding trainable logical operators (e.g., differentiable AND, OR, and NOT) directly within neural layers, allowing for seamless interpolation between symbolic and sub-symbolic reasoning.
The next section reviews existing neurosymbolic frameworks and introduces a novel extension that enhances logic-based computation within neural architectures.
## 5 Shortcomings in Existing Neural Logic Approaches
Neurosymbolic AI combines sub-symbolic learning (e.g., neural networks) with symbolic reasoning (e.g., logic-based inference), aiming to unify the efficiency of neural methods and the interpretability of symbolic systems (Kautz, 2022; Bader & Hitzler, 2005; Sarker et al., 2021; Wang et al., 2025). Existing taxonomies (e.g., (Kautz, 2022)) categorize these methods by how tightly neural and symbolic components are intertwined, from loosely coupled pipelines to deeply integrated architectures. Here, we focus on four representative frameworksâ Neural Logic Networks (NLNs), Logical Neural Networks (LNNs), Logic Tensor Networks (LTNs), and Neural Logic Machines (NLMs) âeach of which approximates logical operations or embeds logical structures into neural models.
### 5.1 Parameterized Logic Representations: Multi-Ary Atoms
Both NLNs and LNNs seek to generalize classical AND/OR with learnable weights, making logical operations differentiable and enabling partial-truth inputs in $[0,1]$ . This reveals a shared philosophy of embedding logical structure directly into network computations, thereby preserving some interpretability at the operator level.
Neural Logic Networks (NLNs) (Yang et al., 2019) map Boolean operators (e.g., AND, OR, XOR) into weighted, many-valued logic, frequently using product -based formulations:
$$
\displaystyle\textstyle y_{\text{AND}} \displaystyle=\textstyle\prod_{i=1}^{n}\Bigl{[}1-w_{i}\,(1-x_{i})\Bigr{]}, \displaystyle\textstyle y_{\text{OR}} \displaystyle=\textstyle 1-\prod_{i=1}^{n}\Bigl{[}1-w_{i}\,x_{i}\Bigr{]}, \tag{2}
$$
where $w_{i}\in[0,1]$ are membership weights. This approach can excel at discrete algorithmic tasks (e.g., addition, sorting). However, product-based logic can lead to near-zero outputs when multiple inputs $x_{i}<1$ , potentially undermining stability in high-dimensional or uncertain settings.
Logical Neural Networks (LNNs) (Riegel et al., 2020), in contrast, implement AND/OR with weighted summations:
$$
\displaystyle\textstyle y_{\text{AND}} \displaystyle=\textstyle f\!\Bigl{(}\beta-\!\sum_{i\in I}w_{i}\,(1-x_{i})\Bigr
{)}, \displaystyle\textstyle y_{\text{OR}} \displaystyle=\textstyle f\!\Bigl{(}1-\beta+\!\sum_{i\in I}w_{i}\,x_{i}\Bigr{)}, \tag{4}
$$
where $x_{i}\in[0,1]$ are partial-truth inputs, $I$ represents the input set, $w_{i}\geq 0$ are learnable weights, $\beta\geq 0$ is a bias term, and $f(\cdot)$ is typically a clipping function or a ReLU-like activation ensuring $y\in[0,1]$ . This sum -based formulation can be more robust than a pure product, but calibrating weights/biases in deeper networks may be complex, and LNNs often focus on higher-level rule constraints rather than layer-by-layer embedding.
### 5.2 Neural Approximation of Logical Quantifiers via Perceptrons
Logic Tensor Networks (LTNs) (Serafini & Garcez, 2016) embed first-order logic into tensor-based neural systems by mapping logical predicates to continuous embeddings, then applying fuzzy t-norms for AND/OR. They unify multiple tasks (e.g., classification, clustering, relational reasoning) yet can become complex in deep architectures where large numbers of predicates and constraints must be managed.
Neural Logic Machines (NLMs) (Dong et al., 2019) extend multi-valued logic with iterative layers for multi-hop inference. They use MLP modules to approximate quantifiers ( $\forall,\exists$ ), enabling advanced relational reasoning (e.g., family-tree or grid-world tasks). Although powerful, NLMs can be challenging to scale due to their reliance on standard MLP blocks for quantifier operations, which can hinder full interpretability in large domains.
## 6 Introducing Logical Neural Units (LNUs) for Modular and Scalable Logical Reasoning
Existing neurosymbolic approaches incorporate logical operations in various ways but face several limitations. Some rely on product-based fuzzy logic for Boolean connectives, which can become numerically unstable in high-dimensional settings (Yang et al., 2019). Others inherit the constraints of perceptron-based architectures, limiting their expressivity for structured reasoning (Riegel et al., 2020). While some methods extend to first-order logic or multi-hop relational quantifiers, they often require additional modules or iterative mechanisms, impacting scalability and interpretability (Serafini & Garcez, 2016; Dong et al., 2019).
### 6.1 Core Principles for Logical Neural Units (LNUs)
These challenges motivate the development of Logical Neural Units (LNUs), a framework that embeds logical operations directly into deep networks, ensuring stable and scalable neurosymbolic reasoning. LNUs enable smooth interpolation between soft and hard logic, allowing adaptation across tasks and domains.
Scalability is a key consideration. Product-based logic can suffer from vanishing gradients, while MLP-based quantifiers often lack interpretability. LNUs must balance numerical stability with effective layering in deep networks. A hierarchical design that stacks multiple LNUs while maintaining transparent parameters offers both efficiency and clarity.
LNUs integrate seamlessly with existing architectures. Instead of replacing all neural components, LNUs selectively substitute dense layers while retaining standard MLP or attention-based modules for feature extraction. This hybrid approach maintains compatibility with established architectures such as Transformers while minimizing design overhead.
### 6.2 Illustrative LNU Architecture
LNUs generalize and unify prior differentiable logic methods within a single neural unit. Instead of standard Linear + ReLU layers, we define an LNU block as:
$$
\mathrm{LNU}(x_{1},\dots,x_{n};\theta)=\Bigl{[}T_{\land}(g_{1}(x);\theta_{1}),
T_{\lor}(g_{2}(x);\theta_{2}),\dots\Bigr{]}, \tag{6}
$$
where $T_{\land}$ and $T_{\lor}$ are learnable t-norm and t-conorm operators approximating logical AND and OR functions. The functions $g_{i}(x)$ map the input vector $x=(x_{1},\dots,x_{n})$ to real-valued logic features. The parameter set $\theta$ consists of adaptive weights that dynamically adjust logical compositions during training, allowing LNUs to learn task-specific dependencies.
### 6.3 LNU Layer Composition
Each input feature, interpreted as a truth degree in $[0,1]$ , is combined via learnable logic (e.g., softmin, softmax, or polynomial expansions) to approximate logic operations.
#### Differentiable Logic Approximation.
Classical Boolean and fuzzy logic operations, such as AND and OR, are often computed using $\min$ and $\max$ functions, as in Gödel T-norm and T-conorm (Hajek, 1998; van Krieken et al., 2022). To facilitate smooth gradient-based learning, LNUs approximate these operations using softmin and softmax:
$$
\text{softmax}(\beta\mathbf{z})_{i}\approx\max(\mathbf{z}),~{}\text{softmin}(
\beta\mathbf{z})_{j}\approx\min(\mathbf{z})~{}\text{as }\beta\to\infty.
$$
Here, each input $z_{i}$ is defined as a weighted feature:
$$
z_{i}=x_{i}\cdot w_{i},\quad\text{where }x_{i}\in\mathbf{x},~{}w_{i}\in\mathbf
{w},~{}i=1,\dots,d. \tag{7}
$$
The parameter $\beta\geq 0$ controls the sharpness of the approximation, with larger values making the function more Boolean-like. The softmin function is efficiently computed via: $\text{softmin}(\beta\mathbf{z})=\text{softmax}(-\beta\mathbf{z}).$
#### Locally Gated Logical Consistency.
To ensure logical consistency, each featureâs contribution is weighted via learned importance factors. Given an input vector $\mathbf{z}=(z_{1},\dots,z_{d})$ , the differentiable logic functions are:
$$
\displaystyle\texttt{soft-OR}(\mathbf{z}) \displaystyle=\textstyle\sum_{i=1}^{d}\bigl{[}\text{softmax}(\beta\mathbf{z})
\bigr{]}_{i}\cdot z_{i}, \displaystyle\texttt{soft-AND}(\mathbf{z}) \displaystyle=\textstyle\sum_{i=1}^{d}\bigl{[}\text{softmin}(\beta\mathbf{z})
\bigr{]}_{i}\cdot z_{i}. \tag{8}
$$
For high-dimensional inputs, normalizing outputs by $\sqrt{d}$ mitigates activation scaling issues and ensures numerical stability. Additionally, the negation operator is defined as: $\texttt{soft-NOT}(x)=1-x,~{}~{}~{}\text{or}~{}~{}~{}1-\sigma(W_{\mathrm{not}}x),$ where $\sigma$ is the sigmoid function and $W_{\mathrm{not}}$ is a learnable parameter.
#### LNU Layer Architecture.
LNUs replace standard dense layers by embedding differentiable logical operators instead of matrix multiplication and activation functions. Given an input matrix $\mathbf{X}\in\mathbb{R}^{n\times d}$ and trainable parameter matrices $\mathbf{W}_{\text{AND}},\mathbf{W}_{\text{OR}}\in\mathbb{R}^{d\times o}$ , we apply broadcasted element-wise multiplication to implement the Locally Gated Logical Consistency mechanism (details in Appendix B).
#### Deep Logical Networks with LNU Stacking.
Stacking multiple LNU layers enables a deep hierarchical architecture where each layer progressively refines logical inferences at increasing levels of abstraction:
$$
\mathbf{X}^{(\ell)}=\mathrm{LNU}(\mathbf{X}^{(\ell-1)}),\quad\ell=1,\dots,L. \tag{10}
$$
To enhance gradient flow and iterative logical refinement, we incorporate logical residual connections:
$$
\mathbf{X}_{\mathrm{res}}=\texttt{soft-IMPLY}(\mathbf{X},F(\mathbf{X})), \tag{11}
$$
where $\texttt{soft-IMPLY}(A,B)=\texttt{soft-OR}(1-A,B)$ , ensuring stable logical propagation across layers. Additionally, other logical connectives can be incorporated using soft-AND and soft-OR, enabling more expressive reasoning within deep architectures.
<details>
<summary>extracted/6177073/decision_boundaries.png Details</summary>
### Visual Description
## Heatmap Grid: Comparison of Hard Logic, NLU, and MLP Decision Boundaries
### Overview
A 2x3 grid of heatmaps comparing decision boundaries for:
1. Hard Logic gates (OR, AND)
2. NLU soft gates (Soft-OR, Soft-AND)
3. Dense+ReLU neural networks with different biases
Each heatmap uses x1 (0-1) and x2 (0-1) axes with color-coded decision intensity.
### Components/Axes
- **X-axis (x1)**: 0.0 to 1.0 in 0.2 increments
- **Y-axis (x2)**: 0.0 to 1.0 in 0.2 increments
- **Color scales**:
- Rightmost colorbar: 0.000 to 1.000 (decision intensity)
- Individual heatmap scales vary (see Detailed Analysis)
- **Legend**: Positioned right of all heatmaps, showing gradient from purple (low) to yellow (high)
### Detailed Analysis
1. **Hard Logic OR** (Top-left)
- Binary output: Purple (0) below diagonal, yellow (1) above
- Sharp transition at x1=0.5, x2=0.5
- Scale: 0-1 (binary)
2. **Hard Logic AND** (Top-right)
- Binary output: Purple (0) except top-right quadrant
- Yellow square at x1>0.5, x2>0.5
- Scale: 0-1 (binary)
3. **NLU Soft-OR** (Middle-left)
- Gradual transition from purple to yellow
- Beta=100 creates near-binary behavior
- Scale: 0.222-0.889
4. **NLU Soft-AND** (Middle-right)
- Complex boundary with multiple transition zones
- Beta=100 creates sharp but non-binary outputs
- Scale: 0.000-0.667
5. **Dense+ReLU (Bias=0.0)** (Bottom-left)
- Diagonal gradient from purple to yellow
- w=(0.5,0.5) creates linear decision boundary
- Scale: 0.000-1.000
6. **Dense+ReLU (Bias=-0.5)** (Bottom-right)
- Uniform purple output across all inputs
- w=(0.5,0.5) with negative bias suppresses activation
- Scale: 0.000-0.111
### Key Observations
1. **Binary vs Continuous Outputs**:
- Hard Logic gates show perfect binary separation
- NLU models introduce soft transitions
- ReLU networks demonstrate bias-dependent activation
2. **Boundary Complexity**:
- OR/AND gates have simple geometric boundaries
- Soft-OR/Soft-AND show more complex decision surfaces
- ReLU networks exhibit linear/non-linear separability
3. **Bias Impact**:
- Positive bias (0.0) enables full activation range
- Negative bias (-0.5) severely limits output variation
### Interpretation
This visualization demonstrates:
1. **Model Complexity Tradeoffs**:
- Hard Logic provides perfect separation but lacks nuance
- NLU models balance binary decisions with probabilistic outputs
- ReLU networks show how parameter choices affect decision boundaries
2. **Activation Function Behavior**:
- ReLU's bias parameter critically influences output distribution
- Negative bias can effectively "disable" certain regions of input space
3. **Practical Implications**:
- For strict binary classification, Hard Logic gates are optimal
- For probabilistic reasoning, NLU soft gates offer better generalization
- ReLU networks require careful parameter tuning for effective decision boundaries
The data suggests that model selection should consider both the nature of the decision boundary required and the tradeoff between binary certainty and probabilistic reasoning capabilities.
</details>
Figure 1: Comparison of logical function approximations across different models. The first row represents Hard Logic, where AND and OR operations are applied to binarized inputs ( $x_{i}>0.5,\forall i$ ). The second row shows LNUs with fixed weights ( $w_{i}=0.5,\forall i$ ) and a sharp gating parameter $\beta=100$ . The third row presents an inner-product unit with ReLU activation, using the same fixed weights ( $w_{i}=0.5,\forall i$ ) and different biases ( $0.0\text{ or }-0.5$ ) over the input domain $(x_{1},x_{2})\in[0,1]$ . LNUs closely approximate discrete logical functions, resembling the behavior of hard logic in the first row. Since inputs range over $[0,1]$ , LNU outputs remain continuous, allowing for an adjustable decision boundary around the corresponding hard logic outputs. In contrast, the inner-product unit, which relies on summation-based arithmetic, does not exhibit logical function behavior. Additional results for varying values of $\beta$ in LNUs are provided in Appendix A.
### 6.4 Evaluating LNU Interpretability and Decision Boundaries
Logical operations such as AND and OR follow explicit compositional rules, making their decisions inherently transparent. Neural Logic Units (NLUs) enhance interpretability by structuring decisions through learned logical operations. To assess this interpretability, we analyze decision boundaries and compare them against standard inner-product-based layers.
Visualizing decision boundaries is a common approach to understanding model behavior. Here, we constrain input data to the range $[0,1]$ and fix the weights for both NLUs and dense (inner-product) layers, ensuring an even distribution that aligns with hard AND/OR logic. This setup allows us to evaluate whether decision-making can be explained at the level of a single computation unitâs output. Specifically, we examine how NLUs approximate hard logic functions such as AND and OR.
As shown in Figure 1, NLUs exhibit decision boundaries that closely align with logical structures, whereas inner-product-based layers rely on learned decision surfaces that are harder to interpret. The key advantage of NLUs is their structured decision-making, which inherently follows predefined logical rules, unlike traditional dense layers that require additional interpretation.
When irrelevant features are present, NLUs can filter them out through a gating mechanism, similar to self-attention in a learnable weight setting. This interpretability arises from two key properties: (1) the gating process highlights dominant features, enhancing transparency in the decision process, and (2) predefined logical operations provide structured reasoning, reducing the need for post-hoc interpretation. These properties contribute to NLUsâ ability to offer more explainable and structured decision-making compared to conventional neural models.
### 6.5 Toy Example: Learning a Logical Function
To evaluate the potential of logic-friendly modules, we conduct experiments on a toy logical task. This example showcases the advantages of the LNU Layer in capturing discrete logical patterns, particularly in low-data regimes.
#### Toy Logical Functions.
We test a simple logical expression to assess the modelâs ability to perform logical inference.
Task: Given the logical function: $f(x_{1},x_{2},x_{3})=(x_{1}\lor x_{2})\land\lnot(x_{3})$ , the goal is to learn this function from a subset of possible $(x_{1},x_{2},x_{3})$ combinations. We generate random input values in the range $[0,1]$ , using 20 samples for training and 200 samples for testing. In data generation, inputs are binarized such that $x_{i}>0.5$ is mapped to 1, and otherwise 0. This ensures that the target outputs adhere strictly to logical evaluations in the set $\{0,1\}$ .
For comparison, we benchmark Logicron (LNU-based model) against a standard perceptron. Both models consist of a single hidden unit, where Logicron uses an LNU layer and Perceptrons use an inner-product-based linear unit with non-linear activation functions (Sigmoid, ReLU, and GeLU). The output layer in both models consists of a dense layer with a linear transformation followed by a sigmoid activation. These models are referred to as Logicron and Perceptron in Figure 2.
Observation: In the toy logical function task, Logicron generalizes more reliably during testing, likely due to its built-in logical gating mechanisms. In contrast, Perceptrons exhibit lower test accuracy, even though all models achieve 100% training accuracy. To ensure a fair comparison, we maintain a similar number of parameters across models: (Perceptron with 97, Logicron with 90, Logicrons +negation unit with 110). At the final epoch (30), the test accuracy (mean $\pm$ standard deviation) for each model is as follows, MLP-Sigmoid: 77.4 $\$ $\pm$ 1.6, MLP-ReLU : 80.2 $\$ $\pm$ 1.6, MLP-GeLU : 80.3 $\$ $\pm$ 1.4, Logicron (softmax) : 84.7 $\$ $\pm$ 1.7, Logicron neg (softmax): 84.7 $\$ $\pm$ 1.8.
<details>
<summary>extracted/6177073/Figure_1_.png Details</summary>
### Visual Description
## Line Chart: Model Comparison (Avg. Acc. ± Std over 5 Runs)
### Overview
The chart compares the test accuracy of four neural network models over 25 training epochs. Each model's performance is represented by a line with a shaded confidence interval (± standard deviation). The y-axis shows test accuracy (%), and the x-axis shows training epochs.
### Components/Axes
- **X-axis (Epoch)**: Ranges from 0 to 25 in increments of 5.
- **Y-axis (Test Accuracy)**: Ranges from 60% to 90% in increments of 5%.
- **Legend**: Located at bottom-right, with four entries:
- **Black line**: Perceptrons (Sigmoid)
- **Gray line**: Perceptrons (ReLU)
- **Light gray line**: Perceptrons (GeLU)
- **Green line**: Logicrons (+Negation)
### Detailed Analysis
1. **Logicrons (+Negation)** (Green):
- Starts at 60% accuracy at epoch 0.
- Sharp upward trend, reaching 78% by epoch 10.
- Plateaus at ~85% from epoch 15 onward.
- Confidence interval widens initially (epochs 0-5) but stabilizes after epoch 10.
2. **Perceptrons (Sigmoid)** (Black):
- Begins at 65% accuracy at epoch 0.
- Gradual increase to 78% by epoch 15.
- Slows to a plateau at ~78% by epoch 20.
- Confidence interval remains narrow throughout.
3. **Perceptrons (ReLU)** (Gray):
- Starts at 70% accuracy at epoch 0.
- Steady rise to 80% by epoch 15.
- Flattens at ~80% by epoch 20.
- Confidence interval shows moderate variability.
4. **Perceptrons (GeLU)** (Light Gray):
- Begins at 68% accuracy at epoch 0.
- Increases to 79% by epoch 15.
- Stabilizes at ~79% by epoch 20.
- Confidence interval shows slight widening after epoch 10.
### Key Observations
- **Logicrons (+Negation)** outperform all Perceptron variants after epoch 10, achieving the highest final accuracy (~85%).
- **Perceptrons (Sigmoid)** starts strongest but underperforms by epoch 20.
- **ReLU** and **GeLU** variants show similar trajectories, with ReLU slightly ahead.
- All models exhibit diminishing returns after epoch 15-20.
### Interpretation
The data demonstrates that **Logicrons with negation** achieve superior long-term performance despite slower initial progress. Perceptron variants with ReLU/GeLU activation functions show moderate performance, while Sigmoid activation leads to early gains but poor scalability. The shaded confidence intervals highlight that Logicrons exhibit higher variability in early training, suggesting sensitivity to hyperparameters or initialization. The plateauing trends across all models indicate potential convergence limits, emphasizing the importance of architectural choices over mere activation function selection.
</details>
Figure 2: Test accuracy (mean $\pm$ standard deviation, with shading) across epochs for different models, including Perceptrons with various activation functions and Logicrons (LNU layer + dense output layer). Logicron with an additional negation unit is marked as â+Negationâ.
## 7 Discussion and Future Directions
### 7.1 Position Summary
We have demonstrated that while standard inner-product-based neural networks excel at sub-symbolic pattern recognition, they inherently lack the capacity for structured logical reasoning. As a result, achieving consistent logical inference often requires large datasets, extensive parameter tuning, and auxiliary learning strategies.
Modern AI models have followed a scaling law, which describes how system performance improves as model and data sizes increase, often following a power-law relationship (Bahri et al., 2024; Kaplan et al., 2020). This trend is grounded in the universal approximation theorem, which states that sufficiently large neural networks can approximate a broad range of functions. Consequently, current inductive learning frameworks rely on ever-increasing model size and data volume to capture complex relationships across different domains. However, while scaling enhances function approximation, it does not inherently enable structured, rule-based reasoning.
This paper argues for integrating logic-friendly modules into large-scale neural models to enable implicit logical reasoning. By explicitly incorporating deductive reasoning, these modules enhance logical inference without requiring fundamental architectural changes. We extend existing neurosymbolic frameworks by introducing Logical Neural Units (LNUs), which embed approximate logical operations (e.g., AND, OR, NOT) directly into neural architectures using weighted real-valued logic. These units bridge the gap between sub-symbolic efficiency and symbolic reasoning, providing a scalable alternative to brute-force model expansion. By unifying neural and logical reasoning, we propose a shift away from continuous scaling, which consumes vast computational resources and energy, towards a more structured and efficient approach to AI development.
### 7.2 Critical Limitations and Challenges
LNUs provide a promising bridge between sub-symbolic learning and symbolic reasoning, yet several challenges must be addressed to enhance their effectiveness. One major limitation is their current focus on propositional logic, which restricts their ability to handle more expressive reasoning frameworks. LNUs do not yet incorporate first-order logic constructs such as quantifiers ( $\forall,\exists$ ) or domain iteration. Extending LNUs to inductive reasoning frameworks, such as Inductive Logic Programming (ILP) (Muggleton, 1996) or differentiable logical networks (Sen et al., 2022), could significantly improve their ability to capture structured relationships while maintaining scalability.
Another challenge is ensuring logical soundness and completeness. Traditional symbolic reasoning systems guarantee that every derived statement is valid (soundness) and that all logically entailed statements can be derived (completeness). Neural networks, however, rely on approximate optimization and do not inherently provide formal correctness guarantees. While LNUs introduce logical biases into learning, future research should explore differentiable proof-checking mechanisms to enforce logical validity in neural architectures.
The symbol grounding problem remains a fundamental issue in neurosymbolic AI. Learned representations in neural networks often fail to meaningfully align with symbolic concepts, leading to inconsistencies between learned models and human-interpretable logical reasoning. The challenge of mapping abstract symbols to real-world meaning (Harnad, 1990) necessitates structured constraints or hybrid approaches that integrate both perceptual and logical representations (dâAvila Garcez & Lamb, 2023; Barsalou, 1999). Ensuring that real-valued embeddings preserve logical consistency is critical for effective reasoning.
Computational complexity is another challenge. Logical gating mechanisms, such as weighted t-norms with elementwise products, introduce additional overhead, especially in deep architectures. Efficient approximations, such as polynomial expansions as alternatives to softmax-based formulations, can help mitigate these costs while preserving interpretability.
### 7.3 Alternative Views
Despite the advantages of modular neural logic units (e.g., LNUs), alternative perspectives question their necessity. One argument is that sufficiently large MLPs can approximate logical functions without explicitly embedding logical operations. Given sufficient data and computational resources, neural networks can learn complex nonlinear functions that approximate logical inference. However, this approach is inefficient, requiring significantly larger datasets and lacking interpretability. The absence of interpretability makes debugging and improving specific weaknesses in large models challenging. Addressing localized errors may require retraining the entire model, potentially disrupting global reasoning outcomes.
Another perspective suggests that classical fuzzy logic and Gödelâs $\min/\max$ operators already provide logical approximations without modifying network architectures. While these methods capture partial truth values, they lack trainable feature weighting and smooth differentiability, making seamless integration with neural models difficult. Modular neural logic units extend these ideas by embedding logical operations directly into neural computations, allowing for greater adaptability.
Chain-of-thought prompting in transformers has demonstrated that structured reasoning can emerge through large-scale training and carefully designed prompts. However, this remains a heuristic rather than an architectural guarantee of logical consistency and often requires additional optimization during model training. Modular neural logic units complement such approaches by embedding structured logical operations within the network itself, reducing reliance on post-hoc prompt engineering.
Logical constraint learning, as exemplified by SATNet (Wang et al., 2019) and neural theorem proving (RocktÀschel & Riedel, 2017), explicitly encodes logical rules within neural models. While effective, these methods often require manually specified rules, limiting scalability. Modular neural logic units aim to internalize logical inference within neural networks, balancing adaptability with the ability to perform formal reasoning without external supervision.
### 7.4 Future Directions
Several research directions remain for advancing modular neural logic units (e.g., LNUs). One key area is extending LNUs beyond propositional logic by incorporating quantifiers and domain-specific rule induction. This would enable higher-order reasoning, bridging the gap between propositional and first-order logic. Another important step is integrating LNUs with sequence-based models, such as Transformers and RNNs, to enhance structured reasoning in natural language processing and decision-making tasks.
Empirical benchmarking in real-world applications, such as legal reasoning, medical diagnostics, and scientific discovery, is essential to demonstrate the practical effectiveness of LNUs. Additionally, rigorous evaluation frameworks for logical consistency are needed. Establishing soundness and completeness benchmarks for neuro-symbolic reasoning will ensure that LNU-based models adhere to formal logical principles while maintaining the flexibility of neural learning.
### 7.5 Concluding Remarks
This paper has argued that the arithmetic foundations of standard neural networksâinner products and non-linear activationsâare insufficient for achieving robust logical intelligence. While large-scale models can approximate logical functions, they fail to ensure logical consistency, interpretability, or rule-based deduction without auxiliary reasoning mechanisms. By seamlessly integrating neural logic units into the core of neural networks, large-scale models may mitigate the challenges associated with scaling while improving their reasoning capabilities.
## Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
## References
- Aristotle (4th century BCE) Aristotle. Organon, 4th century BCE.
- Bader & Hitzler (2005) Bader, S. and Hitzler, P. Dimensions of neural-symbolic integration â a structured survey. In Artemov, S., Barringer, H., Garcez, A. S. d., Lamb, L. C., and Woods, J. (eds.), We Will Show Them: Essays in Honour of Dov Gabbay, volume 1, pp. 167â194. Kingâs College Publications, 2005.
- Bahri et al. (2024) Bahri, Y., Dyer, E., Kaplan, J., Lee, J., and Sharma, U. Explaining neural scaling laws. Proceedings of the National Academy of Sciences, 121(27):e2311878121, 2024. doi: 10.1073/pnas.2311878121. URL https://www.pnas.org/doi/abs/10.1073/pnas.2311878121.
- Barsalou (1999) Barsalou, L. W. Perceptual symbol systems. Behavioral and Brain Sciences, 22(4):577â660, 1999.
- Besold et al. (2017) Besold, T. R., dâAvila Garcez, A., Bader, S., Bowman, H., Domingos, P., Hitzler, P., KĂŒhnberger, K.-U., Lamb, L. C., Lowd, D., Lima, P. M. V., de Penning, L., Pinkas, G., Poon, H., and Zaverucha, G. Neural-symbolic learning and reasoning: A survey and interpretation. arXiv preprint arXiv:1711.03902, 2017. URL https://arxiv.org/abs/1711.03902.
- Bubeck et al. (2023) Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., and Zhang, Y. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
- Cai et al. (2023) Cai, T., Liu, H., Liu, Z., Abbeel, P., Song, D., and Mordatch, I. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning (ICML), 2023.
- Cattell (1943) Cattell, R. B. The measurement of adult intelligence. Psychological Bulletin, 40(3):153â193, 1943.
- Cattell (1963) Cattell, R. B. Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54:1â22, 1963.
- Cignoli et al. (2013) Cignoli, R., Dubuc, E., and Mundici, D. Algebraic Foundations of Many-Valued Reasoning. Springer Science & Business Media, 2013.
- Creswell et al. (2022) Creswell, A., Shanahan, M., and Higgins, I. Selection-inference: Exploiting large language models for interpretable logical reasoning. In International Conference on Learning Representations (ICLR), 2022. URL https://openreview.net/forum?id=3Pf3Wg6o-A4.
- Cybenko (1989) Cybenko, G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2(4):303â314, 1989.
- dâAvila Garcez et al. (2019) dâAvila Garcez, A., Gori, M., Lamb, L. C., Serafini, L., Spranger, M., and Tran, S. N. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv preprint arXiv:1905.06088, 2019. URL https://arxiv.org/abs/1905.06088.
- Dennett (1969) Dennett, D. C. The nature of images and the introspective trap. In Content and Consciousness, pp. 132â146. Routledge, London, 1969.
- Dong et al. (2019) Dong, H., Mao, J., Lin, T., Wang, C., Li, L., and Zhou, D. Neural logic machines. arXiv preprint arXiv:1904.11694, 2019.
- Dubois & Prade (1980) Dubois, D. and Prade, H. Fuzzy sets and systems: Theory and applications. Academic Press, 1980.
- dâAvila Garcez & Lamb (2023) dâAvila Garcez, A. and Lamb, L. C. Neurosymbolic ai: The 3rd wave. Artificial Intelligence Review, 56(11):12387â12406, nov 2023.
- Fagin & Halpern (1990) Fagin, R. and Halpern, J. Y. Uncertainty, belief, and probability. Computational Intelligence, 7(1):160â173, 1990.
- Fodor (1975) Fodor, J. A. The Language of Thought. Harvard University Press, Cambridge, MA, 1975.
- Getoor & Taskar (2007) Getoor, L. and Taskar, B. (eds.). Introduction to Statistical Relational Learning. Adaptive Computation and Machine Learning. The MIT Press, 2007. ISBN 978-0-262-07286-1. URL https://doi.org/10.7551/mitpress/7432.001.0001.
- Hajek (1998) Hajek, P. Metamathematics of Fuzzy Logic. Springer Science & Business Media, 1998.
- Harnad (1990) Harnad, S. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3):335â346, 1990.
- Hatzilygeroudis & Prentzas (2000) Hatzilygeroudis, I. and Prentzas, J. Neurules: Integrating symbolic rules and neurocomputing. In Fotiades, D. and Nikolopoulos, S. (eds.), Advances in Informatics, pp. 122â133. World Scientific, Singapore, 2000.
- Haugeland (1985) Haugeland, J. Artificial Intelligence: The Very Idea. MIT Press, Cambridge, MA, 1985.
- Hilario (1995) Hilario, M. An overview of strategies for neurosymbolic integration. In Proceedings of the Workshop on Connectionist-Symbolic Integration: From Unified to Hybrid Approaches, Montreal, Canada, 1995.
- Hornik et al. (1989) Hornik, K., Stinchcombe, M., and White, H. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359â366, 1989.
- Hölldobler & Kalinke (1994) Hölldobler, S. and Kalinke, Y. Towards a massively parallel computational model for logic programming. In Proceedings of the ECAIâ94 Workshop on Combining Symbolic and Connectionist Processing, pp. 68â77, 1994.
- Kahneman (2011) Kahneman, D. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.
- Kambhampati et al. (2024) Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L. P., and B Murthy, A. Position: LLMs canât plan, but can help planning in LLM-modulo frameworks. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 22895â22907. PMLR, 21â27 Jul 2024.
- Kaplan et al. (2020) Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
- Kautz (2022) Kautz, H. The third ai summer: Aaai robert s. engelmore memorial lecture. AI Magazine, 43(1):105â125, 2022.
- Kleene (1952) Kleene, S. C. Introduction to Metamathematics. North-Holland Publishing Company, 1952.
- Klement et al. (2000) Klement, E. P., Mesiar, R., and Pap, E. Triangular Norms. Springer, 2000.
- Manhaeve et al. (2018) Manhaeve, R., DumanÄiÄ, S., Kimmig, A., Demeester, T., and Raedt, L. D. Deepproblog: Neural probabilistic logic programming. In Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pp. 3749â3759, 2018.
- Marra et al. (2024) Marra, G., DumanÄiÄ, S., Manhaeve, R., and De Raedt, L. From statistical relational to neurosymbolic artificial intelligence: A survey. Artificial Intelligence, 328:104062, 2024.
- McCarthy & Hayes (1969) McCarthy, J. and Hayes, P. J. Some philosophical problems from the standpoint of artificial intelligence. Machine Intelligence, 4:463â502, 1969.
- McClelland & Rumelhart (1986) McClelland, J. L. and Rumelhart, D. E. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models. MIT Press, Cambridge, MA, 1986.
- McCulloch & Pitts (1943) McCulloch, W. S. and Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5:115â133, 1943.
- Menger (1942) Menger, K. Statistical metrics. Proceedings of the National Academy of Sciences of the United States of America, 28(12):535â537, 1942.
- Minsky (1961) Minsky, M. Steps toward artificial intelligence. Proceedings of the IRE, 49(1):8â30, 1961. doi: 10.1109/JRPROC.1961.287775. URL https://ieeexplore.ieee.org/document/4066245.
- Muggleton (1996) Muggleton, S. Learning from positive data. In Proceedings of the 6th International Workshop on Inductive Logic Programming (ILP), pp. 358â376, 1996.
- Newell (1965) Newell, A. Limitations of the current stock of ideas about problem solving. In Kent, A. and Taulbee, O. (eds.), Proceedings of the Conference on Electronic Information Handling, pp. 195â208, New York, 1965. Spartan Books.
- Newell & Simon (1956) Newell, A. and Simon, H. The logic theory machineâa complex information processing system. IRE Transactions on Information Theory, 2(3):61â79, 1956.
- Newell & Simon (1972) Newell, A. and Simon, H. A. Human Problem Solving. Prentice-Hall, Englewood Cliffs, NJ, 1972.
- Nilsson (1980) Nilsson, N. J. Principles of Artificial Intelligence. Tioga Publishing, 1980.
- Nilsson (1986) Nilsson, N. J. Probabilistic logic. Artificial Intelligence, 28(1):71â87, 1986.
- Pearl (1988) Pearl, J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
- Rae et al. (2021) Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., Rutherford, E., Hennigan, T., Menick, J., Cassirer, A., Powell, R., van den Driessche, G., Hendricks, L. A., Rauh, M., Huang, P.-S., Glaese, A., Welbl, J., Dathathri, S., Huang, S., Uesato, J., Mellor, J., Higgins, I., Creswell, A., McAleese, N., Wu, A., Elsen, E., Jayakumar, S., Buchatskaya, E., Budden, D., Sutherland, E., Simonyan, K., Paganini, M., Sifre, L., Martens, L., Li, X. L., Kuncoro, A., Nematzadeh, A., Gribovskaya, E., Donato, D., Lazaridou, A., Mensch, A., Lespiau, J.-B., Tsimpoukelli, M., Grigorev, N., Fritz, D., Sottiaux, T., Pajarskas, M., Pohlen, T., Gong, Z., Toyama, D., de Masson dâAutume, C., Li, Y., Terzi, T., Mikulik, V., Babuschkin, I., Clark, A., de Las Casas, D., Guy, A., Jones, C., Bradbury, J., Johnson, M., Hechtman, B., Weidinger, L., Gabriel, I., Isaac, W., Lockhart, E., Osindero, S., Rimell, L., Dyer, C., Vinyals, O., Ayoub, K., Stanway, J., Bennett, L., Hassabis, D., Kavukcuoglu, K., and Irving, G. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021. URL https://arxiv.org/abs/2112.11446.
- Raedt et al. (2020) Raedt, L. D., DumanÄiÄ, S., Manhaeve, R., and Marra, G. From statistical relational to neuro-symbolic artificial intelligence. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20), pp. 4943â4950, 2020.
- Rescher (1969) Rescher, N. Many-Valued Logic. McGraw-Hill, New York, 1969.
- Riegel et al. (2020) Riegel, R., Gray, A., and Laird, J. E. Logical neural networks. arXiv preprint arXiv:2006.13155, 2020.
- RocktÀschel & Riedel (2017) RocktÀschel, T. and Riedel, S. End-to-end differentiable proving. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
- Rumelhart & McClelland (1986) Rumelhart, D. E. and McClelland, J. L. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. MIT Press, Cambridge, MA, 1986.
- Russell & Norvig (2020) Russell, S. and Norvig, P. Artificial Intelligence: A Modern Approach. Pearson, 4th edition, 2020.
- Sarker et al. (2021) Sarker, M. K., Zhou, L., Eberhart, A., and Hitzler, P. Neuro-symbolic artificial intelligence. AI Commun., 34(3):197â209, January 2021. ISSN 0921-7126.
- Schweizer & Sklar (1961) Schweizer, B. and Sklar, A. Associative functions and statistical triangle inequalities. Publications of the Mathematical Society of Japan, 8:169â186, 1961.
- Schweizer & Sklar (1983) Schweizer, B. and Sklar, A. Probabilistic Metric Spaces. Elsevier, 1983.
- Seda (2006) Seda, A. K. On the integration of connectionist and logic-based systems. Electronic Notes in Theoretical Computer Science, 161:109â130, 2006. ISSN 1571-0661. doi: https://doi.org/10.1016/j.entcs.2006.04.028. URL https://www.sciencedirect.com/science/article/pii/S1571066106004038. Proceedings of the Third Irish Conference on the Mathematical Foundations of Computer Science and Information Technology (MFCSIT 2004).
- Sen et al. (2022) Sen, P., de Carvalho, B. W. S. R., Riegel, R., and Gray, A. Neuro-symbolic inductive logic programming with logical neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8212â8219, 2022.
- Serafini & Garcez (2016) Serafini, L. and Garcez, A. d. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. In Conference of the Italian Association for Artificial Intelligence, pp. 334â348. Springer, 2016.
- Smith (1988) Smith, K. C. Multiple-valued logic: A tutorial and appreciation. Computer, 21(4):17â27, 1988.
- Sun et al. (2024) Sun, H., Xu, W., Liu, W., Luan, J., Wang, B., Shang, S., Wen, J.-R., and Yan, R. DetermLR: Augmenting LLM-based logical reasoning from indeterminacy to determinacy. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9828â9862, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.531. URL https://aclanthology.org/2024.acl-long.531/.
- Tarski (1944) Tarski, A. The semantic conception of truth. Philosophy and Phenomenological Research, 4:341â376, 1944.
- Tian et al. (2021) Tian, J., Li, Y., Chen, W., Xiao, L., He, H., and Jin, Y. Diagnosing the first-order logical reasoning ability through LogicNLI. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3738â3747, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.303. URL https://aclanthology.org/2021.emnlp-main.303/.
- van Krieken et al. (2022) van Krieken, E., Acar, E., and van Harmelen, F. Analyzing differentiable fuzzy logic operators. Artificial Intelligence, 302:103602, 2022. ISSN 0004-3702. doi: https://doi.org/10.1016/j.artint.2021.103602. URL https://www.sciencedirect.com/science/article/pii/S0004370221001533.
- Wang et al. (2019) Wang, P.-W., Donti, P., Wilder, B., and Kolter, Z. Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In Proceedings of the International Conference on Machine Learning (ICML), pp. 6545â6554. PMLR, 2019.
- Wang et al. (2025) Wang, W., Yang, Y., and Wu, F. Towards data-and knowledge-driven ai: A survey on neuro-symbolic computing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(2):878â899, 2025.
- Wei et al. (2022) Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv.org/abs/2201.11903.
- Whitehead & Russell (1910) Whitehead, A. N. and Russell, B. Principia Mathematica. Cambridge University Press, 1910.
- Yang et al. (2019) Yang, F., Song, L., and Zhang, M. Learning algorithms via neural logic networks. arXiv preprint arXiv:1904.01554, 2019.
- Yang et al. (2020) Yang, Z., Ishay, A., and Lee, J. Neurasp: Embracing neural networks into answer set programming. In Bessiere, C. (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 1755â1762. International Joint Conferences on Artificial Intelligence Organization, 7 2020.
- Yao et al. (2023) Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
- Zadeh (1965) Zadeh, L. A. Fuzzy sets. Information and Control, 8(3):338â353, 1965.
- Ćukasiewicz (1920) Ćukasiewicz, J. O logice trĂłjwartoĆciowej. Ruch Filozoficzny, 5:169â171, 1920.
## Appendix A Observations on Decision Boundary
Expanding on Figure 1, we analyze the effect of different gating parameters $\beta\in\{1,10,100\}$ on LNU decision boundaries. As $\beta$ increases, the boundaries transition from smooth to sharper logical separations, demonstrating a controllable interpolation between continuous and discrete decision-making within a single neural computation unit, such as AND/OR.
<details>
<summary>extracted/6177073/decision_boundaries_more.png Details</summary>
### Visual Description
## Heatmap Grid: Comparison of Hard Logic, NLU, and MLP Decision Boundaries
### Overview
The image presents a 2x5 grid of heatmaps comparing decision boundaries for logical operations (OR, AND) across three model types: Hard Logic, NLU (Neural Logic Unit), and MLP (Dense + ReLU). Each heatmap visualizes decision intensity (0.000-1.000) across input space (xâ, xâ â [0,1]). The rightmost color bar serves as a shared legend for all heatmaps.
### Components/Axes
- **Axes**:
- Horizontal: xâ (0.0 to 1.0)
- Vertical: xâ (0.0 to 1.0)
- **Legends**:
- Right-aligned color bar labeled "Decision Intensity" (0.000-1.000)
- Color gradient: Purple (low) â Yellow (high)
- **Heatmap Titles**:
- Top row: "Hard Logic OR" (left), "Hard Logic AND" (right)
- Middle rows: NLU Soft-OR/Soft-AND with ÎČ=1, ÎČ=10, ÎČ=100
- Bottom row: Dense+ReLU with Bias=0.0 and Bias=-0.5
### Detailed Analysis
1. **Hard Logic OR (Top Left)**:
- Sharp boundary at xâ=0.5, xâ=0.5
- High intensity (yellow) in top-right quadrant (xââ„0.5 OR xââ„0.5)
- Low intensity (purple) in bottom-left quadrant
2. **Hard Logic AND (Top Right)**:
- Sharp corner boundary at xâ=0.5, xâ=0.5
- High intensity only in top-right quadrant (xââ„0.5 AND xââ„0.5)
3. **NLU Soft-OR/Soft-AND (ÎČ=1 to 100)**:
- **ÎČ=1**:
- Gradual transition from purple to yellow
- OR: Diagonal boundary at 45°
- AND: Curved boundary forming a quarter-circle
- **ÎČ=10**:
- Sharper boundaries approaching hard logic
- OR: Near-vertical/horizontal steps
- AND: Near-corner steps
- **ÎČ=100**:
- Almost identical to hard logic boundaries
- Minimal gradient regions
4. **Dense+ReLU (Bias Variations)**:
- **Bias=0.0**:
- Diagonal boundary from bottom-left to top-right
- High intensity above the diagonal
- **Bias=-0.5**:
- Steeper diagonal boundary
- High intensity concentrated in top-right corner
### Key Observations
1. **NLU Sensitivity to ÎČ**:
- Lower ÎČ (1): Smooth, continuous transitions
- Higher ÎČ (100): Approaches hard logic behavior
- AND operation shows more complex boundary evolution than OR
2. **Bias Impact in Dense+ReLU**:
- Negative bias (-0.5) shifts decision boundary upward
- Creates larger high-intensity region in top-right quadrant
3. **Model Comparison**:
- Hard Logic: Perfect binary boundaries
- NLU: Intermediate between hard logic and MLP
- MLP: Linear/non-linear boundaries depending on bias
### Interpretation
The heatmaps demonstrate how different architectures handle logical operations:
- **NLU** provides a tunable middle ground between hard logic and MLP through ÎČ parameterization
- **Dense+ReLU** shows linear decision boundaries that can be adjusted via bias
- The AND operation requires more complex boundary shapes than OR across all models
- Higher ÎČ values in NLU make it increasingly similar to hard logic, suggesting potential for exact logical computation with sufficient parameterization
The comparison highlights tradeoffs between model complexity and interpretability, with hard logic offering perfect boundaries at the cost of flexibility, while NLU and MLP provide adjustable approximations.
</details>
Figure 3: Effect of gating parameter $\beta\in\{1,10,100\}$ on LNU decision boundaries with fixed weights ( $w_{i}=0.5,\forall i$ ). Increasing $\beta$ sharpens the boundary, approaching hard logic.
## Appendix B Matrix operations
Given an input matrix $\mathbf{X}\in\mathbb{R}^{n\times d}$ and trainable parameter matrices $\mathbf{W}_{\text{AND}},\mathbf{W}_{\text{OR}}\in\mathbb{R}^{d\times o}$ , we apply broadcasted element-wise multiplication to implement the Locally Gated Logical Consistency mechanism
$$
\mathbf{Z}_{\text{AND}}=\mathbf{X}\odot\mathbf{W}_{\text{AND}},\quad\mathbf{Z}
_{\text{OR}}=\mathbf{X}\odot\mathbf{W}_{\text{OR}}.
$$
Here, broadcasting ensures:
$$
z_{ijk}=x_{ij}\cdot w_{jk},\quad x_{ij}\text{ is shared across }k,\quad w_{jk}
\text{ is shared across }i.
$$
Logical transformations are then applied to produce the final outputs:
$$
\mathbf{X}_{\mathrm{AND}}=\texttt{soft-AND}(\mathbf{Z}_{\mathrm{AND}}),~{}~{}~
{}\mathbf{X}_{\mathrm{OR}}=\texttt{soft-OR}(\mathbf{Z}_{\mathrm{OR}}).
$$
The final LNU layer output is formed by concatenating these logical transformations:
$$
\mathbf{X}_{\mathrm{out}}=\bigl{[}\mathbf{X}_{\mathrm{AND}}\;\|\;\mathbf{X}_{
\mathrm{OR}}\bigr{]}.
$$
Optionally, soft-NOT transformations can be included to enhance representational flexibility.