# Towards Unified Neurosymbolic Reasoning on Knowledge Graphs
> Qika Lin, Kai He, and Mengling Feng are with the Saw Swee Hock School of Public Health, National University of Singapore, 117549, Singapore.
Fangzhi Xu and Jun Liu are with the School of Computer Science and Technology, Xiβan Jiaotong University, Xiβan, Shaanxi 710049, China.
Hao Lu is with the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.
Rui Mao and Erik Cambria are with the College of Computing and Data Science, Nanyang Technological University, 639798, Singapore.
Abstract
Knowledge Graph (KG) reasoning has received significant attention in the fields of artificial intelligence and knowledge engineering, owing to its ability to autonomously deduce new knowledge and consequently enhance the availability and precision of downstream applications. However, current methods predominantly concentrate on a single form of neural or symbolic reasoning, failing to effectively integrate the inherent strengths of both approaches. Furthermore, the current prevalent methods primarily focus on addressing a single reasoning scenario, presenting limitations in meeting the diverse demands of real-world reasoning tasks. Unifying the neural and symbolic methods, as well as diverse reasoning scenarios in one model is challenging as there is a natural representation gap between symbolic rules and neural networks, and diverse scenarios exhibit distinct knowledge structures and specific reasoning objectives. To address these issues, we propose a unified neurosymbolic reasoning framework, namely Tunsr, for KG reasoning. Tunsr first introduces a consistent structure of reasoning graph that starts from the query entity and constantly expands subsequent nodes by iteratively searching posterior neighbors. Based on it, a forward logic message-passing mechanism is proposed to update both the propositional representations and attentions, as well as first-order logic (FOL) representations and attentions of each node. In this way, Tunsr conducts the transformation of merging multiple rules by merging possible relations at each step. Finally, the FARI algorithm is proposed to induce FOL rules by constantly performing attention calculations over the reasoning graph. Extensive experimental results on 19 datasets of four reasoning scenarios (transductive, inductive, interpolation, and extrapolation) demonstrate the effectiveness of Tunsr.
Index Terms: Neurosymbolic AI, Knowledge graph reasoning, Propositional reasoning, First-order logic, Unified model
1 Introduction
As a fundamental and significant topic in the domains of knowledge engineering and artificial intelligence (AI), knowledge graphs (KGs) have been spotlighted in many real-world applications [1], such as question answering [2, 3], recommendation systems [4, 5], relation extraction [6, 7] and text generation [8, 9]. Thanks to their structured manner of knowledge storage, KGs can effectively capture and represent rich semantic associations between real entities using multi-relational graphical structures. Factual knowledge is often stored in KGs using the fact triple as the fundamental unit, represented in the form of (subject, relation, object), such as (Barack Obama, bornIn, Hawaii) in Figure 1. However, most common KGs, such as Freebase [10] and Wikidata [11], are incomplete due to the limitations of current human resources and technical conditions. Furthermore, incomplete KGs can degrade the accuracy of downstream intelligent applications or produce completely wrong answers. Therefore, inferring missing facts from the observed ones is of great significance for downstream KG applications, which is called link prediction that is one form of KG reasoning [12, 13].
The task of KG reasoning is to infer or predict new facts using existing knowledge. For instance, in Figure 1, KG reasoning involves predicting the validity of the target missing triple (Barack Obama, nationalityOf, U.S.A.) based on other available triples. Using two distinct paradigms, connectionism, and symbolicism, which serve as the foundation for implementing AI systems [14, 15], existing methods can be categorized into neural, symbolic, and neurosymbolic models.
Neural methods, drawing inspiration from the connectionism of AI, typically employ neural networks to learn entity and relation representations. Subsequently, a customized scoring function, such as translation-based distance or semantic matching strategy, is utilized for model optimization and query reasoning, which is illustrated in the top part of Figure 1. However, such an approach lacks transparency and interpretability [16, 17]. On the other hand, symbolic methods draw inspiration from the idea of symbolicism in AI. As shown in the bottom part of Figure 1, they first learn logic rules and then apply these rules, based on known facts to deduce new knowledge. In this way, symbolic methods offer natural interpretability due to the incorporation of logical rules. However, owing to the limited modeling capacity given by discrete representation and reasoning strategies of logical rules, these methods often fall short in terms of reasoning performance [18].
<details>
<summary>extracted/6596839/fig/ns.png Details</summary>
### Visual Description
## Diagram: Knowledge Graph and Reasoning Methods
### Overview
The image presents a diagram illustrating two methods of reasoning (Neural and Symbolic) to infer the nationality of Barack Obama from a knowledge graph. The knowledge graph contains entities like people, cities, and countries, connected by relations such as "bornIn," "marriedTo," and "locatedInCountry." The diagram shows how both neural and symbolic reasoning approaches can be used to derive the relationship "nationalityOf" between Barack Obama and the U.S.A.
### Components/Axes
**Left Panel: Knowledge Graph**
* Entities: Represented as ovals containing names (e.g., Barack Obama, Michelle Obama, Chicago, U.S.A., Honolulu, Hawaii, Ann Dunham, Malia Obama, Harvard University).
* People are in light blue ovals.
* Locations are in light yellow ovals.
* Harvard University is in a light orange oval.
* Relations: Represented as arrows connecting entities, labeled with the type of relationship (e.g., bornIn, marriedTo, fatherOf, motherOf, placeIn, locatedInCountry, hasCity, graduateFrom).
* Green arrows: bornIn, locatedInCountry
* Blue arrows: locatedInCountry, hasCity
* Purple arrows: bornIn, marriedTo, placeIn
* Black arrows: fatherOf, motherOf, graduateFrom
* Question Mark: A red dashed arrow with a question mark indicates the relationship to be inferred (nationalityOf).
* Numbers:
* 1: Near the arrow from Hawaii to U.S.A.
* 2: Near the arrow from Honolulu to Hawaii.
* 3: Near the arrow from Michelle Obama to Chicago.
**Right Panel: Reasoning Methods**
* **(1) Neural Reasoning:**
* KGE (Knowledge Graph Embedding): A box containing a network diagram.
* Relation Embedding: A matrix of green and light green squares.
* Entity Embedding: A matrix of blue and light blue squares.
* Score Function: A box containing a network diagram.
* **(2) Symbolic Reasoning:**
* Rule Set: A box containing three logical rules (gamma1, gamma2, gamma3) for inferring nationality.
* Ξ³1: 0.89 βX, Y, Z bornIn(X, Y) β§ locatedInCountry(Y, Z) β nationalityof(X, Z)
* Ξ³2: 0.65 βX, Y1, Y2, Z bornIn(X, Y1) β§ hasCity(Y1, Y2) β§ locatedInCountry(Y2, Z) β nationalityof(X, Z)
* Ξ³3: 0.54 βX, Y1, Y2, Z marriedTo(X, Y1) β§ bornIn(Y1, Y2) β§ placeIn(Y2, Z) β nationalityof(X, Z)
* Inferred Relationship: Barack Obama nationalityOf U.S.A.
### Detailed Analysis or Content Details
**Knowledge Graph Details:**
* Michelle Obama bornIn Chicago.
* Michelle Obama marriedTo Barack Obama.
* Malia Obama fatherOf Barack Obama.
* Ann Dunham motherOf Barack Obama.
* Barack Obama graduateFrom Harvard University.
* Barack Obama bornIn Honolulu.
* Barack Obama locatedInCountry Hawaii.
* Chicago placeIn U.S.A.
* Honolulu hasCity Hawaii.
* Honolulu locatedInCountry U.S.A.
* Hawaii locatedInCountry U.S.A.
**Reasoning Methods Details:**
* Neural Reasoning: Uses Knowledge Graph Embeddings (KGE) to learn representations of entities and relations, then applies a score function to predict new relationships.
* Symbolic Reasoning: Applies a set of logical rules to infer new relationships based on existing ones. The rules use probabilities (0.89, 0.65, 0.54) to indicate the confidence in each rule.
### Key Observations
* The knowledge graph contains information about Barack Obama's family, education, and places of birth and residence.
* Both neural and symbolic reasoning methods are presented as ways to infer Barack Obama's nationality.
* The symbolic reasoning rules use a combination of "bornIn," "locatedInCountry," "hasCity," "marriedTo," and "placeIn" relationships to infer "nationalityOf."
* The rules have different confidence levels (0.89, 0.65, 0.54), suggesting varying degrees of reliability in the inference.
### Interpretation
The diagram illustrates how knowledge graphs and reasoning methods can be used to infer relationships between entities. The example focuses on inferring Barack Obama's nationality using both neural and symbolic approaches. The symbolic reasoning rules demonstrate how different combinations of relationships can be used to derive the same conclusion, with varying degrees of confidence. The diagram highlights the potential of knowledge graphs and reasoning methods for automated knowledge discovery and inference. The red dashed arrow with the question mark indicates the goal of the reasoning process, which is to determine the relationship between Barack Obama and the U.S.A. in terms of nationality. The diagram effectively shows two different approaches to achieve this goal.
</details>
Figure 1: Illustration of neural and symbolic methods for KG reasoning. Neural methods learn entity and relation embeddings to calculate the validity of the specific fact. Symbolic methods perform logic deduction using known facts on learned or given rules (like $\gamma_{1}$ , $\gamma_{2}$ and $\gamma_{3}$ ) for inference.
TABLE I: Classical studies for KG reasoning. PL and FOL denote the propositional and FOL reasoning, respectively. SKG T, SKG I, TKG I, and TKG E represent transductive, inductive, interpolation, and extrapolation reasoning. β $\checkmark$ β means the utilized reasoning manners (neural and logic) or their vanilla application scenarios.
| Model | Neural | Logic | Reasoning Scenarios | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- |
| PL | FOL | SKG T | SKG I | TKG I | TKG E | | |
| TransE [19] | β | | | β | | | |
| AMIE [20] | | | β | β | | | |
| Neural LP [21] | β | | β | β | | | |
| TAPR [22] | β | β | | β | | | |
| RLogic [23] | β | | β | β | | | |
| LatentLogic [24] | β | | β | β | | | |
| PSRL [25] | β | β | | β | | | |
| ConGLR [26] | β | | β | | β | | |
| TeAST [27] | β | | | | | β | |
| TLogic [28] | | | β | | | | β |
| TR-Rules [29] | | | β | | | | β |
| TECHS [30] | β | β | β | | | | β |
| Tunsr | β | β | β | β | β | β | β |
To leverage the strengths of both neural and symbolic methods while mitigating their respective drawbacks, there has been a growing interest in integrating them to realize neurosymbolic systems [31]. Several approaches such as Neural LP [21], DRUM [32], RNNLogic [33], and RLogic [23] have emerged to address the learning and reasoning of rules by incorporating neural networks into the whole process. Despite achieving some successes, there remains a notable absence of a cohesive modeling approach that integrates both propositional and first-order logic (FOL) reasoning. Propositional reasoning on KGs, generally known as multi-hop reasoning [34], is dependent on entities and predicts answers through specific reasoning paths, which demonstrates strong modeling capabilities by providing diverse reasoning patterns for complex scenarios [35, 36]. On the other hand, FOL reasoning utilizes learned FOL rules to infer information from the entire KG by variable grounding, ultimately scoring candidates by aggregating all possible FOL rules. FOL reasoning is entity-independent and exhibits good transferability. Unfortunately, as shown in Table I, mainstream methods have failed to effectively combine these two reasoning approaches within a single framework, resulting in suboptimal models.
Moreover, as time progresses and society undergoes continuous development, a wealth of new knowledge consistently emerges. Consequently, simple reasoning on static KGs (SKGs), i.e., transductive reasoning, can no longer meet the needs of practical applications. Recently, there has been a gradual shift in the research communityβs focus toward inductive reasoning with emerging entities on SKGs, as well as interpolation and extrapolation reasoning on temporal KGs (TKGs) [37] that introduce time information to facts. The latest research, which predominantly concentrated on individual scenarios, proved insufficient in providing a comprehensive approach to address various reasoning scenarios simultaneously. This limitation significantly hampers the modelβs generalization ability and its practical applicability. To sum up, by comparing the state-of-the-art recent studies on KG reasoning in Table I, it is observed that none of them has a comprehensive unification across various KG reasoning tasks, either in terms of methodology or application perspective.
The challenges in this domain can be categorized into three main aspects: (1) There is an inherent disparity between the discrete nature of logic rules and the continuous nature of neural networks, which presents a natural representation gap to be bridged. Thus, implementing differentiable logical rule learning and reasoning is not directly achievable. (2) It is intractable to solve the transformation and integration problems for propositional and FOL rules, as they have different semantic representation structures and reasoning mechanisms. (3) Diverse scenarios on SKGs or TKGs exhibit distinct knowledge structures and specific reasoning objectives. Consequently, a model tailored for one scenario may encounter difficulties when applied to another. For example, each fact on SKGs is in a triple form while that of TKGs is quadruple. Conventional embedding methods for transductive reasoning fail to address inductive reasoning as they do not learn embeddings of emerging entities in the training phase. Similarly, methods employed for interpolation reasoning cannot be directly applied to extrapolation reasoning, as extrapolation involves predicting facts with future timestamps that are not present in the training set.
To address the above challenges, we propose a unified neurosymbolic reasoning framework (named Tunsr) for KG reasoning. Firstly, to realize the unified reasoning on different scenarios, we introduce a consistent structure of reasoning graph. It starts from the query entity and constantly expands subsequent nodes (entities for SKGs and entity-time pairs for TKGs) by iteratively searching posterior neighbors. Upon this, we can seamlessly integrate diverse reasoning scenarios within a unified computational framework, while also implementing different types of propositional and FOL rule-based reasoning over it. Secondly, to combine neural and symbolic reasoning, we propose a forward logic message-passing mechanism. For each node in the reasoning graph, Tunsr learns an entity-dependent propositional representation and attention using the preceding counterparts. Besides, it utilizes a gated recurrent unit (GRU) [38] to integrate the current relation and preceding FOL representations as the edgesβ representations, following which the entity-independent FOL representation and attention are calculated by message aggregation. In this process, the information and confidence of the preceding nodes in the reasoning graph are passed to the subsequent nodes and realize the unified neurosymbolic calculation. Finally, with the reasoning graph and learned attention weights, a novel Forward Attentive Rule Induction (FARI) algorithm is proposed to induce different types of FOL rules. FARI gradually appends rule bodies by searching over the reasoning graph and viewing the FOL attentions as rule confidences. It is noted that our reasoning form for link prediction is data-driven to learn rules and utilizes grounding to calculate the fact probabilities, while classic Datalog [39] and ASP (Answer Set Programming) reasoners [40, 41] usually employ declarative logic programming to conduct precise and deterministic deductive reasoning on a set of rules and facts.
In summary, the contribution can be summarized as threefold:
$\bullet$ Combining the advantages of connectionism and symbolicism of AI, we propose a unified neurosymbolic framework for KG reasoning from both perspectives of methodology and reasoning scenarios. To the best of our knowledge, this is the first attempt to do such a study.
$\bullet$ A forward logic message-passing mechanism is proposed to update both the propositional representations and attentions, as well as FOL representations and attentions of each node in the expanding reasoning graph. Meanwhile, a novel FARI algorithm is introduced to induce FOL rules using learned attentions.
$\bullet$ Extensive experiments are carried out on the current mainstream KG reasoning scenarios, including transductive, inductive, interpolation, and extrapolation reasoning. The results demonstrate the effectiveness of our Tunsr and verify its interpretability.
This study is an extension of our model TECHS [30] published at the ACL 2023 conference. Compared with it, Tunsr has been enhanced in three significant ways: (1) From the theoretical perspective, although propositional and FOL reasoning are integrated in TECHS for extrapolation reasoning on TKGs, these two reasoning types are entangled together in the forward process, which limits the interpretability of the model. However, the newly proposed Tunsr framework presents a distinct separation of propositional and FOL reasoning in each reasoning step. Finally, they are combined for the reasoning results. This transformation enhances the interpretability of the model from both propositional and FOL rulesβ perspectives. (2) For the perspective of FOL rule modeling, not limited to modeling temporal extrapolation Horn rules in TECHS, the connected and closed Horn rules, and the temporal interpolation Horn rules are also included in the Tunsr framework. (3) From the application perspective, the TECHS model is customized for the extrapolation reasoning on TKGs. Based on the further formalization of the reasoning graph and FOL rules, we can utilize the Tunsr model for current mainstream reasoning scenarios of KGs, including transductive, inductive, interpolation, and extrapolation reasoning. The experimental results demonstrate that our Tunsr model performs well in all those scenarios.
2 Preliminaries
2.1 KGs, Variants, and Reasoning Scenarios
Generally, a static KG (SKG) can be represented as $\mathcal{G}=\{\mathcal{E},\mathcal{R},\mathcal{F}\}$ , where $\mathcal{E}$ and $\mathcal{R}$ denote the set of entities and relations, respectively. $\mathcal{F}β\mathcal{E}Γ\mathcal{R}Γ\mathcal{E}$ is the fact set. Each fact is a triple, such as ( $s$ , $r$ , $o$ ), where $s$ , $r$ , and $o$ denote the head entity, relation, and tail entity, respectively. By introducing time information in the knowledge, a TKG can be represented as $\mathcal{G}=\{\mathcal{E},\mathcal{R},\mathcal{T},\mathcal{F}\}$ , where $\mathcal{T}$ denotes the set of time representations (timestamps or time intervals). $\mathcal{F}β\mathcal{E}Γ\mathcal{R}Γ\mathcal{E}Γ\mathcal{T}$ is the fact set. Each fact is a quadruple, such as $(s,r,o,t)$ where $s,oβ\mathcal{E}$ , $rβ\mathcal{R}$ , and $tβ\mathcal{T}$ .
For these two types of KGs, there are mainly the following reasoning types (query for predicting the head entity can be converted to the tail entity prediction by adding reverse relations), which is illustrated in Figure 2:
$\bullet$ Transductive Reasoning on SKGs: Given a background SKG $\mathcal{G}=\{\mathcal{E},\mathcal{R},\mathcal{F}\}$ , the task is to predict the missing entity for the query $(\tilde{s},\tilde{r},?)$ . The true answer $\tilde{o}β\mathcal{E}$ , and $\tilde{s}β\mathcal{E}$ , $\tilde{r}β\mathcal{R}$ , $(\tilde{s},\tilde{r},\tilde{o})β\mathcal{F}$ .
$\bullet$ Inductive Reasoning on SKGs: It indicates that there are new entities appearing in the testing stage, which were not present during the training phase. Formally, the training graph can be expressed as $\mathcal{G}_{t}=\{\mathcal{E}_{t},\mathcal{R},\mathcal{F}_{t}\}$ . The inductive graph $\mathcal{G}_{i}=\{\mathcal{E}_{i},\mathcal{R},\mathcal{F}_{i}\}$ shares the same relation set with $\mathcal{G}_{t}$ . However, their entity sets are disjoint, i.e., $\mathcal{E}_{t}\cap\mathcal{E}_{i}=\varnothing$ . A model needs to predict the missing entity $\tilde{o}$ for the query $(\tilde{s},\tilde{r},?)$ , where $\tilde{s}β\mathcal{E}_{i}$ , $\tilde{o}β\mathcal{E}_{i}$ , $\tilde{r}β\mathcal{R}$ , and $(\tilde{s},\tilde{r},\tilde{o})β\mathcal{F}_{i}$ .
$\bullet$ Interpolation Reasoning on TKGs: For a query $(\tilde{s},\tilde{r},?,\tilde{t})$ in the testing phase based on a training TKG $\mathcal{G}_{t}=\{\mathcal{E}_{t},\mathcal{R}_{t},\mathcal{T}_{t},\mathcal{F}_%
{t}\}$ , a model needs to predict the answer entity $\tilde{o}$ using the facts in the TKG. It denotes that $min(\mathcal{T}_{t})β€slant\tilde{t}β€slant max(\mathcal{T}_{t})$ , where $min$ and $max$ denote the functions to obtain the minimum and maximum timestamp within the set, respectively. Also, the query satisfies $\tilde{s}β\mathcal{E}_{t}$ , $\tilde{o}β\mathcal{E}_{t}$ , $\tilde{r}β\mathcal{R}_{t}$ , and $(\tilde{s},\tilde{r},\tilde{o},\tilde{t})β\mathcal{F}_{t}$ .
$\bullet$ Extrapolation Reasoning on TKGs: It is similar to the interpolation reasoning that predicts the target entity $\tilde{o}$ for a query $(\tilde{s},\tilde{r},?,\tilde{t})$ in the testing phase, based on a training TKG $\mathcal{G}_{t}=\{\mathcal{E}_{t},\mathcal{R}_{t},\mathcal{T}_{t},\mathcal{F}_%
{t}\}$ . Differently, this task is to predict future facts, which means the prediction utilizes the facts that occur earlier than $\tilde{t}$ in TKGs, i.e., $\tilde{t}>max(\mathcal{T}_{t})$ .
<details>
<summary>extracted/6596839/fig/transductive.png Details</summary>
### Visual Description
## Diagram: Obama Family and Location
### Overview
The image is a diagram illustrating relationships and locations associated with Barack Obama and Michelle Obama. It uses nodes representing people, places, and entities, connected by arrows indicating relationships such as "marriedTo," "liveIn," "bornIn," and "nationality." The diagram also includes images of the Obamas, the US flag, and seals of Honolulu and Hawaii.
### Components/Axes
* **Nodes:**
* Michelle Obama (top-left, purple border) - Image of Michelle Obama with label "Michelle Obama"
* Barack Obama (bottom-left, green border) - Image of Barack Obama with label "Barack Obama"
* U.S.A. (center, blue border) - Image of the US flag with label "U.S.A."
* Honolulu (top-right, brown border) - Seal of the City and County of Honolulu with label "Honolulu"
* Hawaii (bottom-right, gold border) - Seal of the State of Hawaii with label "Hawaii"
* Several unlabeled gray nodes are scattered throughout the diagram, connected by gray lines.
* **Relationships (Arrows):**
* "marriedTo" (black arrow) - Connects Barack Obama to Michelle Obama
* "liveIn" (black arrow) - Connects Michelle Obama to U.S.A.
* "bornIn" (black arrow) - Connects Barack Obama to Hawaii
* "nationality Of ?" (red dashed arrow) - Connects Barack Obama to U.S.A.
* "locatedIn Country" (black arrow) - Connects U.S.A. to Honolulu
* "locatedIn Country" (black arrow) - Connects U.S.A. to Hawaii
* "hasCity" (black arrow) - Connects Hawaii to Honolulu
### Detailed Analysis or ### Content Details
* **Michelle Obama:** The node contains a photograph of Michelle Obama. The label "Michelle Obama" is below the image.
* **Barack Obama:** The node contains a photograph of Barack Obama. The label "Barack Obama" is below the image.
* **U.S.A.:** The node contains an image of the American flag. The label "U.S.A." is below the flag.
* **Honolulu:** The node contains the seal of the City and County of Honolulu. The label "Honolulu" is below the seal. The seal includes the text "CITY AND COUNTY OF HONOLULU".
* **Hawaii:** The node contains the seal of the State of Hawaii. The label "Hawaii" is below the seal. The seal includes the text "STATE OF HAWAII 1959", "UA MAU KE EA O KA AINA I KA PONO".
* **Relationships:**
* Michelle Obama is "marriedTo" Barack Obama.
* Michelle Obama "liveIn" the U.S.A.
* Barack Obama was "bornIn" Hawaii.
* Barack Obama's "nationality Of ?" is the U.S.A.
* The U.S.A. "locatedIn Country" Honolulu.
* The U.S.A. "locatedIn Country" Hawaii.
* Hawaii "hasCity" Honolulu.
### Key Observations
* The diagram focuses on the relationships between Barack Obama, Michelle Obama, the U.S.A., Hawaii, and Honolulu.
* The "nationality Of ?" relationship is marked with a red dashed arrow, suggesting a question or uncertainty.
* The gray nodes and lines in the background seem to represent a broader network or context, but their specific meaning is not defined.
### Interpretation
The diagram visually represents the familial and geographical connections of the Obamas. It highlights that Michelle Obama is married to Barack Obama and lives in the U.S.A. It also shows that Barack Obama was born in Hawaii and his nationality is American. The question mark associated with Barack Obama's nationality suggests a potential point of inquiry or discussion, possibly related to his birthplace and citizenship. The inclusion of Honolulu and Hawaii emphasizes the geographical ties of the Obama family to these locations within the U.S.A. The gray nodes and lines in the background could represent other individuals, locations, or entities related to the Obamas, but without labels, their specific roles are unclear.
</details>
(a) Transductive reasoning on SKGs.
<details>
<summary>extracted/6596839/fig/inductive.png Details</summary>
### Visual Description
## Knowledge Graph: Christopher Nolan and Syncopy Inc.
### Overview
The image is a knowledge graph illustrating relationships between Christopher Nolan, Emma Thomas, Syncopy Inc., London, and the United Kingdom. The graph uses nodes (circles containing images and labels) and edges (arrows labeled with relationships) to represent these entities and their connections.
### Components/Axes
* **Nodes:**
* Christopher Nolan (top-left, teal border): A person, represented by a photograph.
* Emma Thomas (bottom-left, maroon border): A person, represented by a photograph.
* Syncopy Inc. (top-center, gold border): A company, represented by its logo. The logo shows the word "SYNCOPY" in blue neon-style lettering on a black background.
* London (top-right, green border): A city, represented by a photograph of the Houses of Parliament and Big Ben.
* United Kingdom (bottom-right, purple border): A country, represented by its flag.
* Gray nodes: Representing other entities not explicitly labeled.
* **Edges (Relationships):**
* `cofounderOf`: Connects Christopher Nolan and Emma Thomas to Syncopy Inc. (black arrows).
* `marriedTo`: Connects Christopher Nolan to Emma Thomas (black arrow).
* `hasofficeIn`: Connects Syncopy Inc. to London (black arrow).
* `capitalOf`: Connects London to the United Kingdom (black arrow).
* `nationalityOf`: Connects Christopher Nolan and Emma Thomas to the United Kingdom (red dashed arrows). The relationship is marked with a question mark, indicating uncertainty.
* `bornIn`: Connects Emma Thomas to the United Kingdom (black arrow).
* **Labels:** Each node has a label indicating the entity it represents (e.g., "Christopher Nolan," "Syncopy Inc.").
### Detailed Analysis or Content Details
* **Christopher Nolan:**
* Relationship: `cofounderOf` Syncopy Inc.
* Relationship: `marriedTo` Emma Thomas
* Relationship: `nationalityOf` United Kingdom (uncertain, dashed red arrow)
* **Emma Thomas:**
* Relationship: `cofounderOf` Syncopy Inc.
* Relationship: `nationalityOf` United Kingdom (uncertain, dashed red arrow)
* Relationship: `bornIn` United Kingdom
* **Syncopy Inc.:**
* Relationship: `hasofficeIn` London
* **London:**
* Relationship: `capitalOf` United Kingdom
### Key Observations
* The graph highlights the professional and personal relationships of Christopher Nolan and Emma Thomas, particularly their co-founding of Syncopy Inc.
* The connection between Christopher Nolan and the United Kingdom is marked as uncertain regarding nationality.
* Emma Thomas is explicitly stated to be born in the United Kingdom.
* Syncopy Inc. has an office in London, which is the capital of the United Kingdom.
### Interpretation
The knowledge graph visually represents the interconnectedness of people, organizations, and locations. It suggests that Christopher Nolan and Emma Thomas are key figures in Syncopy Inc., which has ties to London. The uncertainty around Christopher Nolan's nationality, contrasted with Emma Thomas's confirmed birth in the United Kingdom, raises questions about their specific connections to the country. The graph serves as a starting point for exploring these relationships in more detail.
</details>
(b) Inductive reasoning on SKGs using training data in 2.
<details>
<summary>extracted/6596839/fig/interpolation.png Details</summary>
### Visual Description
## Political Interaction Diagram: Temporal Diplomatic Relations
### Overview
The image is a diagram illustrating political interactions between various countries and individuals (Barack Obama and Angela Merkel) over three time points: t<sub>i-2</sub>, t<sub>i-1</sub>, and t<sub>i</sub>. The diagram uses nodes representing countries or individuals, connected by lines indicating diplomatic actions or relationships.
### Components/Axes
* **Nodes:** Represent countries (China, Russia, Singapore, Pakistan, South Korea, North Korea) and individuals (Barack Obama, Angela Merkel). Each node contains a picture representing the entity and a label with the entity's name.
* China (Blue border)
* Russia (Purple border)
* Singapore (Peach border)
* Pakistan (Light Blue border)
* South Korea (Peach border)
* North Korea (Green border)
* Barack Obama (Green border)
* Angela Merkel (Blue border)
* **Edges:** Represent diplomatic actions or relationships between entities. Edges are labeled with the type of interaction (e.g., "make VisitTo", "negotiate", "express ExtendTo", "consult", "consure?", "sign Agreement", "make Statement").
* **Time Axis:** A horizontal axis labeled "time" with markers at t<sub>i-2</sub>, t<sub>i-1</sub>, and t<sub>i</sub>, indicating the temporal progression of interactions.
### Detailed Analysis or Content Details
**Time t<sub>i-2</sub>:**
* Barack Obama (Green border) "express ExtendTo" South Korea (Peach border).
* Barack Obama (Green border) "express ExtendTo" Angela Merkel (Blue border).
* Barack Obama (Green border) "make VisitTo" China (Blue border).
* Russia (Purple border) "negotiate" South Korea (Peach border).
**Time t<sub>i-1</sub>:**
* Angela Merkel (Blue border) "consult" Singapore (Peach border).
* Angela Merkel (Blue border) "consure?" Pakistan (Light Blue border) - Indicated by a red dashed line.
* Angela Merkel (Blue border) "express ExtendTo" North Korea (Green border).
* Angela Merkel (Blue border) "sign Agreement" South Korea (Peach border).
**Time t<sub>i</sub>:**
* Barack Obama (Green border) "make Statement" North Korea (Green border).
* Barack Obama (Green border) points to Pakistan (Light Blue border).
### Key Observations
* The diagram shows a network of diplomatic interactions evolving over time.
* Barack Obama is a central figure at time t<sub>i-2</sub>, engaging with multiple countries.
* Angela Merkel becomes a central figure at time t<sub>i-1</sub>, engaging with multiple countries.
* The "consure?" interaction between Angela Merkel and Pakistan is marked with a red dashed line, suggesting uncertainty or questioning.
* At time t<sub>i</sub>, Barack Obama's interaction is focused on North Korea and Pakistan.
### Interpretation
The diagram provides a visual representation of diplomatic relations and interactions between countries and individuals over time. It highlights the changing focus of diplomatic efforts, with Barack Obama's initial engagement with China and South Korea shifting towards North Korea and Pakistan. Angela Merkel's interactions at t<sub>i-1</sub> suggest a focus on European and Asian relations, with a possible point of contention or uncertainty ("consure?") with Pakistan. The diagram demonstrates the dynamic nature of international relations and the evolving priorities of different actors. The use of different colored borders around the nodes may indicate different political alignments or groupings.
</details>
(c) Interpolation reasoning on TKGs.
<details>
<summary>extracted/6596839/fig/extrapolation.png Details</summary>
### Visual Description
## Political Interaction Diagram: Predicting Future Statements
### Overview
The image is a diagram illustrating political interactions between various countries and leaders over time. It visualizes relationships and actions between entities like China, Russia, South Korea, Barack Obama, Angela Merkel, Singapore, Pakistan, and North Korea, and attempts to predict a future interaction. The diagram uses nodes to represent entities and directed edges to represent actions or relationships.
### Components/Axes
* **Nodes:** Represented by circular images with country flags or leader portraits. Each node has a label indicating the country or leader's name.
* China (light blue label)
* Russia (purple label)
* South Korea (pink label)
* Barack Obama (green label)
* Angela Merkel (dark blue label)
* Singapore (orange label)
* Pakistan (light blue label)
* North Korea (green label)
* **Edges:** Represented by arrows indicating the type of interaction between entities.
* "make VisitTo"
* "negotiate"
* "express ExtendTo"
* "consult"
* "consure"
* "sign Agreement"
* "make Statement ?" (dashed red arrow)
* **Time Axis:** A horizontal axis labeled "time" with markers at *t<sub>i-2</sub>*, *t<sub>i-1</sub>*, and *t<sub>i</sub>*.
### Detailed Analysis
* **Time t<sub>i-2</sub>:**
* Barack Obama (green label) "express ExtendTo" South Korea (pink label).
* Barack Obama (green label) "express ExtendTo" Angela Merkel (dark blue label).
* China (light blue label) "make VisitTo" Russia (purple label).
* Russia (purple label) "negotiate" South Korea (pink label).
* **Time t<sub>i-1</sub>:**
* Angela Merkel (dark blue label) "consult" Singapore (orange label).
* Angela Merkel (dark blue label) "consure" Pakistan (light blue label).
* Angela Merkel (dark blue label) "express ExtendTo" North Korea (green label).
* Angela Merkel (dark blue label) "sign Agreement" South Korea (pink label).
* **Prediction at Time t<sub>i</sub>:**
* The diagram predicts whether Barack Obama (green label) will "make Statement" to South Korea (pink label). This is indicated by a dashed red arrow and a question mark.
### Key Observations
* The diagram visualizes a sequence of political interactions over three time points.
* The interactions are directed, indicating the initiator and recipient of the action.
* The diagram attempts to predict a future interaction based on past events.
* Angela Merkel appears to be a central figure at time t<sub>i-1</sub>, engaging in multiple interactions.
### Interpretation
The diagram represents a simplified model of international relations, where countries and leaders interact with each other through various actions. The goal is to predict future interactions based on observed patterns. The use of a question mark suggests uncertainty in the prediction, highlighting the complexity of forecasting political events. The diagram could be used to illustrate a machine learning model that learns from historical interactions to predict future ones. The relationships are not weighted, so it is difficult to determine the strength of the relationship. The diagram is a snapshot of a complex system, and does not account for all possible factors that could influence future interactions.
</details>
(d) Extrapolation reasoning on TKGs.
Figure 2: Illustration of four reasoning scenarios on KGs: transductive, inductive, interpolation, and extrapolation. The red dashed arrows indicate the query fact to be predicted.
2.2 Logic Reasoning on KGs
Logical reasoning involves using a given set of facts (i.e., premises) to deduce new facts (i.e., conclusions) by a rigorous form of thinking [42, 43]. It generally covers propositional and first-order logic (also known as predicate logic). Propositional logic deals with declarative sentences that can be definitively assigned a truth value, leaving no room for ambiguity. It is usually known as multi-hop reasoning [44, 35] on KGs, which views each fact as a declarative sentence and usually reasons over query-related paths to obtain an answer. Thus, propositional reasoning on KGs is entity-dependent. First-order logic (FOL) can be regarded as an expansion of propositional logic, enabling the expression of more refined and nuanced ideas [42, 45]. FOL rules extend the modeling scope and application prospect by introducing quantifiers ( $β$ and $β$ ), predicates, and variables. They encompass variables that belong to a specific domain and encompass objects and relationships among those objects [46]. They are usually in the form of $premiseβ conclusion$ , where $premise$ and $conclusion$ denote the rule body and rule head which are all composed of atomic formulas. Each atomic formula consists of a predicate and several variables, e.g., $bornIn(X,Y)$ in $\gamma_{1}$ of Figure 1, where $bornIn$ is the predicate and $X$ and $Y$ are all entity variables. Thus, FOL reasoning is entity-independent, leveraging consistent FOL rules for different entities [47]. In this paper, we utilize Horn rules [48] to enhance the adaptability of FOL rules to various KG reasoning tasks. These rules entail setting the rule head to a single atomic formula. Furthermore, to make the Horn rules suitable for multiple reasoning scenarios, we introduce the following definitions.
Connected and Closed Horn (CCH) Rule. Based on Horn rules, CCH rules possess two distinct features, i.e., connected and closed. The term connected means the rule body necessitates a transitive and chained connection between atomic formulas through shared variables. Concurrently, the term closed indicates the rule body and rule head utilize identical start and end variables.
CCH rules of length $n$ (the quantifier $β$ would be omitted for better exhibition in the following parts of the paper) are in the following form:
$$
\begin{split}\epsilon,\;\forall&X,Y_{1},Y_{2},\cdots,Y_{n},Z\;\;r_{1}(X,Y_{1})%
\land r_{2}(Y_{1},Y_{2})\land\cdots\\
&\land r_{n}(Y_{n-1},Z)\rightarrow r(X,Z),\end{split} \tag{1}
$$
where atomic formulas in the rule body are connected by variables ( $X,Y_{1},Y_{2},Β·s,Y_{n-1},Z$ ). For example, $r_{1}(X,Y_{1})$ and $r_{2}(Y_{1},Y_{2})$ are connected by $Y_{1}$ . Meanwhile, all variables form a path from $X$ to $Z$ that are the start variable and end variable of rule head $r_{t}(X,Z)$ , respectively. $r_{1},r_{2},Β·s,r_{n},r$ are relations in KGs to represent predicates. To model different credibility of different rules, we configure a rule confidence $\epsilonβ[0,1]$ for each Horn rule. Rule length refers to the number of atomic formulas in the rule body. For example, $\gamma_{1}$ , $\gamma_{2}$ , and $\gamma_{3}$ in Figure 1 are three example Horn rules of lengths 2, 3, and 3. Rule grounding of a Horn rule can be realized by replacing each variable with a real entity, e.g., bornIn(Barack Obama, Hawaii) $\land$ locatedInCountry(Hawaii, U.S.A.) $β$ nationalityOf(Barack Obama, U.S.A.) is a grounding of rule $\gamma_{1}$ . CCH rules can be utilized for transductive and inductive reasoning.
Temporal Interpolation Horn (TIH) Rule. Based on CCH rules on static KGs that require connected and closed variables, TIH rules assign each atomic formula a time variable.
An example of TIH rule can be:
$$
\epsilon,\;\forall X,Y,Z\;\;r_{1}(X,Y):t_{1}\land r_{2}(Y,Z):t_{2}\rightarrow r%
(X,Z):t, \tag{2}
$$
where $t_{1}$ , $t_{2}$ and $t$ are time variables. To expand the model capacity when grounding TIH rules, time variables are virtual and do not have to be instantiated to real timestamps, which is distinct from the entity variables (e.g., $X$ , $Y$ , $Z$ ). However, we model the relative sequence of occurrence. This implies that TIH rules with the same atomic formulas but varying time variable conditions are distinct and may have different degrees of confidence, such as for $t_{1}<t_{2}$ vs. $t_{1}>t_{2}$ .
Temporal Extrapolation Horn (TEH) Rule. Based on CCH rules on static KGs that require connected and closed variables, TEH rules assign each atomic formula a time variable. Unlike TIH rules, TEH rules have the characteristic of time growth, which means the time sequence is increasing and the time in the rule head is the maximum.
For example, the following rule is a TEH rule with length 2:
$$
\begin{split}\epsilon,\;\forall X,Y,Z\;\;&r_{1}(X,Y):t_{1}\land r_{2}(Y,Z):t_{%
2}\\
&\rightarrow r(X,Z):t,\;\;s.t.\;\;t_{1}\leqslant t_{2}<t.\end{split} \tag{3}
$$
Noticeably, for rule learning and reasoning, $t_{1}$ , $t_{2}$ and $t$ are also virtual time variables that are only used to satisfy the time growth and do not have to be instantiated.
<details>
<summary>extracted/6596839/fig/arc.png Details</summary>
### Visual Description
## Diagram: Knowledge Graph Reasoning Process
### Overview
The image is a diagram illustrating a knowledge graph (KG) reasoning process. It shows the flow of information from an input KG and query through multiple logic blocks to produce reasoning scores as output.
### Components/Axes
* **Input:**
* Label: "Input" (top-left, green background)
* KG: A network graph with nodes of varying sizes and colors.
* Query: "(s, r, ?)" or "(s, r, ?, t)" (yellow background)
* Initial Embed (pink background)
* Initialize arrow from Query to Initial Embed
* **Logic Block # 1:**
* Label: "Logic Block # 1" (top-center, green background)
* Neighbor facts (light blue trapezoid)
* Facts: "Fact 1", "Fact 2", "Fact 3", ..., "Fact N-1", "Fact N" (gray rectangles)
* Expanding Reasoning Graph (light blue trapezoid)
* Logical Message-passing (light orange rectangle)
* Reasoning Graph (1 step) (yellow background)
* Updated Emb & Att (pink background)
* **Logic Block # N:**
* Label: "Logic Block # N" (top-right of center, green background)
* KG: A network graph with nodes of varying sizes and colors.
* Neighbor facts (light blue trapezoid)
* Facts: "Fact 1", "Fact 2", "Fact 3", ..., "Fact N-1", "Fact N" (gray rectangles)
* Expanding Reasoning Graph (light blue trapezoid)
* Logical Message-passing (light orange rectangle)
* Reasoning Graph (N-1) (yellow background)
* Updated Emb & Att (pink background)
* Reasoning Graph (N step) (yellow background)
* Updated Emb & Att (pink background)
* **Output:**
* Label: "Output" (top-right, green background)
* Updated Emb & Att (pink background)
* Reasoning scores: A bar graph representing scores (pink bars)
### Detailed Analysis or ### Content Details
The diagram depicts a multi-step reasoning process using a knowledge graph.
1. **Input:** The process starts with a KG and a query. The query is initialized to create an initial embedding.
2. **Logic Blocks:** The core of the process involves multiple logic blocks (Block #1 to Block #N). Each block performs the following steps:
* Neighbor facts are extracted.
* An expanding reasoning graph is constructed based on the facts.
* Logical message-passing is performed.
* The reasoning graph and embeddings/attention are updated.
3. **Output:** After passing through all logic blocks, the final updated embeddings and attention are used to generate reasoning scores.
### Key Observations
* The diagram illustrates an iterative process, where each logic block refines the reasoning graph and embeddings.
* The "..." notation within the facts list in each logic block indicates that there can be a variable number of facts considered.
* The KG is present in the Input and Logic Block #N.
### Interpretation
The diagram represents a system for reasoning over knowledge graphs. The iterative process of logic blocks allows the system to progressively refine its understanding of the KG and the query, leading to more accurate reasoning scores. The use of embeddings and attention mechanisms suggests that the system is learning to focus on the most relevant parts of the KG for answering the query. The diagram highlights the key components and flow of information in such a system, providing a high-level overview of its architecture and functionality.
</details>
Figure 3: An overview of the Tunsr. It utilizes multiple logic blocks to find the answer, where the reasoning graph is constructed and iteratively expanded. Meanwhile, a forward logic message-passing mechanism is proposed to update embeddings and attentions for unified propositional and FOL reasoning.
<details>
<summary>extracted/6596839/fig/rg2.png Details</summary>
### Visual Description
## Diagram: Knowledge Graph Expansion
### Overview
The image depicts a knowledge graph expansion diagram, starting with "Barack Obama" and expanding outwards through three iterations, showing relationships to other entities like "Michelle Obama", "Harvard University", "Hawaii", and others. The diagram uses nodes (circles) to represent entities and directed edges (arrows) to represent relationships between them. The diagram is divided into four vertical sections labeled "O0", "iteration 1", "iteration 2", and "iteration 3".
### Components/Axes
* **Nodes:** Represented by circles, each containing a name of an entity (e.g., "Barack Obama", "Michelle Obama", "Chicago"). The starting node "Barack Obama" is blue, while all other nodes are brown.
* **Edges:** Represented by blue arrows, each labeled with a relationship between the connected entities (e.g., "marriedTo", "graduateFrom", "bornIn").
* **Iterations:** The diagram is divided into four vertical sections, labeled "O0", "iteration 1", "iteration 2", and "iteration 3". These sections represent the expansion of the knowledge graph over successive iterations.
* **Labels:** Text labels are used to identify entities and relationships.
* **Vertical Sections:** The diagram is divided into vertical sections, each enclosed by dashed gray lines.
### Detailed Analysis or Content Details
**Iteration O0:**
* The starting node is a blue circle labeled "Barack Obama".
**Iteration 1:**
* From "Barack Obama", there are edges pointing to:
* "Michelle Obama" with the relationship "marriedTo".
* "Harvard University" with the relationship "graduateFrom".
* An ellipsis ("...") with the relationship "graduateFrom".
* "Hawaii" with the relationship "bornIn".
* "Malia Obama" with the relationship "fatherOf".
**Iteration 2:**
* From "Michelle Obama", there is an edge pointing to "Chicago" with the relationship "bornIn".
* From "Harvard University", there are edges pointing to:
* "Chicago" with the relationship "placeIn".
* "U.S.A." with the relationship "mascot".
* "Harvard University" with the relationship "self".
* From "Hawaii", there is an edge pointing to "Honolulu" with the relationship "capitalOf".
* From "Malia Obama", there is an edge pointing to "Honolulu" with the relationship "growUpIn".
* From the ellipsis ("...") in Iteration 1, there is an edge pointing to "Bill Gates" with the relationship "graduateFrom-1".
* From "Bill Gates", there is an edge pointing to an ellipsis ("...") in Iteration 3 with the relationship "founderOf".
* From "Honolulu", there is an edge pointing to "Honolulu" with the relationship "self".
* From "Sasha Obama", there is an edge pointing to "Sidwell Friends School" with the relationship "graduateFrom".
**Iteration 3:**
* From "Chicago", there is an edge pointing to "Chicago" with the relationship "self".
* From the ellipsis ("...") in Iteration 1, there is an edge pointing to "John Harvard" with the relationship "mascot".
* From "Bill Gates", there is an edge pointing to "Microsoft" with the relationship "founderOf".
* From "Honolulu", there is an edge pointing to "Honolulu" with the relationship "growUpIn".
* From "Sasha Obama", there is an edge pointing to "Sidwell Friends School" with the relationship "graduateFrom".
### Key Observations
* The diagram shows how knowledge about "Barack Obama" can be expanded by exploring relationships to other entities.
* The iterations represent successive levels of knowledge expansion.
* The relationships are diverse, including family relations, education, and location.
* The ellipsis ("...") nodes suggest that there are more relationships and entities that could be explored.
### Interpretation
The diagram illustrates a knowledge graph expansion process, starting from a central entity ("Barack Obama") and branching out to related entities through defined relationships. The iterations demonstrate how the knowledge graph can be expanded iteratively, revealing connections and information about the initial entity. The relationships shown are diverse, indicating the multifaceted nature of knowledge representation. The presence of ellipsis nodes suggests that the graph is not exhaustive and can be further expanded. The diagram highlights the potential for knowledge discovery and inference within a knowledge graph.
</details>
(a) An example of reasoning graph in SKGs.
<details>
<summary>extracted/6596839/fig/rg1.png Details</summary>
### Visual Description
## Diagram: Entity Relationship Iterations
### Overview
The image is a diagram illustrating entity relationships across three iterations, starting with "Catherine Ashton" as the query entity. The diagram shows connections to other entities or entity-time pairs, with labels indicating the type of relationship. The diagram is divided into columns representing iterations, with nodes representing entities and edges representing relationships.
### Components/Axes
* **Nodes:**
* Blue Node: Represents the "start node: query entity" (Catherine Ashton).
* Orange Nodes: Represent "subsequent node: entity or entity-time pair".
* **Edges:** Blue arrows indicating the relationship between entities. The relationship is labeled on the arrow.
* **Iterations:** The diagram is divided into four iterations, labeled "iteration 1", "iteration 2", and "iteration 3". The first iteration is labeled "00", the second "01", the third "02", and the fourth "03".
* **Legend:** Located in the bottom-right corner, explaining the node colors:
* Blue circle: "start node: query entity"
* Orange circle: "subsequent node: entity or entity-time pair"
### Detailed Analysis or ### Content Details
**Iteration 0 (O0):**
* The starting node is "Catherine Ashton" (blue).
* "Catherine Ashton" connects to "Catherine Ashton: 2014-01-01" (orange) via a "self" relationship.
* "Catherine Ashton" connects to "Mohammad Javad: 2014-10-01" (orange) via a "makeStatement" relationship.
* "Catherine Ashton" connects to an unspecified node (orange) via a "hostVisit" relationship. The node is represented by "...".
* "Catherine Ashton" connects to "Cabient: 2014-10-05" (orange) via a "consult" relationship.
**Iteration 1 (O1):**
* "Catherine Ashton: 2014-01-01" connects to "Catherine Ashton: 2014-01-01" (orange) via a "self" relationship.
* "Mohammad Javad: 2014-10-01" connects to "Iran: 2014-11-04" (orange) via a "makeVisit" relationship.
* The unspecified node ("...") connects to "China: 2014-10-30" (orange) via a "hostVisit" relationship.
* "Cabient: 2014-10-05" connects to "Iran: 2014-10-04" (orange) via a "consult" relationship.
* "Cabient: 2014-10-05" connects to "John Kerry: 2014-10-28" (orange) via a "meetTo" relationship.
**Iteration 2 (O2):**
* "Catherine Ashton: 2014-01-01" connects to "O3" (orange) via a "self" relationship.
* "Iran: 2014-11-04" connects to "Oman: 2014-11-04" (orange) via a "makeOptimisticComment" relationship.
* "China: 2014-10-30" connects to an unspecified node (orange) via an "expressIntentTo" relationship. The node is represented by "...".
* "Iran: 2014-10-04" connects to "John Kerry: 2014-11-05" (orange) via a "makeVisit" relationship.
* "John Kerry: 2014-10-28" connects to "Iran: 2014-11-08" (orange) via a "makeVisit" relationship.
**Iteration 3 (O3):**
* No further connections are shown from the nodes in Iteration 2.
* The nodes in this iteration are: "Catherine Ashton: 2014-01-01", "Oman: 2014-11-04", "Oman: 2014-11-08", "Iran: 2014-11-08".
### Key Observations
* The diagram shows a progression of relationships stemming from the initial entity "Catherine Ashton".
* The relationships are diverse, including "self", "makeStatement", "hostVisit", "consult", "makeVisit", "expressIntentTo", "meetTo", and "makeOptimisticComment".
* The diagram includes date information associated with some entities, suggesting a temporal aspect to the relationships.
* The use of "..." indicates that some branches of the relationship graph are not fully expanded.
### Interpretation
The diagram visualizes the network of relationships associated with "Catherine Ashton" over several iterations. It demonstrates how an initial entity can be connected to other entities through various types of interactions or associations. The inclusion of dates suggests that the diagram may be tracking the evolution of these relationships over time. The diagram could be used to analyze the influence or involvement of "Catherine Ashton" in different events or with different individuals. The presence of incomplete branches ("...") suggests that the diagram represents a partial view of the complete relationship network.
</details>
(b) An example of reasoning graph in TKGs.
Figure 4: Examples of the reasoning graph with three iterations. (a) is on SKGs while (b) is on TKGs.
3 Methodology
In this section, we present the technical details of our Tunsr model. It leverages a combination of logic blocks to obtain reasoning results, which involves constructing or expanding a reasoning graph and introducing a forward logic message-passing mechanism for propositional and FOL reasoning. The overall architecture is illustrated in Figure 3.
3.1 Reasoning Graph Construction
For each query of KGs, i.e., $\mathcal{Q}=(\tilde{s},\tilde{r},?)$ for SKGs or $\mathcal{Q}=(\tilde{s},\tilde{r},?,\tilde{t})$ for TKGs, we introduce an expanding reasoning graph to find the answer. The formulation is as follows.
Reasoning Graph. For a specific query $\mathcal{Q}$ , a reasoning graph is defined as $\widetilde{\mathcal{G}}=\{\mathcal{O},\mathcal{R},\widetilde{\mathcal{F}}\}$ for propositional and first-order reasoning. $\mathcal{O}$ is a node set that consists of nodes in different iteration steps, i.e., $\mathcal{O}=\mathcal{O}_{0}\cup\mathcal{O}_{1}\cupΒ·s\cup\mathcal{O}_{L}$ . For SKGs, $\mathcal{O}_{0}$ only contains a query entity $\tilde{s}$ and the subsequent is in the form of entities. $(n_{i}^{l},\bar{r},n_{j}^{l+1})β\widetilde{\mathcal{F}}$ is an edge that links nodes at two neighbor steps, i.e., $n_{i}^{l}β\mathcal{O}_{l}$ , $n_{j}^{l+1}β\mathcal{O}_{l+1}$ and $\bar{r}β\mathcal{R}$ . The reasoning graph is constantly expanded by searching for posterior neighbor nodes. For start node $n^{0}=\tilde{s}$ , its posterior neighbors are $\mathcal{N}(n^{0})=\{e_{i}|(\tilde{s},\bar{r},e_{i})β\mathcal{F}\}$ . For a node in following steps $n_{i}^{l}=e_{i}β\mathcal{O}_{l}$ , its posterior neighbors are $\mathcal{N}(n_{i}^{l})=\{e_{j}|(e_{i},\bar{r},e_{j})β\mathcal{F}\}$ . Its preceding parents are $\widetilde{\mathcal{N}}(n_{i}^{l})=\{(n_{j}^{l-1},\bar{r})|n_{j}^{l-1}β%
\mathcal{O}_{l-1}\land(n_{j}^{l-1},\bar{r},n_{i}^{l})β\widetilde{\mathcal{F}}\}$ . To take preceding nodes into account at the current step, an extra relation self is added. Then, $n_{i}^{l}=e_{i}$ can be obtained at the next step as $n_{i}^{l+1}=e_{i}$ and there have $(n_{i}^{l},self,n_{i}^{l+1})β\widetilde{\mathcal{F}}$ .
For TKGs, $\mathcal{O}_{0}$ also contains a query entity $\tilde{s}$ . But the following nodes are in the form of entity-time pairs. In the interpolation scenarios, for start node $n^{0}=\tilde{s}$ , its posterior neighbors are $\mathcal{N}(n^{0})=\{(e_{i},t_{i})|(\tilde{s},\bar{r},e_{i},t_{i})β\mathcal{%
F}\}$ . For a node in following steps $n_{i}^{l}=(e_{i},t_{i})β\mathcal{O}_{l}$ , its posterior neighbors are $\mathcal{N}(n_{i}^{l})=\{(e_{j},t_{j})|(e_{i},\bar{r},e_{j},t_{j})β\mathcal{%
F}\}$ . Differently, in the extrapolation scenarios, for start node $n^{0}=\tilde{s}$ , its posterior neighbors are $\mathcal{N}(n^{0})=\{(e_{i},t_{i})|(\tilde{s},\bar{r},e_{i},t_{i})β\mathcal{%
F}\land t_{i}<\tilde{t}\}$ . For a node in following steps $n_{i}^{l}=(e_{i},t_{i})β\mathcal{O}_{l}$ , its posterior neighbors are $\mathcal{N}(n_{i}^{l})=\{(e_{j},t_{j})|(e_{i},\bar{r},e_{j},t_{j})β\mathcal{%
F}\land t_{i}β€slant t_{j}\land t_{j}<\tilde{t}\}$ . Similar to the situation of SKGs, the preceding parents of nodes in TKG scenarios are also $\widetilde{\mathcal{N}}(n_{i}^{l})=\{(n_{j}^{l-1},\bar{r})|n_{j}^{l-1}β%
\mathcal{O}_{l-1}\land(n_{j}^{l-1},\bar{r},n_{i}^{l})β\widetilde{\mathcal{F}}\}$ and an extra relation self is also added. Then, $n_{i}^{l}=(e_{i},t_{i})$ can be obtained at the next step as $n_{i}^{l+1}=(e_{i},t_{i})$ ( $t_{i}$ is the minimum time if $l=0$ ) and there have $(n_{i}^{l},self,n_{i}^{l+1})β\widetilde{\mathcal{F}}$ .
Two examples of the reasoning graph with three iterations are shown in Figure 4. Through the above processing, we can model both propositional and FOL reasoning in a unified manner for different reasoning scenarios.
3.2 Modeling of Propositional Reasoning
For decoding the answer for a specific query $\mathcal{Q}$ , we introduce an iterative forward message-passing mechanism in a continuously expanding reasoning graph, regulated by propositional and FOL reasoning. In the reasoning graph, we set two learnable parameters for each node $n_{i}^{l}$ to guide the propositional computation: propositional embedding ${\rm\textbf{x}}_{i}^{l}$ and propositional attention ${\alpha}_{n_{i}^{l}}$ . For a better presentation, we employ the reasoning process on TKGs to illustrate our method. SKGs can be considered a specific case of TKGsβ when the time information of the nodes in the reasoning graph is removed. The initialized embeddings for entity, relation, and time are formalized as h, g, and e. Time embeddings are obtained by the generic time encoding [49] as it is fully compatible with attention to capture temporal dynamics, which is defined as: ${\rm\textbf{e}}_{t}\!=\!\sqrt{\frac{1}{d_{t}}}[{\rm cos}(w_{1}t+b_{1}),Β·s,%
{\rm cos}(w_{d_{t}}t+b_{d_{t}})]$ , where $[w_{1},Β·s,w_{d_{t}}]$ and $[b_{1},Β·s,b_{d_{t}}]$ are trainable parameters for transformation weights and biases. cos denotes the standard cosine function and $d_{t}$ is the dimension of time embedding.
Further, the start node $n^{0}$ = $\tilde{s}$ is initialized as its embedding ${\rm\textbf{x}}_{\tilde{s}}={\rm\textbf{h}}_{\tilde{s}}$ . The node $n_{i}=(e_{i},t_{i})$ at the following iterations is firstly represented by the linear transformation of embeddings: ${\rm\textbf{x}}_{i}$ = ${\rm\textbf{W}}_{n}[{\rm\textbf{h}}_{e_{i}}\|{\rm\textbf{e}}_{t_{i}}]$ (W represents linear transformation and $\|$ denotes the embedding concatenation in the paper). Constant forward computation is required in the reasoning sequence of the target when conducting multi-hop propositional reasoning. Thus, forward message-passing is proposed to pass information (i.e., representations and attention weights) from the preceding nodes to their posterior neighbor nodes. The computation of each node is contextualized with preceding information that contains both entity-dependent parts, reflecting the continuous accumulation of knowledge and credibility in the reasoning process. Specifically, to update node embeddings in step $l$ +1, its own feature and the information from its priors are integrated:
$$
{\rm\textbf{x}}_{j}^{l+1}={\rm\textbf{W}}_{1}^{l}{\rm\textbf{x}}_{j}+\!\!\!\!%
\sum_{(n_{i}^{l},\bar{r})\in\widetilde{\mathcal{N}}(n_{j}^{l+1})}\!\!\!\!%
\alpha_{n_{i}^{l},\bar{r},n_{j}^{l+1}}{\rm\textbf{W}}_{2}^{l}{\rm\textbf{m}}_{%
n_{i}^{l},\bar{r},n_{j}^{l+1}}, \tag{4}
$$
where ${\rm\textbf{m}}_{n_{i}^{l},\bar{r},n_{j}^{l+1}}$ is the message from a preceding node to its posterior node, which is given by the node and relation representations:
$$
{\rm\textbf{m}}_{n_{i}^{l},\bar{r},n_{j}^{l+1}}\!=\!{\rm\textbf{W}}_{3}^{l}[{%
\rm\textbf{n}}_{i}^{l}\|{\rm\textbf{g}}_{\bar{r}}\|{\rm\textbf{n}}_{j}]. \tag{5}
$$
This updating form superficially seems similar to the general message-passing in GNNs [16]. However, they are actually different as ours is in a one-way and hierarchical manner, which is tailored for the tree-like structure of the reasoning graph. The propositional attention weight $\alpha_{n_{i}^{l},\bar{r},n_{j}^{l+1}}$ is for each edge in a reasoning graph. As propositional reasoning is entity-dependent, we compute it by the semantic association of entity-dependent embeddings between the message and the query:
$$
e_{n_{i}^{l},\bar{r},n_{j}^{l+1}}\!=\!\textsc{sigmoid}({\rm\textbf{W}}_{4}^{l}%
[{\rm\textbf{m}}_{n_{i}^{l},\bar{r},n_{j}^{l+1}}\|{\rm\textbf{q}}]), \tag{6}
$$
where ${\rm\textbf{q}}={\rm\textbf{W}}_{q}[{\rm\textbf{h}}_{\tilde{s}}\|{\rm\textbf{g%
}}_{\tilde{r}}\|{\rm\textbf{e}}_{\tilde{t}}]$ is the query embedding. Then, the softmax normalization is utilized to scale edge attentions on this iteration to [0,1]:
$$
\alpha_{\!n_{i}^{l},\bar{r},n_{j}^{l+1}}\!\!=\!\!\frac{\exp(e_{n_{i}^{l},\bar{%
r},n_{j}^{l+1}})}{\sum_{(\!n_{i^{\prime}}^{l},\bar{r}^{\prime})\in\widetilde{%
\mathcal{N}}(n_{j}^{l+1}\!)}\!\!\exp(e_{n_{i^{\prime}}^{l},\bar{r}^{\prime},n_%
{j}^{l+1}}\!)}, \tag{7}
$$
Finally, the propositional attention of new node $n_{j}^{l+1}$ is aggregated from edges for the next iteration:
$$
\begin{split}&\alpha_{n_{j}^{l+1}}\!=\!\!\!\sum_{(n_{i}^{l},\bar{r})\in%
\widetilde{\mathcal{N}}(n_{j}^{l+1})}\!\!\!\!\!\!\!\!\alpha_{n_{i}^{l},\bar{r}%
,n_{j}^{l+1}}.\end{split} \tag{8}
$$
3.3 Modeling of FOL Reasoning
Different from propositional reasoning, FOL reasoning is entity-independent and has a better ability for generalization. As first-order reasoning focuses on the interaction among entity-independent relations, we first obtain the hidden FOL embedding of an edge by fusing the hidden FOL embedding of the preceding node and current relation representation via a GRU [38]. Then, the FOL representation y and attention $b$ are given by:
$$
{\rm\textbf{y}}_{n_{i}^{l},\bar{r},n_{j}^{l+1}}\!=\!\textsc{gru}({\rm\textbf{g%
}}_{\bar{r}},{\rm\textbf{y}}_{n_{i}^{l}}), \tag{9}
$$
$$
b_{n_{i}^{l},\bar{r},n_{j}^{l+1}}\!=\!\textsc{sigmoid}({\rm\textbf{W}}_{5}^{l}%
{\rm\textbf{y}}_{n_{i}^{l},\bar{r},n_{j}^{l+1}}). \tag{10}
$$
Since the preceding node with high credibility leads to faithful subsequent nodes, the attention of the prior ( $\beta$ ) flows to the current edge. Then, the softmax normalization is utilized to scale edge attentions on this iteration to [0,1]:
$$
\begin{split}b_{n_{i}^{l},\bar{r},n_{j}^{l+1}}&=\beta_{\!n_{i}^{l}}\cdot b_{n_%
{i}^{l},\bar{r},n_{j}^{l+1}},\;\;\\
\beta_{\!n_{i}^{l},\bar{r},n_{j}^{l+1}}\!\!&=\!\!\frac{\exp(b_{n_{i}^{l},\bar{%
r},n_{j}^{l+1}})}{\sum_{(\!n_{i^{\prime}}^{l},\bar{r}^{\prime})\in\widetilde{%
\mathcal{N}}(n_{j}^{l+1}\!)}\!\!\exp(b_{n_{i^{\prime}}^{l},\bar{r}^{\prime},n_%
{j}^{l+1}}\!)},\end{split} \tag{11}
$$
Finally, the FOL representation and attention of a new node $n_{j}^{l+1}$ are aggregated from edges for the next iteration:
$$
\begin{split}{\rm\textbf{y}}_{n_{j}^{l+1}}\!&=\!\!\!\sum_{(n_{i}^{l},\bar{r})%
\in\widetilde{\mathcal{N}}(n_{j}^{l+1})}\!\!\!\!\beta_{n_{i}^{l},\bar{r},n_{j}%
^{l+1}}{\rm\textbf{y}}_{n_{i}^{l},\bar{r},n_{j}^{l+1}},\\
&\beta_{n_{j}^{l+1}}\!=\!\!\!\sum_{(n_{i}^{l},\bar{r})\in\widetilde{\mathcal{N%
}}(n_{j}^{l+1})}\!\!\!\!\!\!\!\!\beta_{n_{i}^{l},\bar{r},n_{j}^{l+1}}.\end{split} \tag{12}
$$
Insights of FOL Rule Learning and Reasoning.
Actually, Tunsr introduces a novel FOL learning and reasoning strategy by forward logic message-passing mechanism over reasoning graphs. In general, the learning and reasoning of FOL rules on KGs or TKGs are usually in two-step fashion [20, 50, 51, 33, 28, 23, 18]. First, it searches over whole data to mine rules and their confidences. Second, for a query, the model instantiates all variables to find all groundings of learned rules and then aggregates all confidences of eligible rules. For example, for a target entity $o$ , its score can be the sum of learned rules with valid groundings and rule confidences can be modeled by a GRU. However, this is apparently not differentiable and cannot be optimized in an end-to-end manner because of the discrete rule learning and grounding operations. Thus, our model conducts the transformation of merging multiple rules by merging possible relations at each step, using FOL attention as:
$$
\begin{split}&\underbrace{S_{o}\!=\!\sum_{\gamma\in\Gamma}\beta_{\gamma}\!=\!%
\sum_{\gamma\in\Gamma}f\big{[}\textsc{gru}({\rm\textbf{g}}_{\gamma,h},{\rm%
\textbf{g}}_{\gamma,b^{1}},\cdots,{\rm\textbf{g}}_{\gamma,b^{|\gamma|}})]}_{(a%
)}\\
&\underbrace{\approx\prod_{l=1}^{L}\sum_{n_{j}\in\mathcal{O}_{l}}\bar{f_{l}}%
\big{[}\textsc{gru}({\rm\textbf{g}}_{\bar{r}},{\rm\textbf{o}}_{n_{j}}^{l}))%
\big{]}}_{(b)}.\end{split} \tag{13}
$$
$\beta_{\gamma}$ is the confidence of rule $\gamma$ . ${\rm\textbf{g}}_{\gamma,h}$ and ${\rm\textbf{g}}_{\gamma,b^{i}}$ are the relation embeddings of head $h$ and $i$ -th body $b^{i}$ of this rule. Part (a) utilizes the grounding of the learned rules to calculate reasoning scores, where each ruleβs confidence can be modeled by GRU and feedforward network $f$ . We can conduct reasoning at each step rather than whole multi-step processing, so the previous can approximate to part (b). $\bar{f_{l}}$ is for the attention calculation. In this way, the differentiable process is achieved. This is an extension and progression of Neural LP [21] and DURM [32] by introducing several specific strategies for unified KG reasoning. Finally, the real FOL rules can be easily induced to constantly perform attention calculation over the reasoning graph, which is summarized as the Forward Attentive Rule Induction (FARI) algorithm. It is shown in Algorithm 1, where the situation on TKGs is given and that on SKGs can be obtained by omitting time information. In this way, Tunsr has the ability to capture CCH, TIH, and TEH rules with the specific-designed reasoning graphs as described in Section 3.1. As we add an extra self relation in the reasoning graph, the FARI algorithm can obtain all possible rules (no longer than length L) by deleting existing atoms with the self relation in induced FOL rules.
Input: the reasoning graph $\widetilde{\mathcal{G}}$ , FOL attentions $\beta$ .
Output: the FOL rule set $\Gamma$ .
1 Init $\Gamma=\varnothing$ , $B(n_{\tilde{s}}^{0})=[0,[]]$ , $\mathcal{D}_{0}[n_{\tilde{s}}^{0}]=[1,B(n_{\tilde{s}}^{0})]$ ;
2 for l=1 to L of decoder iterations do
3 Initialize node-rule dictionary $\mathcal{D}_{l}$ ;
4 for node $n_{j}^{l}$ in $\mathcal{O}_{l}$ do
5 Set rule body list $B(n_{j}^{l})$ = [] ;
6 for ( $n_{i}^{l-1},\bar{r}$ ) of $\widetilde{\mathcal{N}}$ ( $n_{j}^{l}$ ) in $\mathcal{O}_{l-1}$ do
7 Prior $e_{i,l-1}^{2}$ , $B(n_{i}^{l-1})$ = $\mathcal{D}_{l-1}[n_{i}^{l-1}]$ ;
8 for weight $\epsilon$ , body $\gamma_{b}$ in $B(n_{i}^{l-1})$ do
9 $\epsilon^{\prime}=e_{i,l-1}^{2}Β· e_{n_{i}^{l-1},\bar{r},n_{j}^{l}}^{2}$ ;
10 $\gamma^{\prime}_{b}=\gamma_{b}.add(\bar{r})$ , $B(n_{j}^{l}).add([\epsilon^{\prime},\gamma^{\prime}_{b}])$ ;
11
12
13 $e_{j,l}^{2}=sum\{[\epsilonβ B(n_{j}^{l})]\}$ ;
14 Add $n_{j}^{l}$ : [ $e_{j,l}^{2}$ , $B(n_{j}^{l})$ ] to $\mathcal{D}_{l}$ ;
15
16 Normalize $e_{j,l}^{2}$ of $n_{j}^{l}$ in $\mathcal{O}_{l}$ using softmax;
17
18 for $n_{i}^{L}$ in $\mathcal{O}_{L}$ do
19 $e_{i,L}^{2}$ , $B(n_{i}^{L})$ = $\mathcal{D}_{L}[n_{j}^{L}]$ ;
20 for $\epsilon,\gamma_{b}$ in $B(n_{i}^{L})$ do
21 $\Gamma.add([\epsilon,\gamma_{b}[1](X,Y_{1}):t_{1}\landΒ·s\land\gamma_{b}[L]%
(Y_{L-1},Z):t_{L}β\tilde{r}(X,Z):t])$
22
Return rule set $\Gamma$ .
Algorithm 1 FARI for FOL rules Induction.
3.4 Reasoning Prediction and Process Overview
After calculation with $L$ logic blocks, the reasoning score for each entity can be obtained. For each entity $o$ at the last step of the reasoning graph for SKGs, we can utilize the representation and attention value of the propositional and FOL reasoning for calculating answer validity:
$$
{\rm\textbf{h}}_{o}=(1-\lambda){\rm\textbf{x}}_{o}+\lambda{\rm\textbf{y}}_{o},%
\gamma_{o}=(1-\lambda)\alpha_{o}+\lambda\beta_{o}, \tag{14}
$$
where $\lambda$ is a learnable weight for the combination of propositional and FOL reasoning. $\alpha_{o}$ and $\beta_{o}$ are learned attention values for propositional and FOL reasoning, respectively. We calculate it dynamically using propositional embedding ${\rm\textbf{x}}_{o}$ , FOL embedding ${\rm\textbf{y}}_{o}$ , and query embedding q. Based on it, the final score is given by:
$$
s(\mathcal{Q},o)={\rm\textbf{W}}_{5}{\rm\textbf{h}}_{o}+\gamma_{o}. \tag{15}
$$
Reasoning scores for those entities that are not in the last step of the reasoning graph are set to 0 as it indicates that there are no available propositional and FOL rules for those entities. Finally, the model is optimized by the multi-class log-loss [52] like RED-GNN:
$$
\mathcal{L}=\sum_{\mathcal{Q}}\Big{[}-s(\mathcal{Q},o)+\log\big{(}\sum_{\bar{o%
}\in\mathcal{E}}\exp(s(\mathcal{Q},\bar{o}))\big{)}\Big{]}, \tag{16}
$$
where $s(\mathcal{Q},o)$ denotes the reasoning score of labeled entity $o$ for query $\mathcal{Q}$ , while $\bar{o}$ is the arbitrary entity. For reasoning situations of TKGs, we need firstly aggregate node embedding and attentions with the same entity to get the entity score. Because the nodes in the reasoning graph of TKGs except the start node are in the form of entity-time pair.
The number of nodes may explode in the reasoning graph as it shows an exponential increase to reach $|\mathcal{N}(n_{i})|^{L}$ by iterations, especially for TKGs. For computational efficiency, we introduce the strategies of iteration fusion and sampling for interpolation and extrapolation reasoning, respectively. In the interpolation scenarios, nodes of entity-time pairs with the same entity are fused to an entity node and then are used to expand the reasoning graph. In the extrapolation scenarios, posterior neighbors of each node are sampled with a maximum of M nodes in each iteration. For sampling M node in the reasoning graph, we follow a time-aware weighted sampling strategy, considering that recent events may have a greater impact on the forecast target. Specifically, for a posterior neighbor node with time $t^{\prime}$ , we compute its sampling weight by $\frac{\exp(t^{\prime}-\tilde{t})}{\sum_{\bar{t}}{\exp(\bar{t}-\tilde{t})}}$ for the query ( $\tilde{s}$ , $\tilde{r}$ ,?, $\tilde{t}$ ), where $\bar{t}$ denotes the time of all possible posterior neighbor nodes for a prior node. After computing attention weights for each edge in the same iteration, we select top- N among them with larger attention weights and prune others.
4 Experiments and Results
4.1 Experiment Setups
The baselines cover a wide range of mainstream techniques and strategies for KG reasoning, with detailed descriptions provided in the Appendix. In the following parts of this section, we will carry out experiments and analyze results to answer the following four research questions.
$\bullet$ RQ1. How does the unified Tunsr perform in KG reasoning compared to state-of-the-art baselines?
$\bullet$ RQ2. How effective are propositional and FOL reasoning, and is it reasonable to integrate them?
$\bullet$ RQ3. What factors affect the reasoning performance of the Tunsr framework?
$\bullet$ RQ4. What is the actual reasoning process of Tunsr?
4.2 Comparison Results (RQ1)
TABLE II: The experiment results of transductive reasoning. The optimal and suboptimal values on each metric are marked in red and blue, respectively. The percent signs (%) for Hits@k metrics are omitted for better presentation. The following tables have a similar setting.
| Model | WN18RR | FB15k237 | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR | Hits@1 | Hits@3 | Hits@10 | MRR | Hits@1 | Hits@3 | Hits@10 | |
| TransE [19] | 0.481 | 43.30 | 48.90 | 57.00 | 0.342 | 24.00 | 37.80 | 52.70 |
| DistMult [53] | 0.430 | 39.00 | 44.00 | 49.00 | 0.241 | 15.50 | 26.30 | 41.90 |
| UltraE [54] | 0.485 | 44.20 | 50.00 | 57.30 | 0.349 | 25.10 | 38.50 | 54.10 |
| ComplEx-DURA [55] | 0.491 | 44.90 | β | 57.10 | 0.371 | 27.60 | β | 56.00 |
| AutoBLM [56] | 0.490 | 45.10 | β | 56.70 | 0.360 | 26.70 | β | 55.20 |
| SE-GNN [57] | 0.484 | 44.60 | 50.90 | 57.20 | 0.365 | 27.10 | 39.90 | 54.90 |
| RED-GNN [58] | 0.533 | 48.50 | β | 62.40 | 0.374 | 28.30 | β | 55.80 |
| CompoundE [59] | 0.491 | 45.00 | 50.80 | 57.60 | 0.357 | 26.40 | 39.30 | 54.50 |
| GATH [60] | 0.463 | 42.60 | 47.50 | 53.70 | 0.344 | 25.30 | 37.60 | 52.70 |
| TGformer [61] | 0.493 | 45.50 | 50.90 | 56.60 | 0.372 | 27.90 | 41.00 | 55.70 |
| AMIE [62] | 0.360 | 39.10 | β | 48.50 | 0.230 | 14.80 | β | 41.90 |
| AnyBURL [63] | 0.454 | 39.90 | β | 56.20 | 0.342 | 25.80 | β | 50.20 |
| SAFRAN [64] | 0.501 | 45.70 | β | 58.10 | 0.370 | 28.70 | β | 53.10 |
| Neural LP [21] | 0.381 | 36.80 | 38.60 | 40.80 | 0.237 | 17.30 | 25.90 | 36.10 |
| DRUM [32] | 0.382 | 36.90 | 38.80 | 41.00 | 0.238 | 17.40 | 26.10 | 36.40 |
| RLogic [23] | 0.470 | 44.30 | β | 53.70 | 0.310 | 20.30 | β | 50.10 |
| RNNLogic [33] | 0.483 | 44.60 | 49.70 | 55.80 | 0.344 | 25.20 | 38.00 | 53.00 |
| LatentLogic [24] | 0.481 | 45.20 | 49.70 | 55.30 | 0.320 | 21.20 | 32.90 | 51.40 |
| RNN+RotE [65] | 0.550 | 51.00 | 57.20 | 63.50 | 0.353 | 26.50 | 38.70 | 52.90 |
| TCRA [66] | 0.496 | 45.70 | 51.10 | 57.40 | 0.367 | 27.50 | 40.30 | 55.40 |
| Tunsr | 0.558 | 51.36 | 58.25 | 65.78 | 0.389 | 28.82 | 41.83 | 57.15 |
TABLE III: The experiment results on 12 inductive reasoning datasets.
| | Model | WN18RR | FB15k-237 | NELL-995 | | | | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| V1 | V2 | V3 | V4 | V1 | V2 | V3 | V4 | V1 | V2 | V3 | V4 | | |
| MRR | GraIL [67] | 0.627 | 0.625 | 0.323 | 0.553 | 0.279 | 0.276 | 0.251 | 0.227 | 0.481 | 0.297 | 0.322 | 0.262 |
| RED-GNN [58] | 0.701 | 0.690 | 0.427 | 0.651 | 0.369 | 0.469 | 0.445 | 0.442 | 0.637 | 0.419 | 0.436 | 0.363 | |
| MLSAA [68] | 0.716 | 0.700 | 0.448 | 0.654 | 0.368 | 0.457 | 0.442 | 0.431 | 0.694 | 0.424 | 0.433 | 0.359 | |
| RuleN [69] | 0.668 | 0.645 | 0.368 | 0.624 | 0.363 | 0.433 | 0.439 | 0.429 | 0.615 | 0.385 | 0.381 | 0.333 | |
| Neural LP [21] | 0.649 | 0.635 | 0.361 | 0.628 | 0.325 | 0.389 | 0.400 | 0.396 | 0.610 | 0.361 | 0.367 | 0.261 | |
| DRUM [32] | 0.666 | 0.646 | 0.380 | 0.627 | 0.333 | 0.395 | 0.402 | 0.410 | 0.628 | 0.365 | 0.375 | 0.273 | |
| Tunsr | 0.721 | 0.722 | 0.451 | 0.656 | 0.375 | 0.474 | 0.462 | 0.456 | 0.746 | 0.427 | 0.455 | 0.387 | |
| Hits@1 | GraIL [67] | 55.40 | 54.20 | 27.80 | 44.30 | 20.50 | 20.20 | 16.50 | 14.30 | 42.50 | 19.90 | 22.40 | 15.30 |
| RED-GNN [58] | 65.30 | 63.30 | 36.80 | 60.60 | 30.20 | 38.10 | 35.10 | 34.00 | 52.50 | 31.90 | 34.50 | 25.90 | |
| MLSAA [68] | 66.20 | 64.50 | 39.10 | 61.20 | 29.20 | 36.60 | 35.60 | 34.00 | 56.00 | 33.30 | 34.30 | 25.30 | |
| RuleN [69] | 63.50 | 61.10 | 34.70 | 59.20 | 30.90 | 34.70 | 34.50 | 33.80 | 54.50 | 30.40 | 30.30 | 24.80 | |
| Neural LP [21] | 59.20 | 57.50 | 30.40 | 58.30 | 24.30 | 28.60 | 30.90 | 28.90 | 50.00 | 24.90 | 26.70 | 13.70 | |
| DRUM [32] | 61.30 | 59.50 | 33.00 | 58.60 | 24.70 | 28.40 | 30.80 | 30.90 | 50.00 | 27.10 | 26.20 | 16.30 | |
| Tunsr | 66.25 | 66.31 | 38.11 | 61.55 | 30.44 | 37.88 | 37.90 | 36.37 | 73.13 | 32.67 | 37.13 | 27.30 | |
| Hits@10 | GraIL [67] | 76.00 | 77.60 | 40.90 | 68.70 | 42.90 | 42.40 | 42.40 | 38.90 | 56.50 | 49.60 | 51.80 | 50.60 |
| RED-GNN [58] | 79.90 | 78.00 | 52.40 | 72.10 | 48.30 | 62.90 | 60.30 | 62.10 | 86.60 | 60.10 | 59.40 | 55.60 | |
| MLSAA [68] | 81.10 | 79.60 | 54.40 | 72.40 | 49.00 | 61.60 | 58.90 | 59.70 | 87.80 | 59.40 | 59.20 | 55.00 | |
| RuleN [69] | 73.00 | 69.40 | 40.70 | 68.10 | 44.60 | 59.90 | 60.00 | 60.50 | 76.00 | 51.40 | 53.10 | 48.40 | |
| Neural LP [21] | 77.20 | 74.90 | 47.60 | 70.60 | 46.80 | 58.60 | 57.10 | 59.30 | 87.10 | 56.40 | 57.60 | 53.90 | |
| DRUM [32] | 77.70 | 74.70 | 47.70 | 70.20 | 47.40 | 59.50 | 57.10 | 59.30 | 87.30 | 54.00 | 57.70 | 53.10 | |
| Tunsr | 85.87 | 83.98 | 60.76 | 73.28 | 55.96 | 63.24 | 61.43 | 63.28 | 88.56 | 62.14 | 61.05 | 58.78 | |
TABLE IV: The experiment results (Hits@10 metrics) on 12 inductive reasoning datasets with 50 negative entities for ranking.
| Model | WN18RR | FB15k-237 | NELL-995 | | | | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| V1 | V2 | V3 | V4 | V1 | V2 | V3 | V4 | V1 | V2 | V3 | V4 | |
| GraIL [67] | 82.45 | 78.68 | 58.43 | 73.41 | 64.15 | 81.80 | 82.83 | 89.29 | 59.50 | 93.25 | 91.41 | 73.19 |
| CoMPILE [70] | 83.60 | 79.82 | 60.69 | 75.49 | 67.64 | 82.98 | 84.67 | 87.44 | 58.38 | 93.87 | 92.77 | 75.19 |
| TACT [71] | 84.04 | 81.63 | 67.97 | 76.56 | 65.76 | 83.56 | 85.20 | 88.69 | 79.80 | 88.91 | 94.02 | 73.78 |
| RuleN [69] | 80.85 | 78.23 | 53.39 | 71.59 | 49.76 | 77.82 | 87.69 | 85.60 | 53.50 | 81.75 | 77.26 | 61.35 |
| Neural LP [21] | 74.37 | 68.93 | 46.18 | 67.13 | 52.92 | 58.94 | 52.90 | 55.88 | 40.78 | 78.73 | 82.71 | 80.58 |
| DRUM [32] | 74.37 | 68.93 | 46.18 | 67.13 | 52.92 | 58.73 | 52.90 | 55.88 | 19.42 | 78.55 | 82.71 | 80.58 |
| ConGLR [26] | 85.64 | 92.93 | 70.74 | 92.90 | 68.29 | 85.98 | 88.61 | 89.31 | 81.07 | 94.92 | 94.36 | 81.61 |
| SymRITa [72] | 91.22 | 88.32 | 73.22 | 81.67 | 74.87 | 84.41 | 87.11 | 88.97 | 64.50 | 94.22 | 95.43 | 85.56 |
| Tunsr | 93.69 | 93.72 | 86.48 | 89.27 | 95.37 | 89.33 | 89.38 | 92.16 | 89.05 | 97.91 | 94.69 | 92.63 |
TABLE V: The experiment results of interpolation reasoning, including ICEWS14, ICEWS0515 and ICEWS18 datasets.
| Model | ICEWS14 | ICEWS0515 | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR | Hits@1 | Hits@3 | Hits@10 | MRR | Hits@1 | Hits@3 | Hits@10 | |
| TTransE [73] | 0.255 | 7.40 | β | 60.10 | 27.10 | 8.40 | β | 61.60 |
| DE-SimplE [74] | 0.526 | 41.80 | 59.20 | 72.50 | 0.513 | 39.20 | 57.80 | 74.80 |
| TA-DistMult [75] | 0.477 | 36.30 | β | 68.60 | 0.474 | 34.60 | β | 72.80 |
| ChronoR [76] | 0.625 | 54.70 | 66.90 | 77.30 | 0.675 | 59.60 | 72.30 | 82.00 |
| TComplEx [77] | 0.610 | 53.00 | 66.00 | 77.00 | 0.660 | 59.00 | 71.00 | 80.00 |
| TNTComplEx [77] | 0.620 | 52.00 | 66.00 | 76.00 | 0.670 | 59.00 | 71.00 | 81.00 |
| TeLM [78] | 0.625 | 54.50 | 67.30 | 77.40 | 0.678 | 59.90 | 72.80 | 82.30 |
| BoxTE [79] | 0.613 | 52.80 | 66.40 | 76.30 | 0.667 | 58.20 | 71.90 | 82.00 |
| RotateQVS [80] | 0.591 | 50.70 | 64.20 | 75.04 | 0.633 | 52.90 | 70.90 | 81.30 |
| TeAST [27] | 0.637 | 56.00 | 68.20 | 78.20 | 0.683 | 60.40 | 73.20 | 82.90 |
| Tunsr | 0.648 | 56.21 | 69.61 | 80.16 | 0.705 | 59.89 | 74.67 | 83.95 |
TABLE VI: The experiment results of extrapolation reasoning, including ICEWS14, ICEWS0515, and ICEWS18 datasets.
| Model | ICEWS14 | ICEWS0515 | ICEWS18 | | | | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR | Hits@1 | Hits@3 | Hits@10 | MRR | Hits@1 | Hits@3 | Hits@10 | MRR | Hits@1 | Hits@3 | Hits@10 | |
| TransE [19] | 0.224 | 13.36 | 25.63 | 41.23 | 0.225 | 13.05 | 25.61 | 42.05 | 0.122 | 5.84 | 12.81 | 25.10 |
| DistMult [53] | 0.276 | 18.16 | 31.15 | 46.96 | 0.287 | 19.33 | 32.19 | 47.54 | 0.107 | 4.52 | 10.33 | 21.25 |
| ComplEx [81] | 0.308 | 21.51 | 34.48 | 49.58 | 0.316 | 21.44 | 35.74 | 52.04 | 0.210 | 11.87 | 23.47 | 39.87 |
| TTransE [73] | 0.134 | 3.11 | 17.32 | 34.55 | 0.157 | 5.00 | 19.72 | 38.02 | 0.083 | 1.92 | 8.56 | 21.89 |
| TA-DistMult [75] | 0.264 | 17.09 | 30.22 | 45.41 | 0.243 | 14.58 | 27.92 | 44.21 | 0.167 | 8.61 | 18.41 | 33.59 |
| TA-TransE [75] | 0.174 | 0.00 | 29.19 | 47.41 | 0.193 | 1.81 | 31.34 | 50.33 | 0.125 | 0.01 | 17.92 | 37.38 |
| DE-SimplE [74] | 0.326 | 24.43 | 35.69 | 49.11 | 0.350 | 25.91 | 38.99 | 52.75 | 0.193 | 11.53 | 21.86 | 34.80 |
| TNTComplEx [77] | 0.321 | 23.35 | 36.03 | 49.13 | 0.275 | 19.52 | 30.80 | 42.86 | 0.212 | 13.28 | 24.02 | 36.91 |
| RE-Net [82] | 0.382 | 28.68 | 41.34 | 54.52 | 0.429 | 31.26 | 46.85 | 63.47 | 0.288 | 19.05 | 32.44 | 47.51 |
| CyGNet [83] | 0.327 | 23.69 | 36.31 | 50.67 | 0.349 | 25.67 | 39.09 | 52.94 | 0.249 | 15.90 | 28.28 | 42.61 |
| AnyBURL [63] | 0.296 | 21.26 | 33.33 | 46.73 | 0.320 | 23.72 | 35.45 | 50.46 | 0.227 | 15.10 | 25.44 | 38.91 |
| TLogic [28] | 0.430 | 33.56 | 48.27 | 61.23 | 0.469 | 36.21 | 53.13 | 67.43 | 0.298 | 20.54 | 33.95 | 48.53 |
| TR-Rules [29] | 0.433 | 33.96 | 48.55 | 61.17 | 0.476 | 37.06 | 53.80 | 67.57 | 0.304 | 21.10 | 34.58 | 48.92 |
| xERTE [84] | 0.407 | 32.70 | 45.67 | 57.30 | 0.466 | 37.84 | 52.31 | 63.92 | 0.293 | 21.03 | 33.51 | 46.48 |
| TITer [85] | 0.417 | 32.74 | 46.46 | 58.44 | β | β | β | β | 0.299 | 22.05 | 33.46 | 44.83 |
| TECHS [30] | 0.438 | 34.59 | 49.36 | 61.95 | 0.483 | 38.34 | 54.69 | 68.92 | 0.308 | 21.81 | 35.39 | 49.82 |
| INFER [86] | 0.441 | 34.52 | 48.92 | 62.14 | 0.483 | 37.61 | 54.30 | 68.52 | 0.317 | 21.94 | 35.64 | 50.88 |
| Tunsr | 0.447 | 35.16 | 50.39 | 63.32 | 0.491 | 38.31 | 55.67 | 69.88 | 0.321 | 22.99 | 36.68 | 51.08 |
The experiments on transductive, inductive, interpolation, and extrapolation reasoning are carried out to evaluate the performance. The results are shown in Tables II, III, V and VI, respectively. It can be observed that our model has performance advantages over neural, symbolic, and neurosymbolic methods.
Specifically, from Table II of transductive reasoning, it is observed that Tunsr achieves the optimal performance. Compared with advanced neural methods, Tunsr shows performance advantages. For example, it improves the Hits@10 values of the two datasets by 8.78%, 16.78%, 8.48%, 8.68%, 9.08%, 3.38%, 8.18%, 12.08% and 4.45%, 15.25%, 3.05%, 1.15%, 1.95%, 1.35%, 2.65%, 4.45% compared with TransE, DistMult, UltraE, ComplEx-DURA, AutoBLM, RED-GNN, CompoundE and GATH model. Moreover, compared with symbolic and neurosymbolic methods, the advantages of the Tunsr are more obvious. For symbolic methods (AMIE, AnyBURL, and SAFRAN), the average achievements of MRR, Hits@1, and Hits@10 values are 0.119, 9.79%, 11.51% and 0.075, 5.72%, 8.75% on two datasets. For advanced neurosymbolic RNNLogic, LatentLogic and RNN+RotE, Tunsr also shows performance advantages, achieving 9.98%, 10.48%, 2.28% and 4.15%, 5.75%, 4.25% of Hits@10 improvements on two datasets.
For inductive reasoning, Tunsr also has the performance advantage compared with all neural, symbolic, and neurosymbolic methods as Table III shows, especially on WN18RR v1, WN18RR v2, WN18RR v3, FB15k-237 v1, and NELL-995 v1 datasets. Specifically, Tunsr is better than neural methods GraIL, MLSAA, and RED-GNN. Compared with the latter, it achieves 5.97%, 5.98%, 8.36%, 1.18%, 7.66%, 0.34%, 1.13%, 1.18%, 1.96%, 2.04%, 1.65%, and 3.18% improvements on the His@10 metric, reaching an average improvement of 3.39%. For symbolic and neurosymbolic methods (RuleN, Neural LP, and DRUM), Tunsr has greater performance advantages. For example, compared with DRUM, Tunsr has achieved an average improvement of 0.069, 8.19%, and 6.05% on MRR, Hits@1, and Hits@10 metrics, respectively. Besides, for equal comparison with CoMPILE [70], TACT [71], ConGLR [26], and SymRITa [72], we carry out the evaluation under their setting which introduces 50 negative entities (rather than all entities) for ranking for each query. The results are shown in Table IV. These results also verify the superiority of our model.
<details>
<summary>extracted/6596839/fig/zhexian1.png Details</summary>
### Visual Description
## Line Chart: Hits@10 Values (%) vs. Training Epochs
### Overview
The image is a line chart comparing the performance of three different models ("Propositional", "FOL", and "Unified") over a range of training epochs. The y-axis represents "Hits@10 Values (%)", indicating the percentage of correct predictions within the top 10 results. The x-axis represents "Training Epochs", showing the progression of training.
### Components/Axes
* **Title:** Hits@10 Values (%) vs. Training Epochs (inferred)
* **X-axis:**
* Label: Training Epochs
* Scale: 1, 6, 11, 16, 21, 26, 31, 36, 41, 46
* **Y-axis:**
* Label: Hits@10 Values (%)
* Scale: 45, 50, 55, 60, 65
* **Legend:** Located at the bottom-center of the chart.
* Propositional (light purple line with circle markers)
* FOL (blue line with square markers)
* Unified (red line with star markers)
### Detailed Analysis
* **Propositional (light purple line with circle markers):**
* Trend: Generally increasing, with a slight plateau towards the end.
* Data Points:
* Epoch 1: ~48.5%
* Epoch 6: ~53.5%
* Epoch 11: ~57.5%
* Epoch 16: ~61.5%
* Epoch 21: ~62.5%
* Epoch 26: ~62.5%
* Epoch 31: ~64%
* Epoch 36: ~63.5%
* Epoch 41: ~63%
* Epoch 46: ~64%
* **FOL (blue line with square markers):**
* Trend: Increasing, with a plateau after epoch 31.
* Data Points:
* Epoch 1: ~46%
* Epoch 6: ~49.5%
* Epoch 11: ~51%
* Epoch 16: ~56.5%
* Epoch 21: ~58.5%
* Epoch 26: ~59%
* Epoch 31: ~61%
* Epoch 36: ~61.5%
* Epoch 41: ~62%
* Epoch 46: ~62%
* **Unified (red line with star markers):**
* Trend: Increasing sharply initially, then plateaus and slightly decreases.
* Data Points:
* Epoch 1: ~52.5%
* Epoch 6: ~57.5%
* Epoch 11: ~60.5%
* Epoch 16: ~62%
* Epoch 21: ~65%
* Epoch 26: ~66%
* Epoch 31: ~65.5%
* Epoch 36: ~65%
* Epoch 41: ~65.5%
* Epoch 46: ~66%
### Key Observations
* The "Unified" model initially performs the best, showing the most rapid increase in Hits@10 values.
* The "FOL" model shows the slowest initial increase but continues to improve over the training epochs.
* All three models plateau in performance after approximately 30 epochs.
### Interpretation
The chart demonstrates the learning curves of three different models during training. The "Unified" model appears to have the best initial learning rate, but its performance plateaus relatively early. The "FOL" model, while starting slower, shows a more consistent improvement over a longer period. The "Propositional" model falls in between. The plateauing of all models suggests that further training beyond 30 epochs may not significantly improve their performance, and other optimization strategies might be needed. The data suggests that the "Unified" model is a good starting point, but the "FOL" model might eventually catch up with more training.
</details>
(a) WN18RR of SKG T.
<details>
<summary>extracted/6596839/fig/zhexian2.png Details</summary>
### Visual Description
## Line Chart: Hits@10 Values (%) vs. Training Epochs
### Overview
The image is a line chart comparing the performance of three different models ("Propositional", "FOL", and "Unified") over a range of training epochs. The y-axis represents "Hits@10 Values (%)", indicating the percentage of correct predictions within the top 10 predictions. The x-axis represents "Training Epochs", showing the number of training iterations.
### Components/Axes
* **Title:** None explicitly present in the image.
* **X-axis:**
* Label: "Training Epochs"
* Scale: 1 to 46, with markers at intervals of 5 (1, 6, 11, 16, 21, 26, 31, 36, 41, 46).
* **Y-axis:**
* Label: "Hits@10 Values (%)"
* Scale: 55 to 80, with markers at intervals of 5 (55, 60, 65, 70, 75, 80).
* **Legend:** Located at the bottom of the chart.
* Propositional (light purple line with circle markers)
* FOL (blue line with square markers)
* Unified (red line with star markers)
### Detailed Analysis
* **Propositional (light purple line with circle markers):**
* Trend: Initially increases sharply, then plateaus and slightly decreases towards the end.
* Data Points:
* Epoch 1: ~58%
* Epoch 6: ~70%
* Epoch 11: ~73%
* Epoch 16: ~75%
* Epoch 21: ~76%
* Epoch 26: ~78%
* Epoch 31: ~77%
* Epoch 36: ~78%
* Epoch 41: ~79%
* Epoch 46: ~78%
* **FOL (blue line with square markers):**
* Trend: Increases steadily, plateaus, and remains relatively stable.
* Data Points:
* Epoch 1: ~59%
* Epoch 6: ~62%
* Epoch 11: ~69%
* Epoch 16: ~72%
* Epoch 21: ~74%
* Epoch 26: ~75.5%
* Epoch 31: ~75%
* Epoch 36: ~76%
* Epoch 41: ~75%
* Epoch 46: ~75.5%
* **Unified (red line with star markers):**
* Trend: Increases rapidly, plateaus, and remains relatively stable at the highest performance level.
* Data Points:
* Epoch 1: ~57.5%
* Epoch 6: ~67%
* Epoch 11: ~74%
* Epoch 16: ~77%
* Epoch 21: ~78%
* Epoch 26: ~79%
* Epoch 31: ~79%
* Epoch 36: ~79%
* Epoch 41: ~79.5%
* Epoch 46: ~79%
### Key Observations
* The "Unified" model consistently outperforms the "Propositional" and "FOL" models.
* All three models show significant performance gains in the initial training epochs (1-26).
* The "Propositional" model shows a slight decrease in performance towards the end of the training epochs.
* The "FOL" model plateaus earlier than the "Unified" model.
### Interpretation
The chart demonstrates the performance of three different models ("Propositional", "FOL", and "Unified") as a function of training epochs. The "Unified" model achieves the highest "Hits@10 Values (%)", suggesting it is the most effective model for this task. The initial rapid increase in performance for all models indicates that the models are learning effectively during the early training stages. The plateauing of performance suggests that the models are reaching their maximum potential with the given training data and architecture. The slight decrease in performance for the "Propositional" model towards the end could indicate overfitting or other issues. The "Unified" model's consistently high performance suggests it is the most robust and well-suited for the task.
</details>
(b) ICEWS14 of TKG I.
<details>
<summary>extracted/6596839/fig/zhexian3.png Details</summary>
### Visual Description
## Line Chart: Hits@10 Values vs. Training Epochs
### Overview
The image is a line chart comparing the performance of three different models (Propositional, FOL, and Unified) over 10 training epochs. The y-axis represents "Hits@10 Values (%)", and the x-axis represents "Training Epochs". The chart shows how the performance of each model changes as the training progresses.
### Components/Axes
* **X-axis:** Training Epochs, labeled from 1 to 10.
* **Y-axis:** Hits@10 Values (%), labeled from 40 to 70 with increments of 5.
* **Legend:** Located at the top of the chart, it identifies each line:
* Propositional (purple line with circle markers)
* FOL (blue line with square markers)
* Unified (red line with star markers)
### Detailed Analysis
* **Propositional (Purple):**
* Epoch 1: Approximately 49%
* Epoch 2: Peaks at approximately 53%
* Epoch 3: Drops to approximately 46%
* Epoch 4: Approximately 47%
* Epoch 5: Approximately 47%
* Epoch 6: Approximately 46%
* Epoch 7: Approximately 45.5%
* Epoch 8: Approximately 44%
* Epoch 9: Approximately 43.5%
* Epoch 10: Approximately 45.5%
* Trend: Starts at 49%, peaks at epoch 2, then generally declines until epoch 9, with a slight increase at epoch 10.
* **FOL (Blue):**
* Epoch 1: Approximately 59%
* Epoch 2: Approximately 60.5%
* Epoch 3: Approximately 60.5%
* Epoch 4: Approximately 62%
* Epoch 5: Approximately 60%
* Epoch 6: Approximately 60%
* Epoch 7: Approximately 61%
* Epoch 8: Approximately 61%
* Epoch 9: Approximately 60.5%
* Epoch 10: Approximately 60%
* Trend: Relatively stable performance, hovering around 60-62% throughout the training epochs.
* **Unified (Red):**
* Epoch 1: Approximately 61%
* Epoch 2: Approximately 62.5%
* Epoch 3: Approximately 63%
* Epoch 4: Approximately 63%
* Epoch 5: Approximately 62%
* Epoch 6: Approximately 61%
* Epoch 7: Approximately 63%
* Epoch 8: Approximately 62%
* Epoch 9: Approximately 61%
* Epoch 10: Approximately 62%
* Trend: Starts at 61%, peaks at epochs 3, 4, and 7, then generally declines slightly, remaining above 60%.
### Key Observations
* The Unified model consistently outperforms the FOL and Propositional models.
* The FOL model shows relatively stable performance across all training epochs.
* The Propositional model has the lowest performance and exhibits a decline after the second epoch.
### Interpretation
The chart suggests that the Unified model is the most effective among the three for this particular task, as it consistently achieves the highest "Hits@10 Values". The FOL model provides a stable but lower performance compared to the Unified model. The Propositional model's performance is significantly lower and decreases after the initial training epochs, indicating it may not be as suitable for this task or requires further optimization. The "Hits@10 Values" metric likely represents the percentage of times the correct answer is within the top 10 results, so higher values indicate better accuracy.
</details>
(c) ICEWS14 of TKG E.
<details>
<summary>extracted/6596839/fig/zhexian4.png Details</summary>
### Visual Description
## Line Chart: Hits@10 Values vs. Training Epochs
### Overview
The image is a line chart comparing the performance of three different models (Propositional, FOL, and Unified) over 10 training epochs. The y-axis represents "Hits@10 Values (%)", ranging from 30 to 60. The x-axis represents "Training Epochs", ranging from 1 to 10. Each model is represented by a different colored line with a distinct marker.
### Components/Axes
* **Title:** None explicitly present in the image.
* **X-axis:**
* Label: "Training Epochs"
* Scale: 1 to 10, incrementing by 1.
* **Y-axis:**
* Label: "Hits@10 Values (%)"
* Scale: 30 to 60, incrementing by 5.
* **Legend:** Located at the top of the chart.
* Propositional: Purple line with circle markers.
* FOL: Blue line with square markers.
* Unified: Red line with star markers.
### Detailed Analysis
* **Propositional (Purple, Circle):** The line starts at approximately 40% at epoch 1, increases steadily to approximately 46.5% by epoch 6, and then plateaus around 46% for the remaining epochs.
* Epoch 1: ~40%
* Epoch 2: ~41.5%
* Epoch 3: ~42.5%
* Epoch 4: ~43.5%
* Epoch 5: ~45.8%
* Epoch 6: ~46.8%
* Epoch 7: ~46.2%
* Epoch 8: ~46.2%
* Epoch 9: ~46.5%
* Epoch 10: ~46%
* **FOL (Blue, Square):** The line starts at approximately 49% at epoch 1, increases slightly to approximately 50.5% by epoch 3, then decreases to approximately 48% by epoch 6, and then increases slightly to approximately 50.5% by epoch 9, and then decreases slightly to approximately 50% by epoch 10.
* Epoch 1: ~49%
* Epoch 2: ~49.3%
* Epoch 3: ~50.5%
* Epoch 4: ~50%
* Epoch 5: ~49.5%
* Epoch 6: ~47.8%
* Epoch 7: ~48.5%
* Epoch 8: ~49%
* Epoch 9: ~50.5%
* Epoch 10: ~50%
* **Unified (Red, Star):** The line starts at approximately 51% at epoch 1, increases slightly to approximately 51.3% by epoch 3, then decreases to approximately 49.5% by epoch 5, then increases slightly to approximately 50.5% by epoch 7, and then plateaus around 50.5% for the remaining epochs.
* Epoch 1: ~51%
* Epoch 2: ~51%
* Epoch 3: ~51.3%
* Epoch 4: ~50.3%
* Epoch 5: ~49.5%
* Epoch 6: ~50.3%
* Epoch 7: ~50.5%
* Epoch 8: ~50.3%
* Epoch 9: ~50.5%
* Epoch 10: ~50.8%
### Key Observations
* The "Unified" model consistently outperforms the "FOL" and "Propositional" models across all training epochs.
* The "Propositional" model shows the most significant improvement in performance during the initial training epochs (1-6), but plateaus afterward.
* The "FOL" and "Unified" models exhibit relatively stable performance throughout the training period, with minor fluctuations.
### Interpretation
The chart suggests that the "Unified" model is the most effective of the three models, achieving the highest "Hits@10 Values (%)" consistently. The "Propositional" model benefits the most from initial training, but its performance plateaus, indicating a potential limitation in its learning capacity. The "FOL" model shows moderate and relatively stable performance. The data implies that the "Unified" model may have a more robust architecture or training methodology that allows it to achieve better results compared to the other two models. The "Hits@10 Values (%)" metric likely represents the percentage of times the correct answer is within the top 10 predictions made by the model.
</details>
(d) ICEWS18 of TKG E.
Figure 5: The impacts of propositional and FOL reasoning on transductive, interpolation, and extrapolation scenarios. It is generally observed that the unified model has a better performance compared with the single propositional or FOL setting, demonstrating the validity and rationality of the unified mechanism in Tunsr.
For interpolation reasoning in Table V, the performance of Tunsr surpasses that of mainstream neural reasoning methods. It achieves optimal results on seven out of eight metrics. Compared with TNTComplEx of the previous classic tensor-decomposition method, the improvement on each metric is 0.028, 4.21%, 3.61%, 4.16%, 0.035, 0.89%, 3.67%, and 2.95%, respectively. Moreover, compared with the state-of-the-art model TeAST that encodes temporal knowledge graph embeddings via the archimedean spiral timeline, Tunsr also has 0.011, 0.21%, 1.41%, 1.96%, 0.022, -0.51%, 1.47%, and 1.05% performance advantages (only slightly smaller on Hits@1 metric of ICEWS0515 dataset).
As Table VI shows for extrapolation reasoning, Tunsr also performs better. Compared with 10 neural reasoning methods, Tunsr has obvious performance advantages. For instance, it achieves 14.19%, 27.02%, and 14,17% Hits@10 improvement on three datasets against the tensor-decomposition method TNTComplEx. Additionally, Tunsr outperforms symbolic rule-based methods, i.e., AnyBURL, TLogic, and TR-Rules, achieving average improvements of 0.061, 5.57%, 7.01%, 6.94%, 0.069, 5.98%, 8.21%, 8.06%, 0.045, 4.08%, 5.36%, and 5.63% on all 12 evaluation metrics. Moreover, Tunsr excels three neurosymbolic methods (xERTE, TITer and INFER) across all datasets. Furthermore, compared with the previous study TECHS, Tunsr also has the performance boost, which shows 1.37%, 0.96%, and 1.26% Hits@10 metric gains.
In summary, the experimental results on four reasoning scenarios demonstrate the effectiveness and superiority of the proposed unified framework Tunsr. It shows the rationality of the unified mechanism at both the methodological and application perspectives and verifies the tremendous potential for future KG reasoning frameworks.
4.3 Ablation Studies (RQ2)
To explore the impacts of propositional and FOL parts on KG reasoning performance, we carry out ablation studies on transductive (WN18RR), interpolation (ICEWS14), and extrapolation (ICEWS14 and ICEWS18) scenarios in Figure 5. As inductive reasoning is entity-independent, we only conduct experiments using FOL reasoning for it. In each line chart, we depict the performance trends associated with propositional, FOL, and unified reasoning throughout the training epochs. In the propositional/FOL setting, we set $\lambda$ in the Eq. (14) as 0/1, indicating the model only uses propositional/FOL reasoning to get the answer. In the unified setting, the value of $\lambda$ is the dynamic learned by embeddings. From the figure, it is generally observed that the unified setting has a better performance compared with the single propositional or FOL setting. It is noteworthy that propositional and FOL display unique characteristics when applied to diverse datasets. For transductive and interpolation reasoning (Figures 5 and 5), the performance of propositional reasoning consistently surpasses that of FOL, despite both exhibiting continuous improvement throughout the training process. However, it is contrary to the results on the extrapolation scenario (Figures 5 and 5), where FOL reasoning has performance advantages. It is noted that propositional reasoning performs well in ICEWS18 while badly in ICEWS14 under the extrapolation setting. This may be caused by the structural differences between ICEWS14 and ICEWS18. Compared with ICEWS14, the graph structure of ICEWS18 is notably denser (8.94 vs. 16.19 in node degree). So propositional reasoning in ICEWS18 can capture more comprehensive pattern semantics and exhibit robust generalization in testing scenarios. These observations indicate that propositional and FOL reasoning emphasizes distinct aspects of knowledge. Thus, combining them allows for the synergistic exploitation of their respective strengths, resulting in an enhanced overall effect.
<details>
<summary>extracted/6596839/fig/bar1.png Details</summary>
### Visual Description
## Bar Chart: Performance Metrics vs. Number of Steps
### Overview
The image is a bar chart comparing the performance of a system across three metrics (MRR, Hits@1, Hits@10) using different numbers of steps (2, 4, 6, and 8). The chart displays the performance score on the y-axis, ranging from 0 to 0.7, for each metric and step combination.
### Components/Axes
* **X-axis:** Categorical axis with three categories: "MRR", "Hits@1", and "Hits@10".
* **Y-axis:** Numerical axis ranging from 0 to 0.7, with increments of 0.1.
* **Legend:** Located at the top of the chart, indicating the number of steps represented by each color:
* Teal: 2 steps
* Blue: 4 steps
* Lavender: 6 steps
* Red: 8 steps
### Detailed Analysis
**MRR (Mean Reciprocal Rank):**
* **2 steps (Teal):** Approximately 0.39
* **4 steps (Blue):** Approximately 0.49
* **6 steps (Lavender):** Approximately 0.55
* **8 steps (Red):** Approximately 0.56
**Hits@1:**
* **2 steps (Teal):** Approximately 0.37
* **4 steps (Blue):** Approximately 0.48
* **6 steps (Lavender):** Approximately 0.50
* **8 steps (Red):** Approximately 0.51
**Hits@10:**
* **2 steps (Teal):** Approximately 0.42
* **4 steps (Blue):** Approximately 0.59
* **6 steps (Lavender):** Approximately 0.65
* **8 steps (Red):** Approximately 0.66
### Key Observations
* For all three metrics, performance generally increases as the number of steps increases from 2 to 8.
* The most significant performance gains are observed between 2 and 4 steps.
* The performance difference between 6 and 8 steps is relatively small compared to the other increases.
* Hits@10 consistently shows the highest performance scores across all step values, followed by MRR and then Hits@1.
### Interpretation
The data suggests that increasing the number of steps generally improves the performance of the system, regardless of the metric used. However, the marginal benefit of increasing steps diminishes as the number of steps increases. This could indicate that there is a point of diminishing returns, where adding more steps does not significantly improve performance. The fact that Hits@10 consistently outperforms MRR and Hits@1 suggests that the system is better at retrieving relevant items within the top 10 results than at ranking the very first result correctly.
</details>
(a) WN18RR of SKG T.
<details>
<summary>extracted/6596839/fig/bar2.png Details</summary>
### Visual Description
## Bar Chart: Performance Metrics vs. Number of Steps
### Overview
The image is a bar chart comparing the performance of a model across three metrics (MRR, Hits@1, Hits@10) using different numbers of steps (2, 4, 6, and 8). The y-axis represents the value of the metric, ranging from 0 to 0.7. The x-axis represents the metric type. Each metric has four bars representing the different step values.
### Components/Axes
* **X-axis:** Categorical axis with three categories: "MRR", "Hits@1", and "Hits@10".
* **Y-axis:** Numerical axis ranging from 0 to 0.7, with increments of 0.1.
* **Legend:** Located at the top-left of the chart.
* Teal: "2 steps"
* Blue: "4 steps"
* Lavender: "6 steps"
* Red: "8 steps"
### Detailed Analysis
**MRR (Mean Reciprocal Rank)**
* **2 steps (Teal):** Approximately 0.35
* **4 steps (Blue):** Approximately 0.45
* **6 steps (Lavender):** Approximately 0.47
* **8 steps (Red):** Approximately 0.44
**Trend:** The MRR generally increases from 2 steps to 6 steps, then slightly decreases at 8 steps.
**Hits@1**
* **2 steps (Teal):** Approximately 0.30
* **4 steps (Blue):** Approximately 0.37
* **6 steps (Lavender):** Approximately 0.38
* **8 steps (Red):** Approximately 0.35
**Trend:** The Hits@1 increases from 2 steps to 6 steps, then decreases at 8 steps.
**Hits@10**
* **2 steps (Teal):** Approximately 0.44
* **4 steps (Blue):** Approximately 0.60
* **6 steps (Lavender):** Approximately 0.61
* **8 steps (Red):** Approximately 0.60
**Trend:** The Hits@10 increases significantly from 2 steps to 4 steps, then plateaus between 4 and 8 steps.
### Key Observations
* For all metrics, performance generally increases as the number of steps increases from 2 to 6.
* The performance at 8 steps is sometimes slightly lower than at 6 steps.
* Hits@10 shows the most significant improvement with increasing steps, especially between 2 and 4 steps.
### Interpretation
The chart suggests that increasing the number of steps generally improves the model's performance, but there might be a point of diminishing returns or even a slight decrease in performance beyond 6 steps. The Hits@10 metric appears to be the most sensitive to the number of steps, indicating that increasing steps significantly improves the model's ability to find relevant results within the top 10. The model performs best with 6 steps across all metrics.
</details>
(b) FB15k-237 v3 of SKG I.
<details>
<summary>extracted/6596839/fig/bar3.png Details</summary>
### Visual Description
## Bar Chart: Performance Metrics Comparison
### Overview
The image is a bar chart comparing the performance of a model across three metrics (MRR, Hits@1, Hits@10) using different numbers of steps (1, 2, 3, and 4). The y-axis represents the score for each metric, ranging from 0 to 0.8. The x-axis represents the three metrics being evaluated.
### Components/Axes
* **X-axis:** Categorical axis with three categories: MRR, Hits@1, and Hits@10.
* **Y-axis:** Numerical axis ranging from 0 to 0.8, with increments of 0.1.
* **Legend:** Located at the top-left of the chart, indicating the number of steps:
* Teal: 1 step
* Blue: 2 steps
* Purple: 3 steps
* Red: 4 steps
### Detailed Analysis
**MRR (Mean Reciprocal Rank):**
* 1 step (Teal): Approximately 0.49
* 2 steps (Blue): Approximately 0.64
* 3 steps (Purple): Approximately 0.65
* 4 steps (Red): Approximately 0.66
**Hits@1:**
* 1 step (Teal): Approximately 0.38
* 2 steps (Blue): Approximately 0.54
* 3 steps (Purple): Approximately 0.57
* 4 steps (Red): Approximately 0.57
**Hits@10:**
* 1 step (Teal): Approximately 0.71
* 2 steps (Blue): Approximately 0.79
* 3 steps (Purple): Approximately 0.80
* 4 steps (Red): Approximately 0.80
### Key Observations
* For all three metrics, performance generally increases as the number of steps increases from 1 to 2.
* The performance increase from 2 steps to 3 steps is smaller than the increase from 1 step to 2 steps.
* The performance increase from 3 steps to 4 steps is minimal, suggesting diminishing returns.
* Hits@10 consistently shows the highest scores compared to MRR and Hits@1 for all step values.
* Hits@1 consistently shows the lowest scores compared to MRR and Hits@10 for all step values.
### Interpretation
The bar chart illustrates the impact of varying the number of steps on the model's performance across different ranking metrics. The data suggests that increasing the number of steps generally improves performance, but the gains diminish beyond 2 steps. The Hits@10 metric, which measures whether the correct item is within the top 10 results, consistently outperforms MRR and Hits@1, indicating that the model is better at including the correct item in the broader top 10 list than at ranking it higher. The relatively low Hits@1 scores suggest that the model struggles to place the correct item at the very top of the ranking. The diminishing returns observed with increasing steps could be due to factors such as overfitting or the model reaching its inherent capacity.
</details>
(c) ICEWS14 of TKG I.
<details>
<summary>extracted/6596839/fig/bar4.png Details</summary>
### Visual Description
## Bar Chart: Performance Metrics vs. Steps
### Overview
The image is a bar chart comparing the performance of a model across different metrics (MRR, Hits@1, Hits@10) using varying numbers of steps (1, 2, 3, and 4). The y-axis represents the performance score, ranging from 0 to 0.7. The x-axis represents the metrics.
### Components/Axes
* **Y-axis:** Performance score, ranging from 0 to 0.7 with increments of 0.1.
* **X-axis:** Metrics: MRR, Hits@1, Hits@10.
* **Legend:** Located at the top of the chart.
* Teal: 1 step
* Blue: 2 steps
* Purple: 3 steps
* Red: 4 steps
### Detailed Analysis
**MRR (Mean Reciprocal Rank):**
* 1 step (Teal): ~0.39
* 2 steps (Blue): ~0.39
* 3 steps (Purple): ~0.45
* 4 steps (Red): ~0.44
**Hits@1:**
* 1 step (Teal): ~0.31
* 2 steps (Blue): ~0.30
* 3 steps (Purple): ~0.36
* 4 steps (Red): ~0.35
**Hits@10:**
* 1 step (Teal): ~0.59
* 2 steps (Blue): ~0.59
* 3 steps (Purple): ~0.64
* 4 steps (Red): ~0.62
### Key Observations
* For all metrics, performance generally increases as the number of steps increases from 1 to 3.
* The performance for 4 steps is slightly lower than 3 steps for all metrics.
* Hits@10 shows the highest performance scores compared to MRR and Hits@1.
* The difference in performance between 1 step and 2 steps is minimal for MRR and Hits@10.
### Interpretation
The chart suggests that increasing the number of steps generally improves the model's performance, but there's a point of diminishing returns or even a slight decrease when going from 3 steps to 4 steps. This could indicate that the model benefits from increased complexity up to a certain point, after which it may start to overfit or encounter other issues. The Hits@10 metric consistently outperforms MRR and Hits@1, indicating that the model is better at retrieving relevant items within the top 10 results than at ranking the very first result correctly.
</details>
(d) ICEWS14 of TKG E.
Figure 6: The impacts of reasoning iterations which correspond to the length of the reasoning rules. It is evident that choosing the appropriate value is crucial for obtaining accurate reasoning results.
<details>
<summary>extracted/6596839/fig/bar3d1.png Details</summary>
### Visual Description
## 3D Bar Chart: Hits@10 Values (%) vs. M and N
### Overview
The image is a 3D bar chart visualizing the relationship between two variables, M and N, and their impact on "Hits@10 Values (%)". The chart uses a color gradient to represent the magnitude of the "Hits@10 Values (%)", with blue indicating lower values and red indicating higher values. The chart shows a general trend of increasing "Hits@10 Values (%)" as both M and N increase.
### Components/Axes
* **X-axis (M):** Represents the variable M, with values ranging from 50 to 1000. The specific values marked are 50, 100, 200, 600, 800, and 1000.
* **Y-axis (N):** Represents the variable N, with values ranging from 40 to 140. The specific values marked are 40, 60, 80, 100, 120, and 140.
* **Z-axis (Hits@10 Values (%)):** Represents the "Hits@10 Values (%)", ranging from 48% to 64%. The specific values marked are 50%, 55%, and 60%.
* **Colorbar:** A vertical colorbar on the right side of the chart maps the color gradient to the "Hits@10 Values (%)". The colorbar ranges from 48 (dark blue) to 64 (dark red), with intermediate values marked at 50, 52, 54, 56, 58, 60, and 62.
### Detailed Analysis
The chart presents data points for each combination of M and N values. The height and color of each bar indicate the corresponding "Hits@10 Values (%)".
* **Trend:** As both M and N increase, the "Hits@10 Values (%)" generally increases. The bars transition from blue/green to yellow/orange/red, indicating higher values.
* **Specific Data Points (Approximate):**
* M=50, N=40: Hits@10 Values (%) β 49% (Dark Blue)
* M=1000, N=140: Hits@10 Values (%) β 63% (Dark Red)
* M=1000, N=40: Hits@10 Values (%) β 54% (Green/Yellow)
* M=50, N=140: Hits@10 Values (%) β 55% (Green/Yellow)
* **Color Mapping Verification:**
* Dark Blue corresponds to values around 48-50%.
* Green corresponds to values around 52-54%.
* Yellow corresponds to values around 56-58%.
* Orange corresponds to values around 60-62%.
* Red corresponds to values around 62-64%.
### Key Observations
* The highest "Hits@10 Values (%)" are achieved when both M and N are at their maximum values (M=1000, N=140).
* The lowest "Hits@10 Values (%)" are observed when both M and N are at their minimum values (M=50, N=40).
* The chart shows a clear positive correlation between M, N, and "Hits@10 Values (%)".
### Interpretation
The chart suggests that increasing both M and N leads to a higher "Hits@10 Values (%)". This indicates that these two variables have a significant positive impact on the performance metric being measured. The relationship appears to be roughly linear, with a consistent increase in "Hits@10 Values (%)" as M and N increase. The data implies that optimizing both M and N is crucial for maximizing the "Hits@10 Values (%)". Further analysis could involve determining the specific functional relationship between M, N, and "Hits@10 Values (%)" to identify the optimal values for M and N.
</details>
(a) Performance on ICEWS14.
<details>
<summary>extracted/6596839/fig/bar3d2.png Details</summary>
### Visual Description
## 3D Bar Chart: GPU Memory Usage vs. M and N
### Overview
The image is a 3D bar chart visualizing GPU memory usage (in GB) as a function of two variables, M and N. The height and color of each bar represent the GPU memory usage for a specific combination of M and N values. The color gradient ranges from blue (low memory usage) to red (high memory usage).
### Components/Axes
* **Vertical Axis (GPU memory):** Labeled "GPU memory (GB)", with a scale from 0 to 40 GB, incrementing by 5 GB.
* **Horizontal Axis 1 (M):** Labeled "M", with values 50, 100, 200, 600, 800, and 1000.
* **Horizontal Axis 2 (N):** Labeled "N", with values 40, 60, 80, 100, 120, and 140.
* **Colorbar:** Located on the right side of the chart, indicating the GPU memory usage corresponding to each color. The colorbar ranges from blue (0 GB) to red (40 GB), with intermediate colors representing values in between.
### Detailed Analysis
The chart displays GPU memory usage for various combinations of M and N. The bars are arranged in a grid, with M increasing along one axis and N increasing along the other. The height and color of each bar indicate the GPU memory usage for that specific (M, N) combination.
Here's a breakdown of the approximate GPU memory usage based on the color and height of the bars:
* **Low Memory Usage (Blue):** The bars in the bottom-left corner (low M and N values) are blue, indicating low GPU memory usage, approximately between 0 and 10 GB. For example, at M=50 and N=40, the GPU memory usage is approximately 5 GB.
* **Medium Memory Usage (Green/Yellow):** As M and N increase, the bars become green and yellow, indicating medium GPU memory usage, approximately between 10 and 30 GB. For example, at M=600 and N=100, the GPU memory usage is approximately 20 GB.
* **High Memory Usage (Red):** The bars in the top-right corner (high M and N values) are red, indicating high GPU memory usage, approximately between 30 and 40 GB. For example, at M=1000 and N=140, the GPU memory usage is approximately 40 GB.
**Specific Data Points (Approximate):**
* M=50, N=40: ~5 GB
* M=100, N=60: ~7 GB
* M=200, N=80: ~12 GB
* M=600, N=100: ~20 GB
* M=800, N=120: ~30 GB
* M=1000, N=140: ~40 GB
**Trends:**
* **Increasing M:** For a fixed N, as M increases, GPU memory usage tends to increase.
* **Increasing N:** For a fixed M, as N increases, GPU memory usage tends to increase.
* **Overall:** The highest GPU memory usage occurs when both M and N are at their maximum values.
### Key Observations
* GPU memory usage is highly dependent on both M and N.
* The relationship between M, N, and GPU memory usage appears to be non-linear.
* The chart suggests a strong correlation between increasing M and N and increasing GPU memory consumption.
### Interpretation
The chart demonstrates how the parameters M and N affect GPU memory usage. The data suggests that increasing either M or N leads to higher GPU memory consumption, with the highest usage observed when both parameters are maximized. This information is crucial for optimizing code and resource allocation when working with GPUs, as it allows developers to understand the memory footprint of their applications and adjust parameters accordingly to avoid exceeding GPU memory limits. The non-linear relationship suggests that the memory usage might scale exponentially or according to a polynomial function of M and N.
</details>
(b) Space on ICEWS14.
<details>
<summary>extracted/6596839/fig/bar3d3.png Details</summary>
### Visual Description
## 3D Bar Chart: Hits@10 Values (%) vs. M and N
### Overview
The image is a 3D bar chart visualizing the "Hits@10 Values (%)" as a function of two variables, M and N. The chart displays a grid of bars, where the height and color of each bar represent the "Hits@10 Values (%)" for specific combinations of M and N. The color gradient ranges from dark blue (low values) to dark red (high values), as indicated by the colorbar on the right.
### Components/Axes
* **Vertical Axis (Hits@10 Values (%)):** Ranges from approximately 38% to 52%.
* **M Axis:** Ranges from 50 to 1000.
* **N Axis:** Ranges from 40 to 140.
* **Colorbar:** A vertical colorbar on the right side of the chart maps the "Hits@10 Values (%)" to a color gradient. The colorbar ranges from dark blue (38) to dark red (52).
### Detailed Analysis
The chart displays a grid of bars, each representing a combination of M and N values. The height and color of each bar correspond to the "Hits@10 Values (%)" for that specific combination.
* **M Values:** 50, 100, 200, 600, 800, 1000
* **N Values:** 40, 60, 80, 100, 120, 140
**Data Points (Approximate):**
To accurately describe the data, I will describe the trend for each 'N' value as 'M' increases.
* **N = 40:** The "Hits@10 Values (%)" starts around 43% (blue-green) for M=50 and gradually increases to approximately 47% (orange) as M increases to 1000.
* **N = 60:** The "Hits@10 Values (%)" starts around 44% (green) for M=50 and gradually increases to approximately 48% (orange-red) as M increases to 1000.
* **N = 80:** The "Hits@10 Values (%)" starts around 45% (green-yellow) for M=50 and gradually increases to approximately 49% (red) as M increases to 1000.
* **N = 100:** The "Hits@10 Values (%)" starts around 46% (yellow) for M=50 and gradually increases to approximately 50% (red) as M increases to 1000.
* **N = 120:** The "Hits@10 Values (%)" starts around 45% (green-yellow) for M=50 and gradually increases to approximately 49% (red) as M increases to 1000.
* **N = 140:** The "Hits@10 Values (%)" starts around 43% (green) for M=50 and gradually increases to approximately 47% (orange) as M increases to 1000.
### Key Observations
* The "Hits@10 Values (%)" generally increases as both M and N increase, but the effect of M seems more pronounced.
* The highest "Hits@10 Values (%)" (around 50-52%) are observed when both M and N are at their highest values (M=1000, N=100).
* The lowest "Hits@10 Values (%)" (around 43%) are observed when both M and N are at their lowest values (M=50, N=40).
### Interpretation
The chart suggests that increasing both M and N leads to higher "Hits@10 Values (%)". The parameter M appears to have a more significant impact on the "Hits@10 Values (%)" than the parameter N. The optimal performance, as indicated by the highest "Hits@10 Values (%)", is achieved when both M and N are maximized. This could indicate that a larger sample size (M) and a higher number of features (N) contribute to better performance in the context of whatever this data represents.
</details>
(c) Performance on ICEWS18.
<details>
<summary>extracted/6596839/fig/bar3d4.png Details</summary>
### Visual Description
## 3D Bar Chart: GPU Memory vs. M and N
### Overview
The image is a 3D bar chart visualizing the relationship between GPU memory (in GB) and two variables, M and N. The height and color of each bar represent the GPU memory required for specific combinations of M and N. The color gradient ranges from blue (low memory) to red (high memory).
### Components/Axes
* **Vertical Axis (GPU memory):** Labeled "GPU memory (GB)", with a scale from 0 to 40 GB in increments of 10 GB.
* **Horizontal Axes:**
* **M Axis:** Ranges from 50 to 1000. The specific values shown are 50, 100, 200, 600, 800, and 1000.
* **N Axis:** Ranges from 40 to 140. The specific values shown are 40, 60, 80, 100, 120, and 140.
* **Colorbar:** Located on the right side of the chart, indicating the GPU memory (GB) corresponding to each color. The colorbar ranges from blue (0 GB) to red (40 GB), with intermediate colors representing values in between.
### Detailed Analysis
The chart displays GPU memory usage for different combinations of M and N. Each bar's height corresponds to the GPU memory required, and its color matches the colorbar scale.
Here's a breakdown of the data trends:
* **Trend with M:** As M increases (from 50 to 1000), the GPU memory generally increases for a given value of N.
* **Trend with N:** As N increases (from 40 to 140), the GPU memory generally increases for a given value of M.
* **Highest Memory Usage:** The highest GPU memory usage (close to 40 GB) occurs when both M and N are at their maximum values (M = 1000, N = 140). The bars in the top-right corner of the chart are red, indicating high memory usage.
* **Lowest Memory Usage:** The lowest GPU memory usage (close to 0 GB) occurs when both M and N are at their minimum values (M = 50, N = 40). The bars in the bottom-left corner of the chart are blue, indicating low memory usage.
Specific data points (approximate due to the 3D nature of the chart):
| M | N | GPU Memory (GB) | Color |
| :--- | :--- | :-------------- | :----------- |
| 50 | 40 | ~2 | Dark Blue |
| 1000 | 40 | ~12 | Light Blue |
| 50 | 140 | ~15 | Light Blue |
| 1000 | 140 | ~38-40 | Red |
| 100 | 60 | ~5 | Blue |
| 800 | 120 | ~35 | Orange-Red |
### Key Observations
* GPU memory usage is positively correlated with both M and N.
* The combination of high M and high N results in the highest GPU memory consumption.
* The combination of low M and low N results in the lowest GPU memory consumption.
* The relationship appears roughly linear, but a more precise analysis would require the exact numerical values.
### Interpretation
The chart demonstrates the impact of parameters M and N on GPU memory usage. It suggests that increasing either M or N will lead to higher memory requirements, with the most significant increase occurring when both parameters are maximized. This information is crucial for resource allocation and optimization when working with GPU-intensive applications. The data suggests that if M and N represent some form of data size or complexity, then the GPU memory scales accordingly. The chart provides a visual guide for estimating GPU memory needs based on the values of M and N.
</details>
(d) Space on ICEWS18.
Figure 7: The impacts of sampling in the reasoning process. Performance and GPU space usage with batch size 64. Large values of M and N can achieve excellent performance at the cost of increased space requirements.
4.4 Hyperparameter Analysis (RQ3)
We run our model with different hyperparameters to explore weight impacts in Figures 6 and 7. Specifically, Figure 6 illustrates the performance variation with different reasoning iterations, i.e., the length of the reasoning rules. At WN18RR and FB15k-237 v3 datasets of transductive and inductive settings, experiments on rule lengths of 2, 4, 6, and 8 are carried out as illustrated in Figures 6 and 6. It is observed that the performance generally improves with the iteration increasing from 2 to 6. When the rule length continues to increase, the inference performance changes little or decreases slightly. The same phenomenon can also be observed in Figures 6 and 6, which corresponds to interpolation and extrapolation reasoning on the ICEWS14 dataset. The rule length ranges from 1 to 4, where the model performance typically displays an initial improvement, followed by a tendency to stabilize or exhibit a marginal decline. This phenomenon occurs due to the heightened rule length, which amplifies the modeling capability while potentially introducing noise into the reasoning process. Therefore, an appropriate value of rule length (reasoning iteration) is significant for KG reasoning.
We also explore the impacts of hyperparameters M for node sampling and N for edge selection on ICEWS14 and ICEWS18 datasets of extrapolation reasoning. The results are shown in Figure 7. For each dataset, we show the reasoning performance (Hits@10 metric) and utilized space (GPU memory) in detail, with the M varies in {50, 100, 200, 600, 800, 1000} while N varies in {40, 60, 80, 100, 120, 140}. It is evident that opting for smaller values results in a significant decline in performance. This decline can be attributed to the inadequate number of nodes and edges, which respectively contribute to insufficient and unstable training. Furthermore, as M surpasses 120, the marginal gains become smaller or even lead to performance degradation. Additionally, when M and N are increased, the GPU memory utilization of the model experiences rapid growth, as depicted in Figure 7 and 7, with a particularly pronounced effect on M.
TABLE VII: Some reasoning cases in transductive, interpolation, and extrapolation scenarios, where both propositional reasoning and learned FOL rules are displayed. β ${-1}$ β denotes the reverse of a specific relation and textual descriptions of some relations are simplified. Values in orange rectangles represent propositional attentions and relations marked with red in FOL rules represent the target relation to be predicted.
| Propositional Reasoning | FOL Rules |
| --- | --- |
|
<details>
<summary>extracted/6596839/fig/case1.png Details</summary>
### Visual Description
## Semantic Network Diagram: Word Relationship Graph
### Overview
The image is a semantic network diagram illustrating relationships between words or concepts represented by nodes and edges. The nodes are labeled with numerical identifiers, and the edges are labeled with the type of relationship between the nodes. Each node also has a numerical value associated with it, displayed in a light orange box. Some nodes have a green checkmark or a red "X" next to them.
### Components/Axes
* **Nodes:** Represented by circles, either blue or brown. Each node is labeled with a numerical identifier (e.g., 00238867, 00239321).
* **Edges:** Represented by blue arrows, indicating the type of relationship between the connected nodes (e.g., verbGroup-1, derivationally RelatedForm, hypernym).
* **Values:** Each node has a numerical value associated with it, displayed in a light orange box (e.g., 0.21, 0.23, 0.13).
* **Validation Marks:** Some nodes have a green checkmark (β) or a red "X" next to them, possibly indicating validation or correctness.
* **Root Node:** A single blue node labeled "00238867" acts as the root of the network.
### Detailed Analysis or ### Content Details
**Root Node:**
* Node ID: 00238867 (Blue node)
**Branch 1 (Top Branch):**
* Edge 1: "verbGroup-1" connects 00238867 to node 00239321.
* Node ID: 00239321
* Value: 0.21
* Edge 2: "self" connects 00239321 to node 00239321.
* Node ID: 00239321
* Value: 0.23
* Edge 3: "self" connects 00239321 to node 00239321.
* Node ID: 00239321
* Value: 0.59
* Validation: Green checkmark (β)
**Branch 2 (Middle Branch):**
* Edge 1: "derivationally RelatedForm-1" connects 00238867 to node 13530408.
* Node ID: 13530408
* Value: 0.13
* Edge 2: "derivationally RelatedForm" connects 13530408 to node 13530408.
* Node ID: 13530408
* Value: 0.11
* Edge 3: "verbGroup-1" connects 13530408 to node 14712036.
* Node ID: 14712036
* Value: 0.10
* Validation: Red "X"
**Branch 3 (Center Branch):**
* Edge 1: "synsetDomain TopicOf" connects 00238867 to node 06084469.
* Node ID: 06084469
* Value: 0.09
* Edge 2: "synsetDomain TopicOf-1" connects 06084469 to node 00025728.
* Node ID: 00025728
* Value: 0.06
* Edge 3: "synsetDomain TopicOf" connects 00025728 to node 06084469.
* Node ID: 06084469
* Value: 0.05
* Validation: Red "X"
**Branch 4 (Bottom Branch):**
* Edge 1: "hypernym" connects 00238867 to node 00126264.
* Node ID: 00126264
* Value: 0.23
* Edge 2: "hypernym-1" connects 00126264 to node 00298896.
* Node ID: 00298896
* Value: 0.08
* Edge 3: "alsoSee" connects 00298896 to node 00407848.
* Node ID: 00407848
* Value: 0.07
* Validation: Red "X"
### Key Observations
* The network originates from a single blue node (00238867).
* The edges represent different types of relationships between the nodes.
* The numerical values associated with each node vary.
* Some nodes are marked with a green checkmark or a red "X", possibly indicating validation status.
* The node "00239321" appears multiple times in the top branch, connected by "self" relationships.
### Interpretation
The diagram represents a semantic network, likely used to visualize relationships between words or concepts in a knowledge base. The numerical values associated with each node could represent a measure of importance, relevance, or confidence. The green checkmark and red "X" likely indicate whether the relationships or nodes have been validated or are considered correct. The "self" relationships in the top branch suggest a recursive or self-referential connection within the network. The different types of edges (e.g., "verbGroup-1", "derivationally RelatedForm") indicate the specific nature of the relationships between the connected nodes. The root node "00238867" serves as the starting point for exploring the network and its connections. The presence of inverse relationships (e.g., "derivationally RelatedForm-1") suggests a bidirectional connection between some nodes.
</details>
| [1]
0.21 verbGroup -1 (X,Z) $β$ verbGroup (X,Z) [2]
0.32 verbGroup -1 (X,Y 1) $\land$ derivationallyRelatedForm (Y 1,Y 2) $\land$ derivationallyRelatedForm -1 (Y 2,Z) $β$ verbGroup (X,Z) [3]
0.07 derivationallyRelatedForm -1 (X,Y 1) $\land$ derivationallyRelatedForm -1 (Y 1,Y 2) $\land$ verbGroup -1 (Y 2,Z) $β$ verbGroup (X,Z) [4]
0.05 synsetDomainTopicOf (X,Y 1) $\land$ synsetDomainTopicOf -1 (Y 1,Y 2) $\land$ derivationallyRelatedForm (Y 2,Z) $β$ verbGroup (X,Z) [5]
0.18 hypernym (X,Y 1) $\land$ hypernym -1 (Y 1,Y 2) $\land$ alsoSee (Y 2,Z) $β$ verbGroup (X,Z) |
| Transductive reasoning: query (00238867, verbGroup, ?) in WN18RR | |
|
<details>
<summary>extracted/6596839/fig/case2.png Details</summary>
### Visual Description
## Diagram: Relationship Graph of Events in Ukraine
### Overview
The image is a directed graph illustrating relationships between entities and events related to Ukraine, primarily focusing on the period around 2014. Nodes represent entities (e.g., "Police (Ukraine)", "Protester (Ukraine)", "Party of Regions", "Arseniy Yatsenyuk", "John Kerry", "Benjamin Netanyahu", "Security Service of Ukraine") and events, each labeled with a date. Edges represent relationships between these entities/events, labeled with the type of relationship (e.g., "self", "repression", "obstruct", "consult", "make", "discussBy"). Numerical values are associated with each edge, possibly representing the strength or probability of the relationship. Dashed lines indicate a lower confidence or weaker relationship. Green checkmarks and red crosses are used to indicate positive and negative outcomes, respectively.
### Components/Axes
* **Nodes:** Represented by circles, each labeled with an entity or event and a date (YYYY-MM-DD). Node colors are either blue or brown.
* **Edges:** Represented by arrows, indicating the direction of the relationship. Edge labels describe the type of relationship. Edge colors are blue.
* **Edge Weights:** Numerical values (e.g., 0.32, 0.23, 0.25) placed near each edge, possibly indicating the strength or probability of the relationship.
* **Relationship Types:** "reduceRelations<sup>-1</sup>", "self", "repression<sup>-1</sup>", "obstruct Passage<sup>-1</sup>", "consult", "make AnAppeal<sup>-1</sup>", "make Statement", "discussBy Telephone<sup>-1</sup>"
* **Outcome Indicators:** Green checkmark (positive) and red cross (negative) symbols.
### Detailed Analysis or Content Details
* **Node 1 (Leftmost, Blue):** "Party of Regions"
* **Node 2 (Top-Left, Brown):** "Police (Ukraine): 2014-01-21", Edge weight: 0.32, Relationship: "reduceRelations<sup>-1</sup>" from "Party of Regions"
* **Node 3 (Mid-Left, Brown):** "Protester (Ukraine): 2014-01-13", Edge weight: 0.23, Relationship: "obstruct Passage<sup>-1</sup>" from "Party of Regions"
* **Node 4 (Bottom-Left, Brown):** "Arseniy Yatsenyuk: 2014-03-27", Edge weight: 0.19, Relationship: "consult" from "Party of Regions"
* **Node 5 (Top-Center, Brown):** "Police (Ukraine): 2014-01-21", Edge weight: 0.25, Relationship: "self" from "Police (Ukraine): 2014-01-21"
* **Node 6 (Mid-Center, Brown):** "Police (Ukraine): 2014-01-21", Edge weight: 0.40, Relationship: "repression" from "Police (Ukraine): 2014-01-21"
* **Node 7 (Top-Right, Brown):** "Police (Ukraine): 2014-01-21", Edge weight: 0.24, Relationship: "self"
* **Node 8 (Right-Center, Brown):** "Police (Ukraine): 2014-01-27", Edge weight: 0.39, Relationship: "make AnAppeal<sup>-1</sup>"
* **Node 9 (Bottom-Center, Brown):** "Police (Ukraine): 2014-01-12", Edge weight: 0.31, Relationship: "repression<sup>-1</sup>" from "Protester (Ukraine): 2014-02-18"
* **Node 10 (Bottom-Right, Brown):** "John Kerry: 2014-02-01", Edge weight: 0.12, Relationship: "consult<sup>-1</sup>" from "Arseniy Yatsenyuk: 2014-03-27"
* **Node 11 (Top-Far-Right, Brown):** "Security Service of Ukraine", Edge weight: 0.74, Relationship: Dashed line from "Police (Ukraine): 2014-01-21", Green Checkmark
* **Node 12 (Mid-Far-Right, Brown):** "Security Service of Ukraine: 2014-04-18", Edge weight: 0.05, Relationship: "make Statement" from "Police (Ukraine): 2014-01-12", Dashed line, Red Cross
* **Node 13 (Bottom-Far-Right, Brown):** "Benjamin Netanyahu", Edge weight: 0.10, Relationship: "discussBy Telephone<sup>-1</sup>" from "John Kerry: 2014-02-01", Dashed line
* **Node 14 (Second-From-Bottom-Right, Brown):** "Benjamin Netanyahu: 2014-03-19", Edge weight: 0.08, Dashed line, Red Cross
### Key Observations
* The "Party of Regions" node (blue) acts as a central point, with relationships to "Police (Ukraine)", "Protester (Ukraine)", and "Arseniy Yatsenyuk".
* The "Police (Ukraine)" entity appears multiple times with different dates, suggesting a series of events or actions.
* Dashed lines indicate weaker or less certain relationships, particularly those involving the "Security Service of Ukraine" and "Benjamin Netanyahu".
* The green checkmark associated with the relationship between "Police (Ukraine)" and "Security Service of Ukraine" suggests a positive outcome, while the red crosses suggest negative outcomes.
### Interpretation
The diagram depicts a network of relationships between various entities and events in Ukraine around 2014. It suggests that the "Party of Regions" played a central role, interacting with the police, protesters, and political figures like Arseniy Yatsenyuk. The relationships involving the "Security Service of Ukraine" and "Benjamin Netanyahu" appear less direct or less certain, as indicated by the dashed lines and lower edge weights. The green checkmark and red crosses suggest that some of these relationships led to positive outcomes, while others resulted in negative outcomes. The numerical values associated with each edge likely represent the strength or probability of these relationships, providing a quantitative measure of their significance. The diagram highlights the complex interplay of political actors and events during this period in Ukraine.
</details>
| [1]
0.46 reduceRelations -1 (X,Z) $:t_{1}$ $β$ makeAnAppeal (X,Z) $:t$ [2]
0.19 reduceRelations -1 (X,Y 1) $:t_{1}$ $\land$ repression (Y 1,Y 2) $:t_{2}$ $\land$ makeAnAppeal -1 (Y 2,Z) $:t_{3}$ $β$ makeAnAppeal (X,Z) $:t$ [3]
0.14 obstructPassage -1 (X,Y 1) $:t_{1}$ $\land$ repression -1 (Y 1,Y 2) $:t_{2}$ $\land$ makeStatement (Y 2,Z) $:t_{3}$ $β$ makeAnAppeal (X,Z) $:t$ [4]
0.12 consult (X,Y 1) $:t_{1}$ $\land$ consult -1 (Y 1,Y 2) $:t_{2}$ $\land$ discussByTelephone -1 (Y 2,Z) $:t_{3}$ $β$ makeAnAppeal (X,Z) $:t$ |
| Interpolation reasoning: query (Party of Regions, makeAnAppeal, ?, 2014-05-15) in ICEWS14 | |
|
<details>
<summary>extracted/6596839/fig/case3.png Details</summary>
### Visual Description
## Diagram: Entity Relationship Graph
### Overview
The image is a directed graph representing relationships between entities. The entities are represented by circles, and the relationships are represented by arrows. Each entity has a name and a date associated with it. Each relationship has a label and a numerical value associated with it. The graph starts with "Nasser Bourita" and branches out to other entities. Some paths end with a checkmark or an "X" mark.
### Components/Axes
* **Nodes:** Represent entities, labeled with a name (e.g., "Iran", "United Nations", "Donald Trump") and a date (e.g., "2018-05-02", "2018-07-31"). The starting node is "Nasser Bourita". Nodes are colored orange, except for the starting node, which is blue.
* **Edges:** Represent relationships between entities, labeled with a verb phrase (e.g., "accuse", "express IntentTo", "engageIn Cooperation"). Each edge has a numerical value associated with it, displayed in a light orange box (e.g., 0.26, 0.37, 0.43). Edges are solid blue lines, except for some final edges, which are dashed blue lines.
* **Checkmark/X Marks:** Some paths end with either a green checkmark or a green "X" mark.
### Detailed Analysis or Content Details
* **Starting Node:** Nasser Bourita (blue node)
* **First Level Entities:**
* Iran:2018-05-02 (0.26, accuse)
* Iran:2018-05-03 (0.37, makeVisit)
* Germany:2018-04-12 (0.42, hostVisit^-1)
* **Second Level Entities (connected to Iran:2018-05-02):**
* United Nations:2018-07-31 (0.43, express IntentTo)
* **Second Level Entities (connected to Iran:2018-05-03):**
* Morocco:2018-05-01 (0.34, make Statement)
* Russia:2018-05-14 (0.34, engageIn Cooperation)
* Iran:2018-05-03 (0.33, reject^-1)
* **Second Level Entities (connected to Germany:2018-04-12):**
* Russia:2018-07-06 (0.15, meetAtThird Location)
* **Third Level Entities (connected to United Nations:2018-07-31):**
* United Nations:2018-07-31 (0.48, self)
* **Third Level Entities (connected to Russia:2018-05-14):**
* United Nations:2018-08-02 (0.09, self makeOptimistic Comment)
* **Third Level Entities (connected to Iran:2018-05-03):**
* Donald Trump:2018-06-08 (0.08, makeOptimistic Comment)
* **Third Level Entities (connected to Russia:2018-07-06):**
* Police (Afghanistan):2018-08-22 (0.05, makeOptimistic Comment^-1)
* **Fourth Level Entities (connected to United Nations:2018-07-31):**
* Donald Trump (0.62, United Nations) - Ends with a green checkmark.
* **Fourth Level Entities (connected to United Nations:2018-08-02):**
* Donald Trump:2018-06-08 (0.08) - Ends with a green "X" mark.
* **Fourth Level Entities (connected to Police (Afghanistan):2018-08-22):**
* Police (Afghanistan) (0.06) - Ends with a green "X" mark.
### Key Observations
* The graph represents a network of relationships originating from "Nasser Bourita".
* The numerical values on the edges likely represent some kind of weight or strength of the relationship.
* The checkmark and "X" marks at the end of some paths suggest a positive or negative outcome or evaluation.
* The dates associated with the entities provide a temporal context for the relationships.
### Interpretation
The diagram appears to be a knowledge graph or a semantic network, illustrating relationships between entities involved in some kind of event or situation. The starting point, "Nasser Bourita," seems to be a central figure. The relationships are directed, indicating a flow of influence or action. The numerical values associated with the relationships could represent the strength or frequency of the connection. The checkmark and "X" marks likely indicate whether the path led to a successful or unsuccessful outcome, or perhaps a positive or negative sentiment. The dates provide a timeline of events. The graph suggests a complex network of interactions between countries, organizations, and individuals, with "Nasser Bourita" playing a key role. The "makeOptimistic Comment" and "makeOptimistic Comment^-1" labels suggest that some relationships involve commentary or opinions. The "self" label suggests a self-referential relationship. The "reject^-1" label suggests a rejection.
</details>
| [1]
0.14 accuse (X,Y) $:t_{1}$ $\land$ expressIntentTo (Y,Z) $:t_{2}$ $β$ makeVisit (X,Z) $:t$ [2]
0.09 makeVisit (X,Y 1) $:t_{1}$ $\land$ engageInCooperation (Y 1,Y 2) $:t_{2}$ $\land$ defyNormsLaw (Y 2,Z) $:t_{3}$ $β$ makeVisit (X,Z) $:t$ [3]
0.11 makeStatement (X,Y 1) $:t_{1}$ $\land$ reject -1 (Y 1,Y 2) $:t_{2}$ $\land$ makeOptimisticComment (Y 2,Z) $:t_{3}$ $β$ makeVisit (X,Z) $:t$ [4]
0.25 makeVisit (X,Y) $:t_{1}$ $\land$ makeOptimisticComment (Y,Z) $:t_{2}$ $β$ makeVisit (X,Z) $:t$ [5]
0.17 hostVisit -1 (X,Y 1) $:t_{1}$ $\land$ meetAtThirdLocation (Y 1,Y 2) $:t_{2}$ $\land$ makeOptimisticComment -1 (Y 2,Z) $:t_{3}$ $β$ makeVisit (X,Z) $:t$ |
| Extrapolation reasoning: query (Nasser Bourita, makeVisit, ?, 2018-09-28) in ICEWS18 | |
TABLE VIII: Some reasoning cases in inductive scenarios, where learned FOL rules are displayed. Relations marked with red represent the target relation to be predicted. β ${-1}$ β denotes the reverse of a specific relation and textual descriptions of some relations are simplified.
| Col1 |
| --- |
| [1] 0.41 memberMeronym (X,Y 1) $\land$ hasPart (Y 1,Y 2) $\land$ hasPart -1 (Y 2,Z) $β$ memberMeronym (X,Z) [2] 0.19 hasPart -1 (X,Y 1) $\land$ hypernym (Y 1,Y 2) $\land$ memberOfDomainUsage -1 (Y 2,Z) $β$ memberMeronym (X,Z) [3] 0.25 hypernym (X,Y 1) $\land$ hypernym -1 (Y 1,Y 2) $\land$ memberMeronym (Y 2,Z) $β$ memberMeronym (X,Z) [4] 0.17 hypernym (X,Y 1) $\land$ hypernym -1 (Y 1,Y 2) $\land$ hasPart (Y 2,Z) $β$ memberMeronym (X,Z) |
| Inductive reasoning: query (08174398, memberMeronym, ?) in WN18RR v3 |
| [1] 0.32 filmReleaseRegion (X,Y 1) $\land$ filmReleaseRegion -1 (Y 1,Y 2) $\land$ filmCountry (Y 2,Z) $β$ filmReleaseRegion (X,Z) [2] 0.10 distributorRelation -1 (X,Y 1) $\land$ nominatedFor (Y 1,Y 2) $\land$ filmReleaseRegion -1 (Y 2,Z) $β$ filmReleaseRegion (X,Z) [3] 0.19 filmReleaseRegion (X,Y 1) $\land$ exportedTo -1 (Y 1,Y 2) $\land$ locationCountry (Y 2,Z) $β$ filmReleaseRegion (X,Z) [4] 0.05 filmCountry (X,Y 1) $\land$ filmReleaseRegion -1 (Y 1,Y 2) $\land$ filmMusic (Y 2,Z) $β$ filmReleaseRegion (X,Z) |
| Inductive reasoning: query (/m/0j6b5, filmReleaseRegion, ?) in FB15k-237 v3 |
| [1] 0.46 collaboratesWith -1 (X,Z) $β$ collaboratesWith (X,Z) [2] 0.38 collaboratesWith -1 (X,Y 1) $\land$ holdsOffice (Y 1,Y 2) $\land$ holdsOffice -1 (Y 2,Z) $β$ collaboratesWith (X,Z) [3] 0.03 collaboratesWith -1 (X,Y 1) $\land$ graduatedFrom (Y 1,Y 2) $\land$ graduatedFrom -1 (Y 2,Z) $β$ collaboratesWith (X,Z) [4] 0.03 collaboratesWith -1 (X,Y 1) $\land$ collaboratesWith (Y 1,Y 2) $\land$ graduatedFrom (Y 2,Z) $β$ collaboratesWith (X,Z) |
| Inductive reasoning: query (Hillary Clinton, collaboratesWith, ?) in NELL v3 |
4.5 Case Studies (RQ4)
To show the actual reasoning process of Tunsr, some practical cases are presented in detail on all four reasoning scenarios, which illustrate the transparency and interpretability of the proposed Tunsr. For better presentation, the maximum length of the reasoning iterations is set to 3. Specifically, Table VII shows the reasoning graphs for three specific queries on transductive, interpolation, and extrapolation scenarios, respectively, The propositional attention weights of nodes are listed near them, which represent the propositional reasoning score of each node at the current step. For example, in the first case, the uppermost propositional reasoning path (00238867, verbGroup -1, 00239321) at first step learns a large attention score for the correct answer 00239321. Generally, nodes with more preceding neighbors or larger preceding attention weights significantly impact subsequent steps and the prediction of final entity scores. Besides, we observe that propositional and first-order reasoning have an incompletely consistent effect. For example, the FOL rules of β[3]β and β[4]β in the third case have relatively high rule confidence values compared with β[1]β and β[2]β (0.11, 0.25 vs. 0.14, 0.09), but the combination of their corresponding propositional reasoning paths β(Nasser Bourita, makeStatement, Morocco:2018-05-01, reject -1, Iran:2018-05-03, makeOptimisticComment, Donald Trump:2018-06-08)β and β(Nasser Bourita, makeVisit, Iran:2018-05-03, self, Iran:2018-05-03, makeOptimisticComment, Donald Trump:2018-06-08)β has a small propositional attention, i.e., 0.08. This combination prevents the model from predicting the wrong answer Donald Trump. Thus, propositional and FOL reasoning can be integrated to jointly guide the reasoning process, leading to more accurate reasoning results.
Table VIII shows some learned FOL rules of inductive reasoning on WN18RR v3, FB15k-237 v3, and NELL v3 datasets. As the inductive setting is entity-independent, so the propositional reasoning part is not involved here. Each rule presented carries practical significance and is readily understandable for humans. For instance, rule β[1]β collaboratesWith -1 (X, Z) $β$ collaboratesWith(X, Z) in the third case has a relatively high confidence value (0.46). This aligns with human commonsense cognition, as the relation collaboratesWith has mutual characteristics for subject and object and can be derived from each other. If person a has collaborated with person b, it inherently implies person b has collaborated with person a. These results illustrate the effectiveness of the rules learned by Tunsr and its interpretable reasoning process.
5 Conclusion and Future Works
To combine the advantages of connectionism and symbolicism of AI for KG reasoning, we propose a unified neurosymbolic framework Tunsr for both perspectives of methodology and reasoning scenarios, including transductive, inductive, interpolation, and extrapolation reasoning. Tunsr first introduces a consistent structure of reasoning graph that starts from the query entity and constantly expands subsequent nodes by iteratively searching posterior neighbors. Based on it, a forward logical message-passing mechanism is proposed to update both the propositional representations and attentions, as well as FOL representations and attentions of each node in the expanding reasoning graph. In this way, Tunsr conducts the transformation of merging multiple rules by merging possible relations at each step by using FOL attentions. Through gradually adding rule bodies and updating rule confidence, the real FOL rules can be easily induced to constantly perform attention calculation over the reasoning graph, which is summarized as the FARI algorithm. The experiments on 19 datasets of four different reasoning scenarios illustrate the effectiveness of Tunsr. Meanwhile, the ablation studies show that propositional and FOL have different impacts. Thus, they can be integrated to improve the whole reasoning results. The case studies also verify the transparency and interpretability of its computation process.
The future works lie in two folds. Firstly, we aim to extend the application of this idea to various reasoning domains, particularly those necessitating interpretability for decision-making [87], such as intelligent healthcare and finance. We anticipate this will enhance the accuracy of reasoning while simultaneously offering human-understandable logical rules as evidence. Secondly, we intend to integrate the concept of unified reasoning with state-of-the-art technologies to achieve optimal results. For instance, large language models have achieved great success in the community of natural language processing and AI, while they often encounter challenges when confronted with complex reasoning tasks [88]. Hence, there is considerable prospect for large language models to enhance reasoning capabilities.
References
- [1] I. Tiddi and S. Schlobach. Knowledge graphs as tools for explainable machine learning: A survey. Artificial Intelligence, 302:103627, 2022.
- [2] M. Li and M. Moens. Dynamic key-value memory enhanced multi-step graph reasoning for knowledge-based visual question answering. In Thirty-Sixth AAAI Conference on Artificial Intelligence, pp. 10983β10992. AAAI Press, 2022.
- [3] C. Mavromatis et al. Tempoqr: Temporal question reasoning over knowledge graphs. In Thirty-Sixth AAAI Conference on Artificial Intelligence, pp. 5825β5833. AAAI Press, 2022.
- [4] Y. Yang et al. Knowledge graph contrastive learning for recommendation. In The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 1434β1443. ACM, 2022.
- [5] Y. Zhu et al. Recommending learning objects through attentive heterogeneous graph convolution and operation-aware neural network. IEEE Transactions on Knowledge and Data Engineering (TKDE), 35:4178β4189, 2023.
- [6] A. Bastos et al. RECON: relation extraction using knowledge graph context in a graph neural network. In The Web Conference (WWW), pp. 1673β1685. ACM / IW3C2, 2021.
- [7] X. Chen et al. Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In The Web Conference (WWW), pp. 2778β2788. ACM, 2022.
- [8] B. D. Trisedya et al. GCP: graph encoder with content-planning for sentence generation from knowledge bases. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(11):7521β7533, 2022.
- [9] W. Yu et al. A survey of knowledge-enhanced text generation. ACM Comput. Surv., 54(11s):227:1β227:38, 2022.
- [10] K. D. Bollacker et al. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the International Conference on Management of Data (SIGMOD), pp. 1247β1250, 2008.
- [11] D. Vrandecic. Wikidata: A new platform for collaborative data collection. In Proceedings of the 21st World Wide Web Conference (WWW), pp. 1063β1064, 2012.
- [12] Q. Wang et al. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering (TKDE), 29(12):2724β2743, 2017.
- [13] A. Rossi et al. Knowledge graph embedding for link prediction: A comparative analysis. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(2):1β49, 2021.
- [14] S. Pinker and J. Mehler. Connections and symbols. Mit Press, 1988.
- [15] T. H. Trinh et al. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476β482, 2024.
- [16] Q. Lin et al. Contrastive graph representations for logical formulas embedding. IEEE Transactions on Knowledge and Data Engineering, 35:3563β3574, 2023.
- [17] F. Xu et al. Symbol-llm: Towards foundational symbol-centric interface for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 13091β13116, 2024.
- [18] Q. Lin et al. Fusing topology contexts and logical rules in language models for knowledge graph completion. Information Fusion, 90:253β264, 2023.
- [19] A. Bordes et al. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems (NeurIPS), pp. 2787β2795, 2013.
- [20] L. A. GalΓ‘rraga et al. AMIE: association rule mining under incomplete evidence in ontological knowledge bases. In 22nd International World Wide Web Conference (WWW), pp. 413β422, 2013.
- [21] F. Yang et al. Differentiable learning of logical rules for knowledge base reasoning. In Advances in Neural Information Processing Systems (NeurIPS), pp. 2319β2328, 2017.
- [22] Y. Shen et al. Modeling relation paths for knowledge graph completion. IEEE Transactions on Knowledge and Data Engineering, 33(11):3607β3617, 2020.
- [23] K. Cheng et al. Rlogic: Recursive logical rule learning from knowledge graphs. In The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 179β189. ACM, 2022.
- [24] J. Liu et al. Latentlogic: Learning logic rules in latent space over knowledge graphs. In Findings of the EMNLP, pp. 4578β4586, 2023.
- [25] C. Jiang et al. Path spuriousness-aware reinforcement learning for multi-hop knowledge graph reasoning. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 3173β3184, 2023.
- [26] Q. Lin et al. Incorporating context graph with logical reasoning for inductive relation prediction. In The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 893β903, 2022.
- [27] J. Li et al. Teast: Temporal knowledge graph embedding via archimedean spiral timeline. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pp. 15460β15474, 2023.
- [28] Y. Liu et al. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In Thirty-Sixth AAAI Conference on Artificial Intelligence, pp. 4120β4127. AAAI Press, 2022.
- [29] N. Li et al. Tr-rules: Rule-based model for link forecasting on temporal knowledge graph considering temporal redundancy. In Findings of the Association for Computational Linguistics (EMNLP), pp. 7885β7894, 2023.
- [30] Q. Lin et al. TECHS: temporal logical graph networks for explainable extrapolation reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1281β1293, 2023.
- [31] E. Cambria et al. SenticNet 7: A commonsense-based neurosymbolic AI framework for explainable sentiment analysis. In LREC, pp. 3829β3839, 2022.
- [32] A. Sadeghian et al. DRUM: end-to-end differentiable rule mining on knowledge graphs. In Advances in Neural Information Processing Systems (NeurIPS), pp. 15321β15331, 2019.
- [33] M. Qu et al. Rnnlogic: Learning logic rules for reasoning on knowledge graphs. In 9th International Conference on Learning Representations (ICLR), 2021.
- [34] Y. Zhang et al. GMH: A general multi-hop reasoning model for KG completion. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3437β3446, 2021.
- [35] J. Zhang et al. Subgraph retrieval enhanced model for multi-hop knowledge base question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 5773β5784, 2022.
- [36] Y. Lan et al. Complex knowledge base question answering: A survey. IEEE Trans. Knowl. Data Eng., 35(11):11196β11215, 2023.
- [37] H. Dong et al. Temporal inductive path neural network for temporal knowledge graph reasoning. Artificial Intelligence, pp. 104085, 2024.
- [38] J. Chung et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.
- [39] S. Abiteboul et al. Foundations of databases, volume 8. Addison-Wesley Reading, 1995.
- [40] M. Gebser et al. Potassco: The potsdam answer set solving collection. Ai Communications, 24(2):107β124, 2011.
- [41] M. Alviano et al. Wasp: A native asp solver based on constraint learning. In Logic Programming and Nonmonotonic Reasoning: 12th International Conference, LPNMR 2013, Corunna, Spain, September 15-19, 2013. Proceedings 12, pp. 54β66. Springer, 2013.
- [42] W. Rautenberg. A Concise Introduction to Mathematical Logic. Springer, 2006.
- [43] G. Ciravegna et al. Logic explained networks. Artificial Intelligence, 314:103822, 2023.
- [44] H. Ren and J. Leskovec. Beta embeddings for multi-hop logical reasoning in knowledge graphs. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
- [45] P. B. Andrews. An Introduction to Mathematical Logic and Type Theory: To Truth Through Proof, volume 27. Springer Science & Business Media, 2013.
- [46] J. Sun et al. A survey of reasoning with foundation models. arXiv preprint arXiv:2312.11562, 2023.
- [47] W. Zhang et al. Knowledge graph reasoning with logics and embeddings: Survey and perspective. CoRR, abs/2202.07412, 2022.
- [48] D. Poole. Probabilistic horn abduction and bayesian networks. Artificial Intelligence, 64(1):81β129, 1993.
- [49] D. Xu et al. Inductive representation learning on temporal graphs. In 8th International Conference on Learning Representations (ICLR), 2020.
- [50] L. GalΓ‘rraga et al. Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal, 24(6):707β730, 2015.
- [51] W. Zhang et al. Iteratively learning embeddings and rules for knowledge graph reasoning. In The World Wide Web Conference (WWW), pp. 2366β2377, 2019.
- [52] T. Lacroix et al. Canonical tensor decomposition for knowledge base completion. In Proceedings of the 35th International Conference on Machine Learning (ICML), volume 80, pp. 2869β2878. PMLR, 2018.
- [53] B. Yang et al. Embedding entities and relations for learning and inference in knowledge bases. In International Conference on Learning Representations (ICLR), 2015.
- [54] B. Xiong et al. Ultrahyperbolic knowledge graph embeddings. In The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 2130β2139. ACM, 2022.
- [55] J. Wang et al. Duality-induced regularizer for semantic matching knowledge graph embeddings. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):1652β1667, 2023.
- [56] Y. Zhang et al. Bilinear scoring function search for knowledge graph learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):1458β1473, 2023.
- [57] R. Li et al. How does knowledge graph embedding extrapolate to unseen data: A semantic evidence view. In Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI), pp. 5781β5791. AAAI Press, 2022.
- [58] Y. Zhang and Q. Yao. Knowledge graph reasoning with relational digraph. In The ACM Web Conference, pp. 912β924. ACM, 2022.
- [59] X. Ge et al. Compounding geometric operations for knowledge graph completion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pp. 6947β6965, 2023.
- [60] W. Wei et al. Enhancing heterogeneous knowledge graph completion with a novel gat-based approach. ACM Transactions on Knowledge Discovery from Data, 2024.
- [61] F. Shi et al. Tgformer: A graph transformer framework for knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering, 2025.
- [62] L. A. GalΓ‘rraga et al. Amie: association rule mining under incomplete evidence in ontological knowledge bases. In Proceedings of the 22nd international conference on World Wide Web, pp. 413β422, 2013.
- [63] C. Meilicke et al. Anytime bottom-up rule learning for knowledge graph completion. In IJCAI, pp. 3137β3143, 2019.
- [64] S. Ott et al. SAFRAN: an interpretable, rule-based link prediction method outperforming embedding models. In 3rd Conference on Automated Knowledge Base Construction (AKBC), 2021.
- [65] A. Nandi et al. Simple augmentations of logical rules for neuro-symbolic knowledge graph completion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pp. 256β269, 2023.
- [66] J. Guo et al. A unified joint approach with topological context learning and rule augmentation for knowledge graph completion. In Findings of the Association for Computational Linguistics, pp. 13686β13696, 2024.
- [67] K. Teru et al. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning, pp. 9448β9457, 2020.
- [68] K. Sun et al. Incorporating multi-level sampling with adaptive aggregation for inductive knowledge graph completion. ACM Transactions on Knowledge Discovery from Data, 2024.
- [69] C. Meilicke et al. Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion. In 17th International Semantic Web Conference, pp. 3β20, 2018.
- [70] S. Mai et al. Communicative message passing for inductive relation reasoning. In Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 4294β4302, 2021.
- [71] J. Chen et al. Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 6271β6278, 2021.
- [72] Y. Pan et al. A symbolic rule integration framework with logic transformer for inductive relation prediction. In Proceedings of the ACM Web Conference, pp. 2181β2192, 2024.
- [73] J. Leblay and M. W. Chekol. Deriving validity time in knowledge graph. In Companion of the The Web Conference (WWW), pp. 1771β1776. ACM, 2018.
- [74] R. Goel et al. Diachronic embedding for temporal knowledge graph completion. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, pp. 3988β3995, 2020.
- [75] A. GarcΓa-DurΓ‘n et al. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4816β4821, 2018.
- [76] A. Sadeghian et al. Chronor: Rotation based temporal knowledge graph embedding. In Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 6471β6479, 2021.
- [77] T. Lacroix et al. Tensor decompositions for temporal knowledge base completion. In 8th International Conference on Learning Representations (ICLR), 2020.
- [78] C. Xu et al. Temporal knowledge graph completion using a linear temporal regularizer and multivector embeddings. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp. 2569β2578, 2021.
- [79] J. Messner et al. Temporal knowledge graph completion using box embeddings. In Thirty-Sixth AAAI Conference on Artificial Intelligence, pp. 7779β7787, 2022.
- [80] K. Chen et al. Rotateqvs: Representing temporal information as rotations in quaternion vector space for temporal knowledge graph completion. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 5843β5857, 2022.
- [81] T. Trouillon et al. Complex embeddings for simple link prediction. In International Conference on Machine Learning (ICML), volume 48, pp. 2071β2080, 2016.
- [82] W. Jin et al. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6669β6683, 2020.
- [83] C. Zhu et al. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In Thirty-Fifth AAAI Conference on Artificial Intelligence, pp. 4732β4740, 2021.
- [84] Z. Han et al. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In 9th International Conference on Learning Representations (ICLR), 2021.
- [85] H. Sun et al. Timetraveler: Reinforcement learning for temporal knowledge graph forecasting. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8306β8319, 2021.
- [86] N. Li et al. Infer: A neural-symbolic model for extrapolation reasoning on temporal knowledge graph. In The Thirteenth International Conference on Learning Representations (ICLR), 2025.
- [87] E. Cambria et al. Seven pillars for the future of artificial intelligence. IEEE Intelligent Systems, 38(6):62β69, 2023.
- [88] F. Xu et al. Are large language models really good logical reasoners? a comprehensive evaluation and beyond. IEEE Transactions on Knowledge and Data Engineering, 2025.