2402.10767v2

# Inference to the Best Explanation in Large Language Models **Authors**: Email: d.dalal1@universityofgalway.ie ## Abstract While Large Language Models (LLMs) have found success in real-world applications, their underlying explanatory process is still poorly understood. This paper proposes IBE-Eval, a framework inspired by philosophical accounts on Inference to the Best Explanation (IBE) to advance the interpretation and evaluation of LLM explanations. IBE-Eval estimates the plausibility of natural language explanations through a combination of explicit logical and linguistic features including: consistency, parsimony, coherence, and uncertainty. Extensive experiments are conducted on Causal Question Answering (CQA), where IBE-Eval is tasked to select the most plausible causal explanation amongst competing ones generated by the LLM (e.g. GPT 3.5 or LLaMA 2). The experiments reveal that IBE-Eval can successfully identify the best explanation with up to 77% accuracy ( $\approx 27\$ above random), improving upon a GPT 3.5-as-a-judge baseline ( $\approx+17\$ ) while being intrinsically more efficient and interpretable. Additional analysis suggests that, despite LLM-specific variances, generated explanations tend to conform to IBE criteria and that IBE-Eval is significantly correlated with human judgment, opening up opportunities for future development of automated explanation verification tools. <details> <summary>x1.png Details</summary>  ### Visual Description ## Flowchart: Causal Reasoning and Explanation Evaluation ### Overview The image depicts a structured reasoning framework for evaluating causal explanations. It consists of four interconnected sections: 1. **Causal Question** (top-left) 2. **Competing Hypotheses** (bottom-left) 3. **Competing Explanations** (center-right) 4. **Inference to the Best Explanation (IBE)** (rightmost) The flowchart uses color-coded boxes, dashed arrows, and textual annotations to represent logical relationships and evaluation criteria. --- ### Components/Axes #### Causal Question Section - **Text Box**: - Label: "Causal Question" - Content: - Premise: "The balloon expanded." - Question: "What was the cause?" - Options: - A) "I blew into it." (highlighted green) - B) "I pricked it." (highlighted red) #### Competing Hypotheses Section - **Two Hypothesis Boxes**: 1. **Green Box (Premise 1)**: - Premise: "I blew into the balloon." - Conclusion: "The balloon expanded." 2. **Red Box (Premise 2)**: - Premise: "I pricked the balloon." - Conclusion: "The balloon expanded." - **Explanation Prompt**: - Text instructing step-by-step explanation generation for each hypothesis. #### Competing Explanations Section - **Two Explanation Boxes**: 1. **Green Box (E1)**: - **Step 1**: "IF someone blows into a balloon, THEN it can cause the balloon to inflate." - Assumption: Blowing air increases internal air volume. - **Step 2**: "IF the balloon inflates, THEN it can cause the balloon to expand." - Assumption: Inflation stretches the balloon. - Final Statement: "Therefore, since I blew into the balloon, it caused the balloon to inflate, which resulted in its expansion." 2. **Red Box (E2)**: - **Step 1**: "IF a balloon is pricked, THEN the balloon may deflate." - Assumption: Pricking causes air loss. - **Step 2**: "IF a balloon deflates, THEN there is a decrease in air pressure inside the balloon." - Assumption: Deflation reduces internal pressure. - **Step 3**: "IF there is a decrease in air pressure inside the balloon, THEN the external air pressure will cause the balloon to expand." - Assumption: Lower internal pressure allows external pressure to expand the balloon. - Final Statement: "Therefore, since the balloon was pricked, it may have deflated, resulting in a decrease in air pressure inside the balloon, causing the external air pressure to make the balloon expand." #### Inference to the Best Explanation (IBE) Section - **Selection Criteria**: - **Consistency**: 1.0 (E1) vs. 1.0 (E2) - **Parsimony**: -2.0 (E1) vs. -3.0 (E2) - **Coherence**: 0.51 (E1) vs. 0.28 (E2) - **Uncertainty**: 2.0 (E1) vs. 3.0 (E2) - **IBE Output**: - Selected Explanation: E1 (marked with a green checkmark). --- ### Detailed Analysis #### Causal Question Section - **Textual Content**: - Premise: "The balloon expanded." - Question: "What was the cause?" - Options: - A) "I blew into it." (highlighted green) - B) "I pricked it." (highlighted red) #### Competing Hypotheses Section - **Premise 1 (Green Box)**: - Logical Structure: - Premise → Conclusion: "I blew into the balloon" → "The balloon expanded." - **Premise 2 (Red Box)**: - Logical Structure: - Premise → Conclusion: "I pricked the balloon" → "The balloon expanded." #### Competing Explanations Section - **E1 (Green Box)**: - **Step 1**: - Conditional: "IF someone blows into a balloon, THEN it can cause the balloon to inflate." - Assumption: Blowing air increases internal air volume. - **Step 2**: - Conditional: "IF the balloon inflates, THEN it can cause the balloon to expand." - Assumption: Inflation stretches the balloon. - **Final Statement**: Links Premise 1 to the conclusion via inflation. - **E2 (Red Box)**: - **Step 1**: - Conditional: "IF a balloon is pricked, THEN the balloon may deflate." - Assumption: Pricking causes air loss. - **Step 2**: - Conditional: "IF a balloon deflates, THEN there is a decrease in air pressure inside the balloon." - Assumption: Deflation reduces internal pressure. - **Step 3**: - Conditional: "IF there is a decrease in air pressure inside the balloon, THEN the external air pressure will cause the balloon to expand." - Assumption: Lower internal pressure allows external pressure to expand the balloon. - **Final Statement**: Links Premise 2 to the conclusion via deflation and pressure dynamics. #### Inference to the Best Explanation (IBE) Section - **Selection Criteria**: - **Consistency**: Both E1 and E2 score 1.0 (fully consistent with the conclusion). - **Parsimony**: E1 (-2.0) is simpler than E2 (-3.0). - **Coherence**: E1 (0.51) is more logically coherent than E2 (0.28). - **Uncertainty**: E1 (2.0) has lower uncertainty than E2 (3.0). - **IBE Output**: - E1 is selected as the best explanation due to higher parsimony, coherence, and lower uncertainty. --- ### Key Observations 1. **Contradictory Hypotheses, Shared Conclusion**: - Both hypotheses (blowing vs. pricking) lead to the same conclusion ("The balloon expanded"), requiring explanation evaluation. 2. **Explanation Complexity**: - E1 uses two steps (inflation → expansion). - E2 uses three steps (deflation → pressure drop → expansion), making it less parsimonious. 3. **Selection Criteria Trade-offs**: - E1 scores better on parsimony and coherence but has higher uncertainty than E2. - E2's higher uncertainty and lower coherence make it less favorable despite similar consistency. 4. **Logical Flow**: - Dashed arrows connect hypotheses to explanations, which then feed into IBE. --- ### Interpretation The flowchart illustrates a formalized process for causal reasoning: 1. **Hypothesis Generation**: Two competing causes (blowing vs. pricking) are proposed. 2. **Explanation Construction**: Each hypothesis is expanded into a step-by-step causal chain with explicit assumptions. 3. **Evaluation via IBE**: Explanations are scored on four criteria: - **Consistency**: How well the explanation aligns with observed data. - **Parsimony**: Simplicity of the explanation (fewer assumptions). - **Coherence**: Logical consistency of the explanation's steps. - **Uncertainty**: Degree of ambiguity in the explanation. **Why E1 is Selected**: - E1's explanation (blowing → inflation → expansion) is simpler (fewer steps) and more coherent than E2's (pricking → deflation → pressure drop → expansion). - While E2 introduces a plausible physical mechanism (external pressure causing expansion after deflation), its added complexity and higher uncertainty make it less favorable. **Notable Anomaly**: - The conclusion "The balloon expanded" is counterintuitive for Premise 2 ("I pricked it"), as pricking typically causes deflation. E2 resolves this by invoking external pressure dynamics, but this introduces additional assumptions, increasing uncertainty. This framework demonstrates how logical reasoning systems can evaluate explanations by balancing simplicity, coherence, and alignment with observed outcomes. </details> Figure 1: IBE-Eval qualifies LLM-generated explanations with a set of logical and linguistic selection criteria to identify the most plausible hypothesis. The corresponding explanation for each hypothesis is evaluated across the IBE criteria of logical consistency, parsimony, internal coherence, and linguistic uncertainty. A final plausibility score is computed across those features and the hypothesis with highest score is identified as the best explanation. ## 1 Introduction Large Language Models (LLMs) such as OpenAI’s GPT Brown et al. (2020) and LLaMA Touvron et al. (2023) have been highly effective across a diverse range of language understanding and reasoning tasks Liang et al. (2023). While LLM performances have been thoroughly investigated across various benchmarks Wang et al. (2019); Srivastava et al. (2023); Gao et al. (2023); Touvron et al. (2023), the principles and properties behind their step-wise reasoning process are still poorly understood Valentino et al. (2021). LLMs are notoriously black-box and can be difficult to interpret Chakraborty et al. (2017); Danilevsky et al. (2020). Moreover, the commercialization of LLMs has led to strategic secrecy around model architectures and training details Xiang (2023); Knight (2023). Finally, LLMs are susceptible to hallucinations and adversarial perturbations Geirhos et al. (2020); Camburu et al. (2020), often producing plausible but factually incorrect answers Ji et al. (2023); Huang et al. (2023). As the size and complexity of LLM architectures increase, the systematic study of generated explanations becomes crucial to better interpret and validate the LLM’s internal inference and reasoning processes Wei et al. (2022b); Lampinen et al. (2022); Huang and Chang (2022). The automatic evaluation of natural language explanations presents several challenges Atanasova et al. (2023); Camburu et al. (2020). Without resource-intensive annotation Wiegreffe and Marasovic (2021); Thayaparan et al. (2020); Dalvi et al. (2021); Camburu et al. (2018), explanation quality methods tend to rely on either weak supervision, where the identification of the correct answer is taken as evidence of explanation quality, or require the injection of domain-specific knowledge Quan et al. (2024). In this paper, we seek to better understand the LLM explanatory process through the investigation of explicit linguistic and logical properties. While explanations are hard to formalize due to their open-ended nature, we hypothesize that they can be analyzed as linguistic objects, with measurable features that can serve to define criteria for assessing their quality. Specifically, this paper investigates the following overarching research question: “Can the linguistic and logical properties associated with LLM-generated explanations be used to qualify the models’ reasoning process?”. To this end, we propose an interpretable framework inspired by philosophical accounts of abductive inference, also known as Inference to the Best Explanation (IBE) - i.e. the process of selecting among competing explanatory theories Lipton (2017). In particular, we aim to measure the extent to which LLM-generated explanations conform to IBE expectations when attempting to identify the most plausible explanation. To this end, we present IBE-Eval, a framework designed to estimate the plausibility of natural language explanations through a set of explicit logical and linguistic features, namely: logical consistency, parsimony, coherence, and linguistic uncertainty. To evaluate the efficacy of IBE-Eval, we conduct extensive experiments in the multiple-choice Causal Question Answering (CQA) setting. The overall results and contributions of the paper can be summarized as follows: 1. To the best of our knowledge, we are the first to propose an interpretable framework inspired by philosophical accounts on Inference to the Best Explanation (IBE) to automatically assess the quality of natural language explanations. 1. We propose IBE-Eval, a framework that can be instantiated with external tools for the automatic evaluation of LLM-generated explanations and the identification of the best explanation in a multiple-choice CQA setting. 1. We provide empirical evidence that LLM-generated explanations tend to conform to IBE expectations with varying levels of statistical significance correlated to the LLM’s size. 1. We additionally find that uncertainty, parsimony, and coherence are the best predictors of plausibility and explanation quality across all LLMs. However, we also find that the LLMs tend to be strong rationalizers and can produce logically consistent explanations even for less plausible candidates, making the consistency metric less effective in practice. 1. IBE-Eval can successfully identify the best explanation supporting the correct answers with up to 77% accuracy (+ $\approx 27\$ above random and + $\approx 17\$ over GPT 3.5-as-a-Judge baselines) 1. IBE-Eval is significantly correlated with human judgment, outperforming a GPT3.5-as-a-Judge baseline in terms of alignment with human preferences. For reproducibility, our code is made available on Github https://github.com/dhairyadalal/IBE-eval to encourage future research in the field. ## 2 Inference to the Best Explanation (IBE) Explanatory reasoning is a distinctive feature of human rationality underpinning problem-solving and knowledge creation in both science and everyday scenarios Lombrozo (2012); Deutsch (2011). Accepted epistemological accounts characterize the creation of an explanation as composed of two distinct phases: conjecturing and criticism Popper (2014). The explanatory process always involves a conflict between plausible explanations, which is typically resolved through the criticism phase via a selection process, where competing explanations are assessed according to a set of criteria such as parsimony, coherence, unification power, and hardness to variation Lipton (2017); Harman (1965); Mackonis (2013); Thagard (1978, 1989); Kitcher (1989); Valentino and Freitas (2022). As LLMs become interfaces for natural language explanations, epistemological frameworks offer an opportunity for developing criticism mechanisms to understand the explanatory process underlying state-of-the-art models. To this end, this paper considers an LLM as a conjecture device producing linguistic objects that can be subject to criticism. In particular, we focus on a subset of criteria that can be computed on explicit linguistic and logical features, namely: consistency, parsimony, coherence, and uncertainty. To assess the LLM’s alignment to such criteria, we focus on the task of selecting among competing explanations in a multiple-choice CQA setting (Figure 1). Specifically, given a set of competing hypotheses (i.e. the multiple-choice options), $H=\{h_{1},h_{2},\ldots,h_{n}\}$ , we prompt the LLM to generate plausible explanations supporting each hypothesis (Section 3). Subsequently, we adopt the proposed IBE selection criteria to assess the quality of the generated explanations (Section 4). IBE-Eval computes an explanation plausibility score derived from the linear combination of the computed selection criteria. The explanation with the highest score is selected as the predicted answer and additionally assessed as to the extent to which the observable IBE features are correlated with QA accuracy. We hypothesize that IBE-Eval will produce higher scores for the explanation associated with the correct answer and that the IBE criteria should meaningfully differentiate between competing explanations. ## 3 Explanation Generation For the first stage, the LLM is prompted to generate competing explanations for the hypotheses using a modified Chain-of-Thought (CoT) prompt Wei et al. (2022a). Specifically, the COT prompt is modified to instruct the LLM to produce an explanation for each competing hypothesis (see Figure 1). We adopt a methodology similar to Valentino et al. (2021), where the generated explanation is constrained into an entailment form for the downstream IBE evaluation. In particular, we posit that a valid explanation should demonstrate an entailment relationship between the premise and conclusion which are derived from the question-answer pair. To elicit logical connections between explanation steps and facilitate subsequent analysis, the LLM is constrained to use weak syllogisms expressed as If-Then statements. Additionally, the LLM is instructed to produce the associated causal or commonsense assumption underlying each explanation step. This output is then post-processed to extract the explanation steps and supporting knowledge for evaluation via the IBE selection criteria. Additional details and examples of prompts are reported in Appendix A.2. ## 4 Linguistic & Inference Criteria To perform IBE, we investigate a set of criteria that can be automatically computed on explicit logical and linguistic features, namely: consistency, parsimony, coherence, and uncertainty. Consistency. Consistency aims to verify whether the explanation is logically valid. Given a hypothesis, comprised of a premise $p_{i}$ , a conclusion $c_{i}$ , and an explanation consisting of a set of If-Then statements $E={s_{1},...,s_{i}}$ , we define $E$ to be logically consistent if $p_{i}\cup E\vDash c_{i}$ . Specifically, an explanation is logically consistent if it is possible to build a deductive proof linking premise and conclusion. To evaluate logical consistency, we leverage external symbolic solvers along with autoformalization - i.e., the translation of natural language into a formal language Wu et al. (2022). Specifically, the hypotheses and explanations are formalized into a Prolog program which will attempt to generate a deductive proof via backward chaining Weber et al. (2019). To perform autoformalization, we leverage the translation capabilities of GPT 3.5. Specifically, we instruct GPT 3.5 to convert each IF-Then explanation step from the generated explanation into an implication rule and the premise statement into grounding atoms. On the other end, the entailment condition and the conclusion are used to create a Prolog query. The query instructs the Prolog solver to attempt to find a path through the implications rules such that the conclusion be directly connected to the premise. Further details about the autoformalization process can be found in Appendix A.3. After autoformalization, following recent work on neuro-symbolic integration for LLM explanations Quan et al. (2024), we adopt an external Prolog solver for entailment verification https://github.com/neuro-symbolic-ai/explanation_based_ethical_reasoning. The explanation is considered consistent if the Prolog solver can satisfy the query and successfully build a deductive proof. Technical details can be found in Appendix A.5. Parsimony. The parsimony principle, also known as Ockham’s razor, favors the selection of the simplest explanation consisting of the fewest elements and assumptions Sober (1981). Epistemological accounts posit that an explanation with fewer assumptions tends to leave fewer statements unexplained, improving specificity and alleviating the infinite regress Thagard (1978). Further, parsimony is an essential feature of causal interpretability, as only parsimonious solutions are guaranteed to reflect causation in comparative analysis Baumgartner (2015). In this paper, we adopt two metrics as a proxy of parsimony, namely proof depth, and concept drift. Proof depth, denoted as $Depth$ , is defined as the cardinality of the set of rules, $R$ , required by the Prolog solver to connect the conclusion to the premise via backward chaining. Let $h$ be a hypothesis candidate composed of a premise $p$ and a conclusion $c$ , and let $E$ be a formalized explanation represented as a set of rules $R^{\prime}$ . The proof depth is the number of rules $|R|$ , with $R\subseteq R^{\prime}$ , traversed during backward chaining to connect the conclusion $c$ to the premise $p$ : $$ Depth(h)=|R| $$ Concept drift, denoted as $Drift$ , is defined as the number of additional concepts and entities, outside the ones appearing in the hypothesis (i.e., premise and conclusion), that are introduced by the LLM to support the entailment. For simplicity, we consider nouns as concepts. Let $N=\{Noun_{p},Noun_{c},Noun_{E}\}$ be the unique nouns found in the premise, conclusion, and explanation steps. Concept drift is the cardinality of the set difference between the nouns found in the explanation and the nouns in the hypothesis: $$ Drift(h)=|Noun_{E}-(Noun_{p}\cup Noun_{c})| $$ Intuitively, the parsimony principle would predict the most plausible hypothesis as the one supported by an explanation with the smallest observed proof depth and concept drift. Implementation details can be found in Appendix A.6. Coherence. Coherence attempts to measure the logical validity at the level of the specific explanation steps. An explanation can be formally consistent on the surface while still including implausible or ungrounded intermediate assumptions. Coherence evaluates the quality of each intermediate If-Then implication by measuring the entailment strength between the If and Then clauses. To this end, we employ a fine-tuned natural language inference (NLI) model. Formally, let $S$ be a set of explanation steps, where each step $s$ consists of an If-Then statement, $s=(If_{s},Then_{s})$ . For a given step $s_{i}$ , let $ES(s_{i})$ denote the entailment score obtained via the NLI model between $If_{s}$ and $Then_{s}$ clauses. The step-wise entailment score $SWE(S)$ is then calculated as the averaged sum of the entailment scores across all explanation steps $|S|$ : $$ \text{SWE}(S)=\frac{1}{|S|}\sum_{i=1}^{|S|}\text{ES}(s_{i}) $$ We hypothesize that the LLM should generate a higher coherence score for more plausible hypotheses, as such explanations should exhibit stronger step-wise entailment. Additional details can be found in Appendix A.7. Uncertainty. Finally, we consider the linguistic certainty expressed in the generated explanation as a proxy for plausibility. Hedging words such as probably, might be, could be, etc typically signal ambiguity and are often used when the truth condition of a statement is unknown or improbable. Pei and Jurgens (2021) found that the strength of scientific claims in research papers is strongly correlated with the use of direct language. In contrast, they found that the use of hedging language suggested that the veracity of the claim was weaker or highly contextualized. To measure the linguistic uncertainty ( $UC$ ) of an explanation, we consider the explanation’s underlying assumptions ( $A_{i}$ ) and the overall explanation summary ( $S$ ). The linguistic uncertainty score is extracted using the fine-tuned sentence-level RoBERTa model from Pei and Jurgens (2021). The overall linguistic uncertainty score ( $UC_{\text{overall}}$ ) is the sum of the assumption and explanation summary scores: $$ UC_{\text{overall}}=UC(A)+UC(S) $$ Where $UC(A)$ is the sum of the linguistic uncertainty scores ( $UC(A)$ ) across all the assumptions $|A|$ associated with each explanation step $i$ : $$ UC(A)=\sum_{i=1}^{|A|}UC(a_{i}) $$ and linguistic uncertainty of the explanation summary $UC(S)$ . We hypothesize that the LLM will use more hedging language when explaining the weaker hypothesis resulting in a higher uncertainty score. Further details can be found in Appendix A.8. ### 4.1 Inference to Best Explanation After the IBE criteria are computed for each competing hypothesis, they are used to generate the final explanation plausibility score. We define a simple linear regression model $\theta(\cdot)$ , which was fitted on a small set of training examples consisting of extracted IBE features to predict the probability that an explanation $E_{i}$ corresponds to the correct answer. Specifically, we employ IBE-Eval to score each generated explanation independently and then select the final answer $a$ via argmax: $$ a=\operatorname*{argmax}_{i}[\theta(E_{1}),\ldots,\theta(E_{n})] $$ Additional details can be found in Appendix A.9. <details> <summary>extracted/6246183/correlation.png Details</summary>  ### Visual Description ## Heatmap: Model Performance Comparison (COPA vs E-CARE) ### Overview The image presents two side-by-side heatmaps comparing correlation values between different AI models (LLaMA 2 7B, LLaMA 2 13B, GPT 3.5) across five evaluation metrics (Consistency, Depth, Coherence, Uncertainty, Drift) for two frameworks: COPA and E-CARE. Color gradients and statistical significance markers provide additional context. ### Components/Axes **X-Axes (Metrics):** - Consistency - Depth - Coherence - Uncertainty - Drift **Y-Axes (Models):** - COPA section: - LLaMA 2 7B - LLaMA 2 13B - GPT 3.5 - E-CARE section: - LLaMA 2 7B - LLaMA 2 13B - GPT 3.5 **Legend:** - Color gradient: Green (positive correlation) to Red (negative correlation) - Scale: -2.5 (red) to +2.5 (green) - Significance markers: - *** (p < 0.001) - ** (p < 0.01) - * (p < 0.05) **Spatial Layout:** - Legend positioned centrally on the right side - COPA heatmap occupies left half - E-CARE heatmap occupies right half - All values displayed in cell centers with 2-3 decimal precision ### Detailed Analysis **COPA Framework:** | Model | Consistency | Depth | Coherence | Uncertainty | Drift | |----------------|-------------|----------|-----------|-------------|--------| | LLaMA 2 7B | 1.37 | -2.95** | 1.22 | -3.10** | -0.27 | | LLaMA 2 13B | 1.36 | -1.28 | 3.87*** | -2.17* | -3.33*** | | GPT 3.5 | 4.67*** | -4.893***| 3.60*** | -4.34*** | -3.22** | **E-CARE Framework:** | Model | Consistency | Depth | Coherence | Uncertainty | Drift | |----------------|-------------|----------|-----------|-------------|--------| | LLaMA 2 7B | 0.20 | -0.53 | 2.18* | -2.11** | -0.78* | | LLaMA 2 13B | 1.167 | -1.18 | 1.67* | -1.52* | -1.91* | | GPT 3.5 | 3.10** | -2.91** | 0.98 | -2.61** | -5.14*** | ### Key Observations 1. **GPT 3.5 Dominance in COPA Consistency**: Shows strongest positive correlation (4.67) with high significance (***) 2. **LLaMA 2 13B Coherence Peak**: Highest coherence correlation (3.87) in COPA with strongest significance (***) 3. **Drift Vulnerability**: GPT 3.5 exhibits most negative drift correlation (-5.14) in E-CARE 4. **Model Size Impact**: Larger LLaMA models show improved coherence but increased drift sensitivity 5. **Statistical Significance**: 68% of values show at least moderate significance (p < 0.05) ### Interpretation The data reveals fundamental differences in model behavior between frameworks: - **COPA** emphasizes consistency and coherence, where GPT 3.5 and larger LLaMA models excel - **E-CARE** shows greater drift sensitivity, particularly affecting GPT 3.5 - Model size appears to enhance coherence but introduces drift vulnerability - Statistical significance markers confirm robust patterns, with 14/18 values showing p < 0.05 - Color gradients visually reinforce the correlation strength, with red cells (negative) dominating in depth and drift metrics The findings suggest framework-specific optimization requirements: COPA benefits from models with strong consistency/coherence, while E-CARE requires drift-resistant architectures. The statistical significance markers provide confidence in these observed patterns, particularly for GPT 3.5's extreme drift correlation in E-CARE. </details> Figure 2: A regression analysis measuring the correlation between IBE criteria and question accuracy. All the LLMs tend to conform to IBE expectations with GPT 3.5 exhibiting the most consistent and significant alignment. Linguistic uncertainty is the strongest IBE predictor for explanation quality, where higher uncertainty is negatively correlated with question accuracy. Statistical significance is noted as: ‘***’ p < 0.001, ‘**’ p < 0.01 ‘*’ p < 0.05. | Baselines | COPA GPT 3.5 | E-CARE LlaMA 2 13B | LlaMA 2 7B | GPT 3.5 | LlaMA 2 13B | LlaMA 2 7B | | --- | --- | --- | --- | --- | --- | --- | | GPT3.5 Judge | .59 | .47 | .63 | .43 | .61 | .52 | | Human | .95 | 1.0 | .91 | .90 | .91 | .92 | | IBE Features | | | | | | | | Consistency | .51 | .52 | .55 | .54 | .54 | .54 | | Depth (Parsimony) | .67 | .53 | .63 | .66 | .56 | .54 | | Drift (Parsimony) | .67 | .63 | .58 | .66 | .57 | .57 | | Coherence | .66 | .66 | .56 | .56 | .57 | .59 | | Linguistic Uncertainty | .70 | .65 | .61 | .59 | .56 | .60 | | Composed Model | | | | | | | | Random | .50 | .50 | .50 | .50 | .50 | .50 | | + Consistency | .51 | .52 | .55 | .54 | .54 | .54 | | + Depth | .67 | .53 | .63 | .66 | .56 | .56 | | + Drift | .70 | .65 | .65 | .72 | .66 | .65 | | + Coherence | .73 | .71 | .69 | .73 | .68 | .69 | | + Linguistic Uncertainty | .77 | .74 | .70 | .74 | .70 | .73 | Table 1: An ablation study and evaluation of the IBE criteria and the composed IBE-Eval model. IBE-Eval outperforms the GPT 3.5 Judge baseline by an average of +17.5% across all all models and tasks. ## 5 Experimental Setting Causal Question-Answering (CQA) requires reasoning about the causes and effects given an event description. We specifically consider the task of cause and effect prediction in a multiple-choice setting, where given a question and two candidate answers, the LLM must decide which is the most plausible cause or effect. Causal reasoning is a challenging task as the model must both possess commonsense knowledge about causal relationships and consider the event context which would make one option more plausible than the other. For our experiments, we use the Choice of Plausible Alternatives (COPA) Gordon et al. (2012) and E-CARE Du et al. (2022) datasets. COPA. COPA is a multiple-choice commonsense causal QA dataset consisting of 500 train and test examples that were manually generated. Each multiple-choice example consists of a question premise and a set of answer candidates which are potential causes or effects of the premise. COPA is a well-established causal reasoning benchmark that is both a part of SuperGlue Wang et al. (2019) and the CALM-Bench Dalal et al. (2023). E-CARE. E-CARE is a large-scale multiple-choice causal crowd-sourced QA dataset consisting of 15K train and 2k test examples. Similar to COPA, the task requires the selection of the most likely cause or effect provided an event description. We randomly sample 500 examples from the E-CARE test set for our experiments. LLMs. We consider GPT-Turbo-3.5, LLaMA 2 13B, and LLaMA 2 7B for all experiments. GPT 3.5 is a proprietary model Brown et al. (2020) and is highly effective across a wide range of natural language reasoning tasks Laskar et al. (2023). We additionally evaluate the open-source LLaMA 2 model Touvron et al. (2023). We consider both the 13B and 7B variants of Llama 2 as both are seen as viable commodity GPT alternatives and have been widely adopted by the research community for LLM benchmarking and evaluation. Baselines. We employ LLM-as-a-Judge Zheng et al. (2023) and human evaluators as baseline methods for the selection of the best explanation in the CQA setting. (Zheng et al., 2023) found LLMs can align with human judgment and be utilized for automated evaluation and judgment. We specifically uses GPT 3.5 as the LLM judge. For each CQA example, we present the judges with two competing explanations generated by the target LLM. The judge is asked to identify the best and most plausible explanation. Additional details about the baselines can be found in Appendix A.4. ## 6 Preliminary Analysis We conduct a preliminary analysis as a sanity check to measure the extent to which LLMs generate self-evident or tautological explanations - i.e., explanations that simply restate the premises and conclusions. Tautological explanations present a risk for IBE-Eval as the metrics would be theoretically uninformative if the LLM adopts the tested causal relation as the explanation itself (e.g. A → B) without providing additional supporting statements. We consider the parsimony metric to compute the percentage of explanations with proof depth equal to 1 (i.e, explanations containing only one inference step) and concept drift equal to 0 (i.e. no additional concepts other than the ones stated in premises and conclusions appear in the explanation). In such cases, the LLM is effectively generating a self-evident or tautological explanation. We found that about 2% of the cases consist of self-evident explanations. For GPT 3.5, LLaMA 2 13B, and LLaMA 2 7B, 2% of the generated explanations exhibit a concept drift of 0, and on average 1.5% of the explanations have a proof depth of 1. We then conducted an error analysis to evaluate the cases where IBE-Eval selected a self-evident explanation as the best one. Across all LLMs, less than 0.1% of the errors were caused by the selection of such explanations. Our analysis suggests that the impact of self-evident explanations is not significant and that the IBE framework can be robustly applied to identify such cases. ## 7 Results To assess the LLM’s alignment with the proposed IBE framework and evaluate the efficacy of IBE-Eval, we run a regression analysis and conduct a set of ablation studies to evaluate the relationship between IBE and question accuracy. The main results are presented in Figure 2 and Table 1. Our regression analysis finds that the IBE criteria are generally consistent across the LLMs as demonstrated by similar correlation patterns found on both the COPA and E-CARE tasks (Figure 2). GPT 3.5 exhibits the strongest alignment with IBE expectations as we observe nearly all the IBE criteria have statistically significant and directionally aligned correlations across both tasks. Thus our proposed IBE criteria can serve as promising build blocks for future work on automated explanation evaluation. In Table 1 we evaluate the accuracy of the IBE criteria and IBE-Eval in selecting the most plausible explanation in the CQA setting. We find that though independently the IBE criteria are generally limited in their ability to identify the more plausible explanation - they still outperform the GPT-3.5-as-a-judge baseline. IBE-Eval, which considers all IBE criteria, improves the ability to select the best explanation by 17% over both the GPT 3.5-as-a-judge and random baselines. We can achieve up to 77% accuracy utilizing just the extracted IBE criteria demonstrating IBE’s potential value for automatic explanation evaluation. Next, we explore each explanation feature in further detail to better understand the variances across the IBE criteria and LLMs. Consistency. We find that the LLMs are surprisingly strong conjecture models. The LLMs can generate logically consistent explanations for any hypothesis as observed by similar consistency scores for correct and incorrect (Figure 3) explanations. Moreover, we observe that consistency tends to be a statistically insignificant predictor for the LLaMA models. Therefore, we conclude that evidence of logical consistency provides a limited signal for plausibility and is better understood in the context of other IBE criteria. For the incorrect candidate explanations, we find that LLMs over-rationalize and introduce additional premises to demonstrate entailment in their explanations. <details> <summary>extracted/6246183/consistency.png Details</summary>  ### Visual Description ## Bar Chart: Logical Consistency of AI Models ### Overview The chart compares the logical consistency of three AI models (Llama 2 7B, Llama 2 13B, and ChatGPT) using split bars to represent percentages of "correct" and "incorrect" responses. The x-axis shows "% Logically Consistent," while the y-axis lists the models. Green segments represent correct responses, and red segments represent incorrect responses. ### Components/Axes - **X-axis**: Labeled "% Logically Consistent" (percentage scale). - **Y-axis**: Categorical axis listing models: - Llama 2 7B (top) - Llama 2 13B (middle) - ChatGPT (bottom) - **Legend**: - Green = Correct - Red = Incorrect - **Data Labels**: Percentages displayed in white boxes on each bar segment. ### Detailed Analysis 1. **Llama 2 7B**: - Correct: 70% (green) - Incorrect: 76% (red) 2. **Llama 2 13B**: - Correct: 78% (green) - Incorrect: 77% (red) 3. **ChatGPT**: - Correct: 81% (green) - Incorrect: 82% (red) ### Key Observations - **ChatGPT** outperforms both Llama models in correct responses (81% vs. 78% and 70%). - **Llama 2 13B** shows improved logical consistency over Llama 2 7B (+8% correct, -6% incorrect). - All models have incorrect response rates above 75%, with ChatGPT having the highest (82%). - The split-bar design highlights the trade-off between correct and incorrect responses across models. ### Interpretation The data suggests that larger model sizes (e.g., Llama 2 13B) correlate with better logical consistency than smaller models (Llama 2 7B). However, ChatGPT dominates both metrics, indicating superior performance. The high incorrect rates across all models highlight a systemic challenge in achieving logical consistency, even for advanced systems. The red segments (incorrect) consistently exceed green segments (correct) for all models, suggesting a need for targeted improvements in reasoning capabilities. </details> Figure 3: An evaluation of explanation consistency. LLMs are strong rationalizers and can generate logically consistent explanations at equal rates for explanations associated with both correct and incorrect answers options. Parsimony. <details> <summary>extracted/6246183/parsimony.png Details</summary>  ### Visual Description ## Bar Charts: Avg. Proof Depth and Expl. Concept Drift ### Overview The image contains two vertically stacked bar charts comparing three language models (Llama 2 7B, Llama 2 13B, ChatGPT) across two metrics: "Avg. Proof Depth" (top) and "Expl. Concept Drift" (bottom). Each chart uses color-coded bars (green = correct, red = incorrect) to represent performance metrics. ### Components/Axes - **X-Axes**: - Top Chart: "Depth" (numerical scale) - Bottom Chart: "Drift" (numerical scale) - **Y-Axes**: - Categories: Llama 2 7B (top), Llama 2 13B (middle), ChatGPT (bottom) - **Legend**: - Position: Bottom center - Labels: - Green = "correct" - Red = "incorrect" ### Detailed Analysis #### Avg. Proof Depth (Top Chart) - **Llama 2 7B**: - Incorrect (red): 2.94 - Correct (green): 2.68 - **Llama 2 13B**: - Incorrect (red): 3.27 - Correct (green): 3.07 - **ChatGPT**: - Incorrect (red): 3.17 - Correct (green): 2.63 #### Expl. Concept Drift (Bottom Chart) - **Llama 2 7B**: - Incorrect (red): 4.33 - Correct (green): 3.96 - **Llama 2 13B**: - Incorrect (red): 4.55 - Correct (green): 3.67 - **ChatGPT**: - Incorrect (red): 4.18 - Correct (green): 2.95 ### Key Observations 1. **Consistent Pattern**: Incorrect options (red bars) consistently show higher values than correct options (green bars) in both metrics across all models. 2. **Model Comparison**: - Llama 2 7B has the highest values for both metrics (incorrect: 2.94/4.33; correct: 2.68/3.96). - ChatGPT has the lowest values for both metrics (incorrect: 3.17/4.18; correct: 2.63/2.95). 3. **Drift vs. Depth**: Expl. Concept Drift values are uniformly higher than Avg. Proof Depth values for corresponding options. ### Interpretation The data suggests that: - **Incorrect options** require deeper cognitive processing (higher proof depth) and exhibit greater concept drift compared to correct options. - Llama 2 7B demonstrates the most significant disparity between correct and incorrect options, indicating potential inefficiencies in handling erroneous inputs. - ChatGPT shows the smallest gaps between correct and incorrect metrics, suggesting more robust performance across both dimensions. - The consistent red-green value hierarchy implies that model architectures may inherently struggle more with incorrect than correct options, regardless of size (7B vs. 13B parameters). </details> Figure 4: Explanation parsimony is evaluated using proof depth and concept drift. Both metrics are consistently lower for explanations supporting the correct answers suggesting that LLMs are able to generate efficient explanations for the more plausible hypothesis. The results suggest that parsimony has a more consistent effect and is a better predictor of explanation quality. We observe negative correlations between proof depth, concept drift, and question-answering accuracy, suggesting that LLMs tend to introduce more concepts and explanation steps when explaining less plausible hypotheses. On average, we found the depth and drift to be 6% and 10% greater for the incorrect option across all LLMs (Figure 4). Moreover, the results suggest that as the LLM parameter size increases, the tendency to over-rationalize increases as well. This is attested by the fact that the average difference in depth and drift is the greatest in GPT 3.5, suggesting that the model tends to find the most efficient explanations for stronger hypotheses and articulates explanations for weaker candidates. Finally, we found that the LLaMA models tend to generate more complex explanations overall, with LLaMA 2 13B exhibiting the largest concept drift for less plausible hypotheses. The parsimony criterion supports the IBE predictive power with an average of 14% improvement over consistency. Coherence. <details> <summary>extracted/6246183/coherence.png Details</summary>  ### Visual Description ## Bar Chart: Avg. Coherence Scores ### Overview The chart compares average coherence scores for three language models (ChatGPT, Llama 2 13B, Llama 2 7B) across two categories: "Correct" and "Incorrect" responses. A red dashed line labeled "Rel. Difference %" overlays the chart, showing relative differences between correct and incorrect scores. ### Components/Axes - **X-axis**: Language models (ChatGPT, Llama 2 13B, Llama 2 7B) - **Y-axis (left)**: Coherence Score (0.0 to 0.35) - **Legend (right)**: - Green = Correct - Red = Incorrect - **Secondary Y-axis (right)**: Rel. Difference % (0% to 30%) ### Detailed Analysis 1. **ChatGPT**: - Correct: ~0.26 - Incorrect: ~0.20 - Relative Difference: ~20% (red dashed line starts here) 2. **Llama 2 13B**: - Correct: ~0.28 - Incorrect: ~0.19 - Relative Difference: ~30% (peak of red dashed line) 3. **Llama 2 7B**: - Correct: ~0.22 - Incorrect: ~0.19 - Relative Difference: ~10% (red dashed line dips here) ### Key Observations - Llama 2 13B has the highest coherence scores for both correct and incorrect responses. - The relative difference between correct and incorrect scores is largest for Llama 2 13B (30%) and smallest for Llama 2 7B (10%). - ChatGPT shows a moderate relative difference (20%) with higher incorrect scores than Llama 2 7B. ### Interpretation The data suggests that Llama 2 13B demonstrates the strongest performance in coherence, with the largest gap between correct and incorrect responses. Llama 2 7B underperforms in both metrics, while ChatGPT balances moderate coherence with a significant relative difference. The red dashed line emphasizes that Llama 2 13B's superiority is most pronounced in the relative difference metric, indicating it handles correct responses more effectively than the others. The consistent incorrect scores across models (~0.19–0.20) suggest similar error rates, but Llama 2 13B's higher correct scores amplify its relative advantage. </details> Figure 5: An evaluation of the explanation coherence and question accuracy.The average coherence score is consistently higher for explanations corresponding to the correct hypotheses across the LLMs. Similarly to parsimony, we found coherence to be a better indicator of explanation quality being statistically significant for both GPT 3.5 and Llama 2 13B on COPA and both Llama 2 models on E-Care. We found that the average coherence score is consistently greater for the stronger hypothesis across all LLMs and datasets (see Figure 5). Both GPT and Llama 2 13B exhibit a higher relative difference between the correct and incorrect hypotheses in contrast to Llama 2 7B. Uncertainty. <details> <summary>extracted/6246183/uncertainty.png Details</summary>  ### Visual Description ## Bar Charts: Model Performance Metrics ### Overview The image contains three bar charts comparing model performance across three AI systems: ChatGPT, Llama 2 13B, and Llama 2 7B. The charts focus on uncertainty metrics, explanation hedge cues, and their distribution in incorrect explanations. ### Components/Axes 1. **Top-Left Chart: Avg. Uncertainty** - **Y-axis**: Score (0–4) - **X-axis**: Models (ChatGPT, Llama 2 13B, Llama 2 7B) - **Legend**: Green = correct, Red = incorrect - **Data Points**: - ChatGPT: Correct = 3.5, Incorrect = 3.8 - Llama 2 13B: Correct = 3.8, Incorrect = 4.1 - Llama 2 7B: Correct = 3.4, Incorrect = 3.7 - **Trend**: Red bars (incorrect) consistently exceed green bars (correct) by ~10% relative difference. 2. **Top-Right Chart: Avg. Ratio of Expl. Hedge Cues** - **Y-axis**: Ratio (0–0.04) - **X-axis**: Models (same as above) - **Legend**: Green = correct, Red = incorrect - **Data Points**: - ChatGPT: Correct = 0.03, Incorrect = 0.04 - Llama 2 13B: Correct = 0.035, Incorrect = 0.045 - Llama 2 7B: Correct = 0.032, Incorrect = 0.042 - **Trend**: Red bars (incorrect) exceed green bars by ~20% relative difference. 3. **Bottom Chart: Hedge Cues in Incorrect Expl.** - **Y-axis**: Models (same as above) - **X-axis**: Percentage (0–100%) - **Legend**: Red = Conditional, Green = Doxastic, Blue = Epistemic - **Data Points**: - ChatGPT: Epistemic = 75%, Conditional = 20%, Doxastic = 5% - Llama 2 13B: Epistemic = 80%, Conditional = 15%, Doxastic = 5% - Llama 2 7B: Epistemic = 78%, Conditional = 18%, Doxastic = 4% ### Key Observations 1. **Uncertainty Trends**: - Llama 2 13B shows the highest uncertainty in incorrect answers (4.1 vs. 3.8 correct). - ChatGPT has the lowest uncertainty in correct answers (3.5 vs. 3.8 incorrect). 2. **Hedge Cue Ratios**: - All models exhibit ~20% higher hedge cue ratios in incorrect explanations. - Llama 2 13B has the highest ratio (0.045 incorrect vs. 0.035 correct). 3. **Hedge Cue Distribution**: - Epistemic hedges dominate incorrect explanations (75–80%). - Doxastic hedges are minimal (4–5%). - Conditional hedges vary slightly (15–20%). ### Interpretation 1. **Uncertainty and Hedge Cues**: - Higher uncertainty in incorrect answers correlates with increased use of hedge cues (e.g., "might," "could"). - Llama 2 13B’s elevated uncertainty and hedge cue ratio suggest it is more cautious in uncertain scenarios. 2. **Hedge Cue Types**: - Epistemic hedges (e.g., "I think") dominate, indicating models express uncertainty about factual claims. - Doxastic hedges (e.g., "I believe") are rare, suggesting models rarely express personal conviction in incorrect answers. - Conditional hedges (e.g., "If X, then Y") are moderately present, reflecting logical constraints in incorrect reasoning. 3. **Model Differences**: - Llama 2 13B outperforms smaller models in uncertainty metrics but shows similar hedge cue patterns. - ChatGPT’s lower uncertainty in correct answers may reflect better calibration in accurate responses. ### Spatial Grounding - **Legends**: All legends are positioned on the right side of their respective charts. - **Data Alignment**: Bars are aligned horizontally, with red (incorrect) bars consistently taller than green (correct) bars in the first two charts. - **Stacked Bars**: The bottom chart uses vertical stacking to show proportional distributions of hedge cue types. ### Final Notes The data highlights trade-offs between model size, uncertainty, and explanation strategies. Llama 2 13B’s higher uncertainty and hedge cue usage may reflect its larger parameter count, while ChatGPT’s lower uncertainty suggests more confident (but potentially less nuanced) responses. The dominance of epistemic hedges in incorrect explanations underscores a common strategy for models to signal uncertainty when uncertain. </details> Figure 6: Evaluation of linguistic uncertainty in LLM-generated explanations. LLMs tend to use more hedging language in explanations supporting less plausible hypotheses. Across the LLMs, the hedging language is found to be predominantly epistemic A.8. The results reveal that linguistic uncertainty is the strongest predictor of explanation quality and is a statistically significant feature for all LLMs. This suggests that LLMs use more qualifying language when explaining weaker hypotheses (see Figure 6). We found that uncertainty can improve accuracy by 13pp on COPA and 4pp on E-CARE. We also examine the uncertainty cues expressed by LLMs by analyzing both the frequency of hedge words and the types of hedge cues employed in incorrect explanations. We find the distribution of hedge cues across LLMs tends to be similar, with only minor differences between LLMs (Figure 6). Epistemic cues were most frequently used by all three models, with LLaMA 2 7B being more likely to use conditional cues. See Appendix A.8 for further details.. ### 7.1 Correlation with Human Judgement. We first sample 100 generated explanation pairs across both the COPA and E-CARE tasks and evaluated LLMs. Two human evaluators are instructed to evaluate the pair of explanations and to select which explanation is most plausible. No additional information about the original question nor the correct answer is provided to prevent biasing the judge. The human evaluators on average were able to identify the explanation associated with the correct answer 96% (COPA) and 91% (E-Care) of time. We compute the inter-evaluator agreement score between two human evaluators and find that there is Cohen’s Kappa score of .68 suggesting there is a strong agreement between the two evaluators. To evaluate if IBE-Eval is correlated with human judgment, we compute the Spearman’s rank correlation between GPT-3.5-as-a-judge, IBE-Eval and human judgment. We find that GPT-3.5-as-a-judge exhibits a weak and statistically insignificant correlation with human judgment (0.31). In contrast, we find that the IBE-Eval is significantly aligned with human preferences (with a Spearman’s correlation of 0.64 and p < 0.01) further suggesting the IBE’s potential for automatic explanation evaluation. ## 8 Related Work Explorations of LLM reasoning capabilities across various domains (e.g. arithmetic, commonsense, planning, symbolic, etc) are an emerging area of interest Xu et al. (2023); Huang and Chang (2023). Prompt-based methods Wei et al. (2022b); Zhou et al. (2023); Wang et al. (2023), such as CoT, investigate strategies to elicit specific types of reasoning behavior through direct LLM interaction. Olausson et al. (2023) investigate automatic proof generation and propose a neurosymbolic framework with an LLM semantic parser and external solver. Creswell et al. (2022) propose an inference framework where the LLM acts as both a selection and inference module to produce explanations consisting of causal reasoning steps in entailment tasks. Research on LLM faithfulness Atanasova et al. (2023) investigates if LLM explanations are robust to spurious input alterations. Parcalabescu and Frank (2024) propose a self-consistency measure CC-SHAP which measures how specific alterations to a model’s input contribute to the generated explanation. This paper primarily draws inspiration from recent work on the evaluation of natural language explanations Quan et al. (2024); Valentino et al. (2021); Wiegreffe and Marasovic (2021); Thayaparan et al. (2020); Dalvi et al. (2021); Camburu et al. (2018). However, differently from previous methods that require extensive human annotations or specific domain knowledge, we are the first to propose a set of criteria that can be automatically computed on explicit linguistic and logical features. ## 9 Conclusion This paper proposed IBE-Eval, an interpretable framework for LLM explanation evaluation inspired by philosophical accounts of Inference to the Best Explanation (IBE). IBE-Eval can identify the best explanation supporting the correct answer with up to 77% accuracy in CQA scenarios, improving upon a GPT 3.5 Judge baselines by +17%. Our regression study suggests that LLM explanations tend to conform to IBE expectations and that IBE-Eval is strongly correlated with human judgment. Linguistic uncertainty is the stronger IBE predictor for explanation quality closely followed by parsimony and coherence. However, we also found that LLMs tend to be strong conjecture models able to generate logically consistent explanations for less plausible hypotheses, suggesting limited applicability for the logical consistency criterion in isolation. We believe our findings can open new lines of research on external evaluation methods for LLMs as well as interpretability tools for understanding the LLM’s underlying explanatory process. ## 10 Limitations IBE-Eval offers an interpretable explanation evaluation framework utilizing logical and linguistic features. Our current instantiation of the framework is primarily limited in that it does not consider grounded knowledge for factuality. We observe that LLMs can generate factually incorrect but logically consistent explanations. In some cases, the coherence metric can identify those factual errors when the step-wise entailment score is comparatively lower. However, our reliance on aggregated metrics can hide weaker internal entailment especially when the explanation is longer or the entailment strength of the surrounding explanation steps is stronger. Future work can introduce metrics to evaluate grounded knowledge or perform more granular evaluations of explanations to better weight factual inaccuracies. Additionally, IBE-Eval currently does not support single natural language explanations and was evaluated in the limited domain of causal commonsense reasoning. Future work will explore globally calibrating IBE-Eval plausibility scores to extend evaluation to more diverse explanation generation and QA settings. Calibration efforts would allow for IBE-Eval to generate comparable scores across unrelated explanations and could be used to produce global thresholds explanation classification. Finally, the list of criteria considered in this work is not exhaustive and can be extended in future work. However, additional criteria for IBE might not be straightforward to implement (e.g., unification power, hardness to variation) and would probably require further progress in both epistemological accounts and existing NLP technology. ## 11 Ethics Statement The human annotators for computing the human judgment baseline are all authors of the papers and as such were not further compensated for the annotation task. ## Acknowledgements This work was partially funded by the Swiss National Science Foundation (SNSF) project NeuMath (200021_204617), by the EPSRC grant EP/T026995/1 entitled “EnnCore: End-to-End Conceptual Guarding of Neural Architectures” under Security for all in an AI-enabled society, by the CRUK National Biomarker Centre, and supported by the Manchester Experimental Cancer Medicine Centre, the Science Foundation Ireland under grants SFI/18/CRT/6223 (Centre for Research Training in Artificial Intelligence), SFI/12/RC/2289_P2 (Insight), co-funded by the European Regional Development Fund, and the NIHR Manchester Biomedical Research Centre. ## References - Atanasova et al. (2023) Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, and Isabelle Augenstein. 2023. Faithfulness tests for natural language explanations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 283–294, Toronto, Canada. Association for Computational Linguistics. - Baumgartner (2015) Michael Baumgartner. 2015. Parsimony and causality. Quality & Quantity, 49:839–856. - Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computational Linguistics. - Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. - Buitinck et al. (2013) Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. 2013. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122. - Camburu et al. (2018) Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. Advances in Neural Information Processing Systems, 31. - Camburu et al. (2020) OM Camburu, B Shillingford, P Minervini, T Lukasiewicz, and P Blunsom. 2020. Make up your mind! adversarial generation of inconsistent natural language explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. ACL Anthology. - Chakraborty et al. (2017) Supriyo Chakraborty, Richard Tomsett, Ramya Raghavendra, Daniel Harborne, Moustafa Alzantot, Federico Cerutti, Mani Srivastava, Alun Preece, Simon Julier, Raghuveer M. Rao, Troy D. Kelley, Dave Braines, Murat Sensoy, Christopher J. Willis, and Prudhvi Gurram. 2017. Interpretability of deep learning models: A survey of results. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), pages 1–6. - Creswell et al. (2022) Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022. Selection-inference: Exploiting large language models for interpretable logical reasoning. - Dalal et al. (2023) Dhairya Dalal, Paul Buitelaar, and Mihael Arcan. 2023. CALM-bench: A multi-task benchmark for evaluating causality-aware language models. In Findings of the Association for Computational Linguistics: EACL 2023, pages 296–311, Dubrovnik, Croatia. Association for Computational Linguistics. - Dalvi et al. (2021) Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. Explaining answers with entailment trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7358–7370. - Danilevsky et al. (2020) Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable AI for natural language processing. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 447–459, Suzhou, China. Association for Computational Linguistics. - Deutsch (2011) David Deutsch. 2011. The beginning of infinity: Explanations that transform the world. penguin uK. - Du et al. (2022) Li Du, Xiao Ding, Kai Xiong, Ting Liu, and Bing Qin. 2022. e-CARE: a new dataset for exploring explainable causal reasoning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 432–446, Dublin, Ireland. Association for Computational Linguistics. - Farkas et al. (2010) Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The CoNLL-2010 shared task: Learning to detect hedges and their scope in natural language text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task, pages 1–12, Uppsala, Sweden. Association for Computational Linguistics. - Gao et al. (2023) Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. 2023. A framework for few-shot language model evaluation. - Geirhos et al. (2020) Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673. - Gordon et al. (2012) Andrew Gordon, Zornitsa Kozareva, and Melissa Roemmele. 2012. SemEval-2012 task 7: Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pages 394–398, Montréal, Canada. Association for Computational Linguistics. - Harman (1965) Gilbert H Harman. 1965. The inference to the best explanation. The philosophical review, 74(1):88–95. - Honnibal and Montani (2017) Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. - Huang and Chang (2022) Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403. - Huang and Chang (2023) Jie Huang and Kevin Chen-Chuan Chang. 2023. Towards reasoning in large language models: A survey. - Huang et al. (2023) Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. - Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12). - Kitcher (1989) Philip Kitcher. 1989. Explanatory unification and the causal structure of the world. - Knight (2023) Will Knight. 2023. Ai is becoming more powerful-but also more secretive. - Lampinen et al. (2022) Andrew Lampinen, Ishita Dasgupta, Stephanie Chan, Kory Mathewson, Mh Tessler, Antonia Creswell, James McClelland, Jane Wang, and Felix Hill. 2022. Can language models learn from explanations in context? In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 537–563. - Laskar et al. (2023) Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, and Jimmy Xiangji Huang. 2023. A systematic study and comprehensive evaluation of chatgpt on benchmark datasets. - Liang et al. (2023) Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. 2023. Holistic evaluation of language models. - Lipton (2017) Peter Lipton. 2017. Inference to the best explanation. A Companion to the Philosophy of Science, pages 184–193. - Lombrozo (2012) Tania Lombrozo. 2012. Explanation and abductive inference. Oxford handbook of thinking and reasoning, pages 260–276. - Mackonis (2013) Adolfas Mackonis. 2013. Inference to the best explanation, coherence and other explanatory virtues. Synthese, 190(6):975–995. - Nie et al. (2019) Yixin Nie, Haonan Chen, and Mohit Bansal. 2019. Combining fact extraction and verification with neural semantic matching networks. In Association for the Advancement of Artificial Intelligence (AAAI). - Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics. - Olausson et al. (2023) Theo X. Olausson, Alex Gu, Benjamin Lipkin, Cedegao E. Zhang, Armando Solar-Lezama, Joshua B. Tenenbaum, and Roger Levy. 2023. Linc: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. - Parcalabescu and Frank (2024) Letitia Parcalabescu and Anette Frank. 2024. On measuring faithfulness or self-consistency of natural language explanations. - Pei and Jurgens (2021) Jiaxin Pei and David Jurgens. 2021. Measuring sentence-level and aspect-level (un)certainty in science communications. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9959–10011, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. - Popper (2014) Karl Popper. 2014. Conjectures and refutations: The growth of scientific knowledge. routledge. - Quan et al. (2024) Xin Quan, Marco Valentino, Louise A Dennis, and André Freitas. 2024. Enhancing ethical explanations of large language models through iterative symbolic refinement. arXiv preprint arXiv:2402.00745. - R Core Team (2013) R Core Team. 2013. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. - Sober (1981) Elliott Sober. 1981. The principle of parsimony. The British Journal for the Philosophy of Science, 32(2):145–156. - Srivastava et al. (2023) Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, and Ziyi Wu. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. - Thagard (1989) Paul Thagard. 1989. Explanatory coherence. Behavioral and brain sciences, 12(3):435–467. - Thagard (1978) Paul R Thagard. 1978. The best explanation: Criteria for theory choice. The journal of philosophy, 75(2):76–92. - Thayaparan et al. (2020) Mokanarangan Thayaparan, Marco Valentino, and André Freitas. 2020. A survey on explainability in machine reading comprehension. arXiv preprint arXiv:2010.00389. - Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. - Valentino and Freitas (2022) Marco Valentino and André Freitas. 2022. Scientific explanation and natural language: A unified epistemological-linguistic perspective for explainable ai. arXiv preprint arXiv:2205.01809. - Valentino et al. (2021) Marco Valentino, Ian Pratt-Hartmann, and André Freitas. 2021. Do natural language explanations represent valid logical arguments? verifying entailment in explainable NLI gold standards. In Proceedings of the 14th International Conference on Computational Semantics (IWCS), pages 76–86, Groningen, The Netherlands (online). Association for Computational Linguistics. - Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. CoRR, abs/1905.00537. - Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. - Weber et al. (2019) Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, and Tim Rocktäschel. 2019. Nlprolog: Reasoning with weak unification for question answering in natural language. - Wei et al. (2022a) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022a. Chain of thought prompting elicits reasoning in large language models. CoRR, abs/2201.11903. - Wei et al. (2022b) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022b. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837. - Wiegreffe and Marasovic (2021) Sarah Wiegreffe and Ana Marasovic. 2021. Teach me to explain: A review of datasets for explainable natural language processing. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1). - Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. - Wu et al. (2022) Yuhuai Wu, Albert Q. Jiang, Wenda Li, Markus N. Rabe, Charles Staats, Mateja Jamnik, and Christian Szegedy. 2022. Autoformalization with large language models. - Xiang (2023) Chloe Xiang. 2023. Openai’s gpt-4 is closed source and shrouded in secrecy. - Xu et al. (2023) Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu, and Erik Cambria. 2023. Are large language models really good logical reasoners? a comprehensive evaluation and beyond. - Zheng et al. (2023) Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Haotong Zhang, Joseph Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. ArXiv, abs/2306.05685. - Zhou et al. (2023) Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023. Least-to-most prompting enables complex reasoning in large language models. ## Appendix A Appendix ### A.1 Reproducibility All experimental code is available online https://github.com/dhairyadalal/IBE-eval to encourage future research in the field. We additionally summarize all the model implementations and technical resources used for the computation of the proposed IBE criteria below: - We adopt the Prolog solver for neuro-symbolic integration from Quan et al. (2024). - We use Spacy Honnibal and Montani (2017) to tokenize and extract part-of-speech (POS) tags. - To compute coherence, we employ the RoBERTa-based NLI model Nie et al. (2020) that has been finetuned on a range of NLI and fact verification datasets consisting of SNLI Bowman et al. (2015), aNLI Nie et al. (2020), multilingual NLI Williams et al. (2018)), and FEVER-NLI Nie et al. (2019). - To measure sentence-level uncertainty, we employ a finetuned RoBERTa model provided by Pei and Jurgens (2021). - We use a fine-tuned BERT-based token classification model to classify all the words in the generated explanation with uncertainty categories introduced in the 2010 CoNLL shared task on Hedge Detection Farkas et al. (2010). ### A.2 Explanation Prompting <details> <summary>extracted/6246183/eev.png Details</summary>  ### Visual Description ## Table: Cause and Effect Prediction Examples ### Overview The table presents two types of logical reasoning tasks: **Cause Prediction** and **Effect Prediction**. Each type includes a contextual scenario, a question, multiple-choice options, and entailment forms demonstrating how premises lead to conclusions. ### Components/Axes - **Columns**: 1. **Type**: Specifies the reasoning task (Cause Prediction, Effect Prediction). 2. **Example**: Provides a contextual scenario and question. 3. **Entailment Forms**: Shows premise-conclusion pairs for each option. ### Detailed Analysis #### Cause Prediction - **Context**: "The balloon expanded." - **Question**: "What was the cause?" - **Option A**: "I blew into it." - **Option B**: "I pricked it." - **Entailment Forms**: - Premise: "I blew into it." → Conclusion: "The balloon expanded." - Premise: "I pricked it." → Conclusion: "The balloon expanded." #### Effect Prediction - **Context**: "The child punched the stack of blocks." - **Question**: "What was the effect?" - **Option A**: "The stack towered over the boy's head." - **Option B**: "The blocks scattered all over the rug." - **Entailment Forms**: - Premise: "The child punched the stack of blocks." → Conclusion: "The stack towered over the boy's head." - Premise: "The child punched the stack of blocks." → Conclusion: "The blocks scattered all over the rug." ### Key Observations 1. **Cause Prediction** focuses on identifying the cause of an event (balloon expansion) with two plausible causes (blowing vs. pricking). 2. **Effect Prediction** explores outcomes of an action (punching blocks) with two possible effects (stack toppling or scattering). 3. Each entailment form explicitly links a premise to a conclusion, illustrating causal or consequential relationships. ### Interpretation This table demonstrates how logical reasoning tasks can be structured to test understanding of cause-effect relationships. The **Cause Prediction** row emphasizes identifying antecedent events, while **Effect Prediction** highlights consequences. The entailment forms reveal that multiple premises can lead to the same conclusion (e.g., both blowing and pricking cause balloon expansion) or that a single premise can result in multiple outcomes (e.g., punching blocks leads to two distinct effects). The structure suggests applications in natural language processing, cognitive science, or educational tools for teaching logical deduction. </details> Figure 7: To perform IBE we convert the CQA context and answer candidates into an entailment form (i.e., EEV) Valentino et al. (2021). <details> <summary>extracted/6246183/cot-prompt.png Details</summary>  ### Visual Description ## Text-Based Scenario Analysis: Property Banishment Causality ### Overview The image contains a structured logical reasoning exercise. It presents a scenario where a woman bans children from her property, followed by a question asking for the most plausible cause. Two options are provided, each with a step-by-step explanation. The task requires evaluating causal assumptions and selecting the most plausible option. ### Components/Axes - **Context**: "The woman banished the children from her property." - **Question**: "What was the cause?" - **Options**: - (a) The children trampled through her garden. - (b) The children hit a ball into her yard. - **Structure**: - **Option 1 Explanation**: Premise (children trampling), conclusion (banishment), and causal steps (damage → anger → banishment). - **Option 2 Explanation**: Premise (ball in yard), conclusion (banishment), and causal steps (invasion of privacy → anger → banishment). - **Answer**: Explicitly states the selected option. ### Detailed Analysis #### Option 1 Explanation - **Premise**: Children trampled through her garden. - **Conclusion**: Banishment from property. - **Steps**: 1. **IF-THEN Rule**: Trampling → garden damage. 2. **Assumption**: Damage causes upset/anger. 3. **Conclusion**: Banishment to prevent further damage. #### Option 2 Explanation - **Premise**: Children hit a ball into her yard. - **Conclusion**: Banishment from property. - **Steps**: 1. **IF-THEN Rule**: Ball in yard → invasion of privacy. 2. **Assumption**: Invasion of privacy causes anger. 3. **Conclusion**: Banishment as a reaction. ### Key Observations - Both options use **IF-THEN causal logic** but differ in the nature of the premise (physical damage vs. perceived intrusion). - **Option 1** emphasizes tangible harm (garden damage), while **Option 2** focuses on subjective perception (privacy invasion). - The **Answer** explicitly selects **(a)**, prioritizing physical damage over abstract intrusion. ### Interpretation The exercise demonstrates how **causal reasoning** hinges on the perceived severity of a premise. Option 1 is deemed more plausible because trampling directly damages property, which is a universally recognized justification for anger and punitive action. Option 2, while valid, relies on the assumption that a ball in a yard constitutes an invasion of privacy—a less universally agreed-upon premise. The structure mirrors **Peircean abductive reasoning**, where the most likely explanation is selected based on observed effects (banishment) and plausible causes. **Final Answer**: (a) the children trampled through her garden. </details> Figure 8: An example of the modified CoT prompt template for explanation generation. A modified CoT prompt is used to instruct the LLM to generate explanations. The prompt includes a set of instructions for explanation generation and an in-context example. Appended to the end of the prompt are the CQA context, causal question, and answer candidates. The LLM is instructed to first convert the options into the EEV format consisting of a premise and conclusion. The EEV format will differ depending on the directionality of the causal question (see Figure 7). Cause prediction questions will treat the answer candidate as the premise and the context as the conclusion. In contrast, effect prediction reverses the relationship treating the context as the premise and the answer options as the conclusion. After the EEV conversion, the model is instructed to generate a step-by-step explanation consisting of IF-THEN statements and the associated causal or commonsense assumptions. For ease of post-processing, the LLM is instructed to use headers and enumerate steps using the Step # format. A full example of the prompt template is exhibited in Figure 8. <details> <summary>extracted/6246183/autoformalization.png Details</summary>  ### Visual Description ## Textual Content Extraction: Prolog Syntax Conversion Guidelines ### Overview The image contains technical instructions for converting natural language premises, conclusions, and explanations into Prolog syntax. It includes formatting rules, examples, and constraints for logical representation. ### Components/Axes 1. **Instructions Section** - Directives for converting premises, conclusions, and explanations into Prolog syntax - Rules for goal generation, fact extraction, and rule derivation - Constraints on variable usage and predicate structure 2. **Example 1** - **Premise**: "Tom's pancreas was injured." - **Conclusion**: "He has a high blood sugar level." - **Explanation**: Logical steps connecting pancreas injury to insulin dysfunction and blood sugar levels - **Goal**: `has_high_blood_sugar(tom)` - **Formal Goal**: `has_high_blood_sugar(X) :- tom(X)` - **Facts**: - `injured_pancreas(tom)` - `tom(tom)` - **Rules**: - `dysfunctional_pancreas(X) :- injured_pancreas(X)` - `reduced_insulin_production(X) :- dysfunctional_pancreas(X)` - `has_high_blood_sugar(X) :- reduced_insulin_production(X)` 3. **Example 2** - Incomplete placeholder marked with `[....]` ### Detailed Analysis - **Syntax Constraints**: - Single-variable predicates only (e.g., `intoxicated(X)` invalid, `intoxicated(X, main)` invalid) - No multiple constants in goals (e.g., `leaking(water_pipe, frozen)` invalid) - Consistent variable referencing across goals and facts - **Logical Flow in Example 1**: 1. **Premise** → `injured_pancreas(tom)` 2. **Rule Application**: - `dysfunctional_pancreas(tom)` (from `injured_pancreas(tom)`) - `reduced_insulin_production(tom)` (from `dysfunctional_pancreas(tom)`) 3. **Conclusion** → `has_high_blood_sugar(tom)` (from `reduced_insulin_production(tom)`) - **Formalization**: - Natural language statements mapped to Prolog predicates with explicit variable binding (e.g., `tom(X)` links "Tom" to variable `X`). ### Key Observations - **Rule Hierarchy**: - Injuries → Dysfunction → Reduced insulin → High blood sugar (causal chain). - **Placeholder in Example 2**: - Indicates incomplete or reserved space for additional examples. - **Variable Binding**: - All facts and rules use consistent variable naming (e.g., `X` in `has_high_blood_sugar(X)`). ### Interpretation The document outlines a formal method for translating medical reasoning into Prolog logic. The constraints ensure syntactic validity, while the examples demonstrate how causal relationships (e.g., pancreas injury → blood sugar levels) are structured as logical rules. The incomplete Example 2 suggests potential expansion for further scenarios. The emphasis on single-variable predicates and consistent variable usage aligns with Prolog's unification mechanism, enabling automated inference from premises to conclusions. </details> Figure 9: An example of the autoformalization prompt. ### A.3 Autoformalization Autoformalization is the process of translating natural language descriptions into formal specifications Wu et al. (2022). We adopt the translational capabilities of GPT-3.5-Turbo to convert the explanation into a formal entailment hypothesis. The IF-THEN explanation steps are converted into a set of Prolog rules, the entailment description is used to generate Prolog atoms, and the conclusion statement is translated into a Prolog query. We provide an example of the autoformalization prompt in Figure 9 and an example of the formalized output in Figure 11. After autoformalization, we deploy a post-processing script to extract the formalized rules, atoms, and query and generate a Prolog program for entailment verification. ### A.4 LLM-as-a-Judge Baseline <details> <summary>extracted/6246183/llm-judge-prompt.png Details</summary>  ### Visual Description ## Text-Based Prompt: Explanation Evaluation Task ### Overview The image contains a textual prompt instructing the evaluation of two explanations (Explanation 1 and Explanation 2) for logical consistency and correctness. It emphasizes that a "good explanation" must be logically consistent and arrive at the correct conclusion. The user is directed to respond with either "explanation 1" or "explanation 2" as the final answer. ### Components/Axes - **Textual Structure**: - Header: Instructions for evaluating explanations. - Placeholders: - `Explanation 1: ...` (incomplete, marked with ellipses). - `Explanation 2: ...` (incomplete, marked with ellipses). - Footer: Prompt for the final answer ("Answer:"). ### Detailed Analysis - **Text Content**: - The prompt explicitly defines criteria for evaluation: logical consistency and correctness of conclusion. - Placeholders (`...`) indicate missing content for both explanations, suggesting the image is a template or incomplete example. - No numerical data, charts, or diagrams are present. ### Key Observations - The image lacks substantive content for Explanation 1 and Explanation 2, rendering them non-evaluable in their current state. - The task is framed as a logical reasoning exercise but provides no actual data to analyze. ### Interpretation This image appears to be a **template or instructional framework** for evaluating explanations, not a dataset or analytical result. The absence of completed explanations (`...`) implies it is either a placeholder for further input or a demonstration of the evaluation structure. The emphasis on logical consistency aligns with principles of deductive reasoning, but without concrete examples, no meaningful analysis can be performed. The prompt’s design suggests it may be part of a larger system for automated or human evaluation of argumentative content. </details> Figure 10: An example of prompt used by the LLM-as-a-Judge model for evaluating competing explanations. GPT 3.5 is used as the LLM for the LLM-as-a-Judge baseline. Similar to the human evaluators, GPT is presented with both generated explanations and asked to identify which explanation is more plausible. ### A.5 Logical Consistency <details> <summary>extracted/6246183/logical-pipeline.png Details</summary>  ### Visual Description ## Table: Logical Proof Structure for Balloon Expansion ### Overview The image presents a three-column table demonstrating the transformation of a natural language statement into a formal logical proof. It illustrates the process of deriving a conclusion ("The balloon expanded") from a premise ("I blew into it") using intermediate steps, formalized rules, and a generated proof. --- ### Components/Axes 1. **Columns**: - **Column 1 (Explanation)**: Contains natural language reasoning steps. - **Column 2 (Formalized Output)**: Contains Prolog-like logic rules and queries. - **Column 3 (Generated Proof)**: Shows step-by-step logical derivation using the rules. 2. **Key Elements**: - **Premise/Conclusion**: Natural language statements. - **Steps 1-2**: Conditional reasoning with assumptions. - **Prolog Query**: `expanded_balloon(me).` - **Atoms**: `blew_into_balloon(me).` - **Rules**: - `inflated_balloon(X) :- blew_into_balloon(X).` - `expanded_balloon(X) :- inflated_balloon(X).` --- ### Detailed Analysis #### Column 1: Explanation - **Premise**: "I blew into it." - **Conclusion**: "The balloon expanded." - **Step 1**: - **If-Then**: "IF someone blows into a balloon, THEN it can cause the balloon to inflate." - **Assumption**: Blowing air increases internal air volume, leading to inflation. - **Step 2**: - **If-Then**: "IF the balloon inflates, THEN it can cause the balloon to expand." - **Assumption**: Inflation stretches the balloon, increasing its size. - **Conclusion**: Combines Steps 1 and 2 to derive the final conclusion. #### Column 2: Formalized Output - **Prolog Query**: `expanded_balloon(me).` *(Asks if the balloon expanded for "me".)* - **Program**: - **Atoms**: `blew_into_balloon(me).` *(Formalizes the premise.)* - **Rules**: 1. `inflated_balloon(X) :- blew_into_balloon(X).` *(If X blew into the balloon, then X inflated it.)* 2. `expanded_balloon(X) :- inflated_balloon(X).` *(If X inflated the balloon, then X expanded it.)* #### Column 3: Generated Proof - **Logical Derivation**: 1. `expanded_balloon(me) ->` *(Starts with the query.)* 2. `expanded_balloon(X) :- inflated_balloon(X).` *(Applies Rule 2 with X = "me".)* 3. `inflated_balloon(X) :- blew_into_balloon(X).` *(Applies Rule 1 with X = "me".)* 4. `blew_into_balloon(me).` *(Uses the atom from Column 2.)* 5. **Final Result**: `blew_into_balloon(me).` *(Proves the premise leads to the conclusion via the rules.)* --- ### Key Observations 1. **Logical Flow**: The proof mirrors the natural language explanation, showing how each step corresponds to a rule. 2. **Modus Ponens**: The proof uses this inference rule to chain premises and rules. 3. **Variable Binding**: The variable `X` is consistently bound to "me" throughout the derivation. --- ### Interpretation This table demonstrates **automated theorem proving** in logic programming. It shows how a natural language statement ("I blew into it") can be translated into formal logic (Prolog rules) and used to derive a conclusion ("The balloon expanded"). The process highlights: - **Abduction**: Inferring causes from effects (e.g., "I blew into it" → "The balloon expanded"). - **Rule-Based Reasoning**: Using predefined rules to connect observations to conclusions. - **Formalization**: Translating assumptions into executable logic (e.g., `blew_into_balloon(me)`). The structure is critical for AI systems that require explicit, machine-readable reasoning steps to validate conclusions. </details> Figure 11: An example of the autoformalization prompt. An explanation hypothesis is considered logically consistent if the external solver can build a deductive proof connecting the conclusion to the premise. We use NLProlog Weber et al. (2019), a neuro-symbolic Prolog solver integrating backward chaining with word embedding models via a weak unification mechanism. NLProlog allows for a level of flexibility and robustness that is necessary for NLP use cases (e.g. unification applied to synonyms). We provide the autoformalized query, atoms, and rules to NLProlog. If NLProlog can satisfy the entailment query, it will return the proof consisting of the set of rules traversed, the weak unification score, and the proof depth. For simplicity, we assign a score of one if the entailment query is satisfied and zero if it is not. The proof depth score is evaluated as part of the parsimony analysis. An end-to-end example of consistency evaluation can be found in Figure 11. 1 Input : Symbolic KB $kb$ , Goal $goal$ , Glove embedding model $e(\cdot)$ Output : proof chain $chain$ , proof depth $depth$ 2 3 threshold $\leftarrow$ 0.13; 4 5 $depth$ $\leftarrow$ 1 ; 6 $chain$ $\leftarrow$ $emptylist$ ; 7 8 foreach step $t$ in backward_chaining( $kb$ , $goal$ ) do 9 foreach $max\_unification(q,q_{t})$ do 10 $unification\_score$ $\leftarrow$ $CosineSimilarity$ ( $e(q,m_{s}),e(q_{t},m_{s})$ ); 11 $depth$ $\leftarrow$ $depth$ $\times$ $unification\_score$ ; 12 13 end foreach 14 $chain$ $\leftarrow$ backward_chaining( $kb$ , $goal$ ); 15 16 end foreach 17 18 if $chain$ is not empty and $depth$ $>$ threshold then 19 $chain\leftarrow$ current_proof_chain $[0]$ ; 20 21 end if 22 else 23 $depth$ $\leftarrow$ 0 ; 24 end if 25 26 return $chain$ , $depth$ ; Algorithm 1 Neuro-symbolic Solver ### A.6 Parsimony Parsimony measures the complexity of an explanation and is represented by the proof depth and concept drift metrics. Proof depth is automatically calculated by NLProlog and reflects the number of rules traversed by the solver to satisfy the entailment query. If the hypothesis is not logically consistent, depth is set to zero. The concept drift metric measures the entropy of novel concepts introduced to bridge the premise and conclusion. To compute the drift of an explanation, we consider the nouns found in the premise, conclusion, and explanation steps. We use Spacy Honnibal and Montani (2017) to tokenize and extract part-of-speech (POS) tags. All tokens with the ’NOUN’ POS tag extracted. For normalization purposes, we consider the lemma of the tokens. Concept drift then is calculated as the set difference between the unique nouns found across all explanation steps and those found in the premise and conclusion. 1 2 Input : Premise, Conclusion, Explanation, Spacy model $spacy$ ( $\cdot$ ) Output : Drift Score $drift$ 3 4 $Noun_{p}\leftarrow$ $spacy$ ( $Premise$ ); 5 $Noun_{c}\leftarrow$ $spacy$ ( $Conclusion$ ); 6 $Noun_{E}\leftarrow$ $spacy$ ( $Explanation$ ); 7 $N\leftarrow\{Noun_{p},Noun_{c},Noun_{E}\}$ ; 8 $drift\leftarrow length(set(Noun_{E})-set(Noun_{p}\cup Noun_{c})$ ); 9 return $drift$ ; Algorithm 2 Concept Drift ### A.7 Coherence Coherence evaluates the plausibility of the intermediate explanation. We propose stepwise entailment as a metric to measure the entailment strength of the If-then implications. We employ a RoBERTa-based NLI model Nie et al. (2020) that has been finetuned on a range of NLI and fact verification datasets consisting of SNLI Bowman et al. (2015), aNLI Nie et al. (2020), multilingual NLI Williams et al. (2018)), and FEVER-NLI Nie et al. (2019). To compute the stepwise entailment score, we first measure the entailment strength between the If and Then propositions. For example, to calculate the score of the statement “IF a balloon is pricked, THEN the balloon may deflate” we consider “a balloon is pricked” and “the balloon may deflate” as input sentences for the NLI model. The NLI will produce independent scores for the entailment and contradiction labels. We compute the entailment strength by subtracting the contraction label score from the entailment label score. An entailment strength of one indicates the If-then implication is strongly plausible whereas a score of zero suggests that it is likely implausible. The overall stepwise entailment score is the average of entailment strength measures across all explanation steps. 1 2 Input : Explanation $E$ , NLI Model $nli$ ( $\cdot$ ) Output : Average Entailment Strength $strength$ 3 4 $EntailmentStrengthScores\leftarrow$ empty list; 5 6 foreach Step $(If_{s},Then_{s})$ in $E$ do 7 $EntailmentScore\leftarrow$ $nli$ ( $If_{s}$ , $Then_{s}$ ); 8 $ContradictionScore\leftarrow$ $nli$ ( $If_{s}$ , $Then_{s}$ ); 9 $EntailmentStrength\leftarrow EntailmentScore-ContradictionScore$ ; 10 Append $EntailmentStrength$ to $EntailmentStrengthScores$ ; 11 12 end foreach 13 14 $strength\leftarrow$ $Avg$ ( $EntailmentStrengthScores$ ); 15 return $strength$ ; Algorithm 3 Stepwise Entailment ### A.8 Linguistic Uncertainty Linguistic uncertainty measures the confidence of a statement where hedging cues and indirect language suggest ambiguity around the proposition. To measure sentence-level uncertainty, we employ a finetuned RoBERTa model provided by Pei and Jurgens (2021). The model was trained on a sentence-level dataset consisting of findings and statements extracted from new articles and scientific publications and human annotated evaluation of sentence certainty. A scale from one to six was used to annotate sentences where one corresponds to the lowest degree of certainty and six is the highest expressed by the sentence. We invert the scale to retrieve the uncertainty scores. To compute the overall linguistic uncertainty of an explanation, we first compute the uncertainty for each assumption and the explanation summary and then average all the scores. We use a fine-tuned BERT-based token classification model to classify all the words in the generated explanation with uncertainty categories introduced in the 2010 CoNLL shared task on Hedge Detection Farkas et al. (2010). Farkas et al. (2010) classify hedge cues into three categories: epistemic, doxatic, and conditional. Epistemic cues refer to hedging scenarios where the truth value of a proposition can be determined but is unknown in the present (e.g. the blocks may fall). Doxatic cues refer to beliefs and hypotheses that can be held to be true or false by others (e.g. the child believed the blocks would fall). Finally, conditional cues refer to propositions whose truth value is dependent on another proposition’s truth value (e.g. if the balloon is pricked it may deflate). 1 2 Input : Assumptions, Explanation Summary, Uncertainty Estimator Model $uc(\cdot)$ Output : Overall Uncertainty 3 4 $AssumptionUncertaintyList\leftarrow$ empty list; 5 6 foreach Assumption in Assumptions do 7 $UncertaintyScore\leftarrow$ $uc$ ( $UncertaintyModel$ , $Assumption$ ); 8 Append $UncertaintyScore$ to $AssumptionUncertaintyList$ ; 9 10 end foreach 11 12 $AverageAssumptionUncertainty\leftarrow$ $Avg$ ( $AssumptionUncertaintyList$ ); 13 14 $ExplanationUncertainty\leftarrow$ $uc$ ( $UncertaintyModel$ , $ExplanationSummary$ ); 15 16 $OverallExplanationUncertainty\leftarrow AverageAssumptionUncertainty+ExplanationUncertainty$ ; 17 18 return $OverallExplanationUncertainty$ ; Algorithm 4 Linguistic Uncertainty ### A.9 Inference to Best Explanation To perform IBE, we first fit a linear regression model over the extracted explanation features from the COPA train set and 500 random sample train examples from the E-CARE train set. We consider all explanations independently and annotate each explanation with a 1 if it corresponds to a correct answer or 0 if corresponds to an incorrect answer. After the linear model is fitted, we evaluate the COPA and E-CARE test sets. For each example, we use the trained linear model to score each answer candidate explanation and then select a candidate with the highest score. We use the linear regression implementation from scikit-learn Buitinck et al. (2013) for the IBE model. We additionally use the R stats package R Core Team (2013) for conducting our regression analysis. ### A.10 E-CARE Results #### A.10.1 E-CARE Consistency See Figure 12. <details> <summary>extracted/6246183/consistency-ecare.png Details</summary>  ### Visual Description ## Bar Chart: E-CARE Avg. Consistency ### Overview The chart compares the average logical consistency of three AI models (Llama 2 7B, Llama 2 13B, and ChatGPT) across two categories: "correct" and "incorrect" responses. Horizontal bars represent percentages, with green for "correct" and red for "incorrect." ### Components/Axes - **X-axis**: "% Logically Consistent" (scale: 0% to 100%, labeled in increments of 10%). - **Y-axis**: Model names (Llama 2 7B, Llama 2 13B, ChatGPT), ordered from top to bottom. - **Legend**: - Green: "correct" responses. - Red: "incorrect" responses. - **Text Embedded in Bars**: Percentages for each category (e.g., "80%", "74%"). ### Detailed Analysis 1. **Llama 2 7B**: - Correct: 80% (green bar). - Incorrect: 74% (red bar). 2. **Llama 2 13B**: - Correct: 74% (green bar). - Incorrect: 70% (red bar). 3. **ChatGPT**: - Correct: 78% (green bar). - Incorrect: 71% (red bar). ### Key Observations - **Llama 2 7B** has the highest consistency in both "correct" (80%) and "incorrect" (74%) responses, with the smallest gap (6%) between the two categories. - **Llama 2 13B** shows the lowest "correct" consistency (74%) but also the lowest "incorrect" consistency (70%), with a 4% gap. - **ChatGPT** has the most balanced performance, with a 7% gap between "correct" (78%) and "incorrect" (71%). ### Interpretation The data suggests that **Llama 2 7B** is the most consistent overall, though its "correct" and "incorrect" responses are closely matched. **Llama 2 13B** exhibits lower consistency in both categories but with a narrower gap, potentially indicating more polarized performance. **ChatGPT** demonstrates higher accuracy in "correct" responses but slightly lower consistency in "incorrect" responses compared to Llama 2 models. The chart highlights trade-offs between accuracy and consistency across models, with Llama 2 7B prioritizing consistency over accuracy and ChatGPT leaning toward higher correctness. </details> Figure 12: Average consistency comparison between correct and incorrect options for the E-CARE dataset. #### A.10.2 E-CARE Proof Depth See Figure 13. <details> <summary>extracted/6246183/depth-ecare.png Details</summary>  ### Visual Description ## Bar Chart: E-CARE: Avg. Proof Depth ### Overview The chart compares the average proof depth for three language models (Llama 2 7B, Llama 2 13B, and ChatGPT) across two categories: "correct" (green) and "incorrect" (red) options. The x-axis represents "Depth," while the y-axis lists the models. Numerical values are embedded in the bars, with green bars showing lower depths for correct answers and red bars showing higher depths for incorrect answers. ### Components/Axes - **Title**: "E-CARE: Avg. Proof Depth" - **Y-Axis (Categories)**: - Llama 2 7B - Llama 2 13B - ChatGPT - **X-Axis (Scale)**: Labeled "Depth" (numerical values from ~1.8 to 2.2). - **Legend**: - Green: "correct" - Red: "incorrect" - **Bar Structure**: - Each model has two bars (green for correct, red for incorrect). - Values are placed on top of the bars (e.g., "1.86" for Llama 2 7B correct). ### Detailed Analysis - **Llama 2 7B**: - Correct: 1.86 (green) - Incorrect: 2.03 (red) - **Llama 2 13B**: - Correct: 2.15 (green) - Incorrect: 2.21 (red) - **ChatGPT**: - Correct: 1.98 (green) - Incorrect: 2.18 (red) ### Key Observations 1. **Incorrect depths exceed correct depths** for all models, indicating higher average proof depth for incorrect answers. 2. **Llama 2 13B** has the highest incorrect depth (2.21) but also the highest correct depth (2.15), suggesting it performs better on correct answers despite higher incorrect depths. 3. **ChatGPT** has the lowest correct depth (1.98) and the second-highest incorrect depth (2.18), indicating moderate performance. 4. **Llama 2 7B** has the lowest correct depth (1.86) and the lowest incorrect depth (2.03), suggesting it struggles more with correct answers but performs better on incorrect ones. ### Interpretation The data suggests that larger models (e.g., Llama 2 13B) may have higher proof depths for both correct and incorrect answers, potentially due to increased complexity in reasoning. However, Llama 2 13B's higher correct depth (2.15) compared to its incorrect depth (2.21) implies it is more accurate in its reasoning. ChatGPT's performance is intermediate, with a lower correct depth but higher incorrect depth than Llama 2 7B. The chart highlights a trade-off between model size and accuracy, with larger models excelling in correct answers but also showing higher variability in incorrect responses. The values are approximate, and the trends align with the visual representation of bar lengths. </details> Figure 13: Comparison of average proof depth between correct and incorrect options. <details> <summary>extracted/6246183/drift-ecare.png Details</summary>  ### Visual Description ## Bar Chart: E-CARE: Avg. Concept Drift ### Overview The chart compares average concept drift values for three AI models (Llama 2 7B, Llama 2 13B, and ChatGPT) across two option types: "correct" (green) and "incorrect" (red). Values represent drift magnitudes, with higher numbers indicating greater deviation from baseline performance. ### Components/Axes - **Vertical Axis (Categories)**: - Models: Llama 2 7B, Llama 2 13B, ChatGPT (listed top to bottom) - **Horizontal Axis (Values)**: - Labeled "Drift" with no explicit scale, but values range from ~3.6 to 5.87 - **Legend**: - Right-aligned, associates: - Green = "correct" option type - Red = "incorrect" option type - **Bars**: - Grouped horizontally for each model, with numerical labels inside bars ### Detailed Analysis 1. **Llama 2 7B**: - Correct: 5.37 (green bar) - Incorrect: 5.87 (red bar) 2. **Llama 2 13B**: - Correct: 5.19 (green bar) - Incorrect: 5.8 (red bar) 3. **ChatGPT**: - Correct: 3.61 (green bar) - Incorrect: 5.02 (red bar) ### Key Observations - **Inverse Relationship**: For all models, "incorrect" drift values exceed "correct" drift values, suggesting models are more sensitive to drift in incorrect options. - **Model Performance**: - Llama 2 7B shows the highest overall drift (5.87 incorrect). - ChatGPT has the lowest correct drift (3.61) but still exhibits higher incorrect drift (5.02). - **Scale Consistency**: All values cluster between 3.6 and 5.9, indicating relatively similar drift magnitudes across models. ### Interpretation The data suggests that AI models exhibit greater performance degradation when evaluating incorrect options compared to correct ones under concept drift. Llama 2 7B demonstrates the most pronounced vulnerability to drift in incorrect options, while ChatGPT maintains the lowest correct drift but still suffers from significant incorrect drift. This pattern may reflect architectural differences in handling ambiguity or error tolerance. The consistent trend across models highlights a potential systemic challenge in maintaining accuracy under evolving data conditions. </details> Figure 14: Comparison of average concept drift between correct and incorrect options. <details> <summary>extracted/6246183/coherence-ecare.png Details</summary>  ### Visual Description ## Bar Chart: E-CARE: Avg. Coherence ### Overview The chart compares average coherence scores for three language models (ChatGPT, Llama 2 13B, Llama 2 7B) across two categories: "Correct" (green) and "Incorrect" (red) responses. A red dashed line labeled "Rel. Difference %" overlays the chart, showing a trend from 0% to 40%. ### Components/Axes - **X-axis**: Model names (ChatGPT, Llama 2 13B, Llama 2 7B), evenly spaced. - **Y-axis (left)**: Coherence Score (0.0–0.4), linear scale. - **Legend**: - Green = "Correct" responses - Red = "Incorrect" responses - **Secondary Y-axis (right)**: "Rel. Difference %" (0%–40%), aligned with the red dashed line. ### Detailed Analysis 1. **ChatGPT**: - Correct: ~0.38 - Incorrect: ~0.28 - Red dashed line: 0% (bottom of the chart). 2. **Llama 2 13B**: - Correct: ~0.25 - Incorrect: ~0.20 - Red dashed line: ~15%. 3. **Llama 2 7B**: - Correct: ~0.23 - Incorrect: ~0.18 - Red dashed line: ~25%. ### Key Observations - **Coherence Scores**: ChatGPT outperforms both Llama models in both "Correct" and "Incorrect" categories. - **Relative Difference**: The red dashed line increases from ChatGPT to Llama 2 7B, suggesting a growing disparity between correct and incorrect coherence scores as model size decreases. - **Bar Heights**: All "Correct" bars exceed "Incorrect" bars, but the gap narrows for smaller models (Llama 2 7B). ### Interpretation The data suggests that larger models (e.g., ChatGPT) maintain higher coherence scores overall, with a smaller relative gap between correct and incorrect responses. Smaller models (e.g., Llama 2 7B) show reduced coherence, with a larger relative difference between correct and incorrect scores. The red dashed line ("Rel. Difference %") visually reinforces this trend, though its exact calculation (e.g., percentage difference relative to total, incorrect, or another metric) is not explicitly defined in the chart. This could imply that model size impacts not only absolute coherence but also the consistency of responses across correctness categories. </details> Figure 15: Comparison of average coherence scores between correct and incorrect options. <details> <summary>extracted/6246183/uncertainty-ecare.png Details</summary>  ### Visual Description ## Bar Chart: E-CARE: Avg. Uncertainty ### Overview The chart compares the average uncertainty scores of three language models (ChatGPT, Llama 2 13B, Llama 2 7B) across two categories: "correct" and "incorrect" answers. Uncertainty scores range from 0 to 4, with higher values indicating greater uncertainty. A red dashed line connects the "incorrect" bars across models, highlighting trends in uncertainty for incorrect responses. --- ### Components/Axes - **X-axis**: Model names (ChatGPT, Llama 2 13B, Llama 2 7B). - **Y-axis (left)**: Score (0–4), labeled "Score." - **Legend (right)**: - Green = "correct" answers. - Red = "incorrect" answers. - **Dashed Line**: Red, connects "incorrect" bars across models. --- ### Detailed Analysis 1. **ChatGPT**: - Correct: ~3.5 (green bar). - Incorrect: ~3.8 (red bar). 2. **Llama 2 13B**: - Correct: ~3.6 (green bar). - Incorrect: ~2.7 (red bar, lowest among "incorrect" scores). 3. **Llama 2 7B**: - Correct: ~3.3 (green bar). - Incorrect: ~3.6 (red bar). --- ### Key Observations - **Higher Uncertainty for Incorrect Answers**: All models show higher uncertainty scores for incorrect answers compared to correct ones (e.g., ChatGPT: 3.8 vs. 3.5). - **Llama 2 13B Anomaly**: Its "incorrect" score (~2.7) is significantly lower than the other models, creating a dip in the red dashed line. - **Consistency in Correct Answers**: Correct scores are relatively stable across models (3.3–3.6). --- ### Interpretation The data suggests that language models exhibit greater uncertainty when generating incorrect answers, aligning with expectations. However, Llama 2 13B’s lower uncertainty for incorrect responses (~2.7) is anomalous. This could indicate: - **Better Calibration**: Llama 2 13B may be more confident in its errors, possibly due to training data or architecture differences. - **Error Pattern**: The model might produce fewer "high-uncertainty" errors, suggesting a different failure mode (e.g., overconfidence in incorrect outputs). The red dashed line emphasizes the divergence in uncertainty trends, particularly for Llama 2 13B, warranting further investigation into its error behavior. </details> Figure 16: Comparison of average uncertainty scores between correct and incorrect options. <details> <summary>extracted/6246183/hedge-ratio-ecare.png Details</summary>  ### Visual Description ## Bar Chart: E-CARE: Avg. Ratio of Hedge Cues ### Overview The chart compares the average ratio of hedge cues across three language models (ChatGPT, Llama 2 13B, Llama 2 7B) using grouped bars. A red dashed line overlays the chart to show relative differences between models, with percentages labeled above each data point. ### Components/Axes - **X-axis**: Model names (ChatGPT, Llama 2 13B, Llama 2 7B), evenly spaced. - **Y-axis (left)**: "Ratio" (0.00 to 0.04), linear scale. - **Legend**: - Green bars: "Hedge Cues" (ratio values). - Red bars: "Relative Difference" (percentage values). - **Red dashed line**: Connects data points for relative differences, with labels at ~12%, ~5%, and ~15% for each model. ### Detailed Analysis 1. **ChatGPT**: - Green bar (Hedge Cues): ~0.032. - Red bar (Relative Difference): ~12%. 2. **Llama 2 13B**: - Green bar (Hedge Cues): ~0.040. - Red bar (Relative Difference): ~5%. 3. **Llama 2 7B**: - Green bar (Hedge Cues): ~0.035. - Red bar (Relative Difference): ~22%. ### Key Observations - Llama 2 7B has the highest relative difference (~22%) compared to other models. - Llama 2 13B has the highest hedge cue ratio (~0.040) but the lowest relative difference (~5%). - ChatGPT shows moderate hedge cue usage (~0.032) with a mid-range relative difference (~12%). ### Interpretation The data suggests that larger models (e.g., Llama 2 7B) exhibit higher relative differences in hedge cue usage compared to smaller models (e.g., Llama 2 13B). However, the absolute hedge cue ratio peaks at Llama 2 13B, indicating that model size does not linearly correlate with hedge cue frequency. The red dashed line highlights that relative differences are most pronounced in the Llama 2 7B model, potentially due to architectural differences or training data biases. This could imply that smaller models (like Llama 2 13B) are more consistent in their hedge cue usage, while larger models (Llama 2 7B) show greater variability. </details> Figure 17: Comparison of the average ratio of hedge cues between correct and incorrect options. <details> <summary>extracted/6246183/hedge-distrib-ecare.png Details</summary>  ### Visual Description ## Horizontal Bar Chart: E-CARE: Hedge Cue Distrib. ### Overview The chart compares the distribution of three hedge cue categories (Epistemic, Doxastic, Conditional) across three language models: Llama 2 7B, Llama 2 13B, and ChatGPT. Each model's bar is divided into color-coded segments representing the proportion of each cue type. ### Components/Axes - **X-axis**: Labeled "Percentage" with markers at 0%, 25%, 50%, 75%, and 100%. - **Y-axis**: Lists models (Llama 2 7B, Llama 2 13B, ChatGPT) from top to bottom. - **Legend**: Located at the bottom, mapping colors to categories: - **Blue**: Epistemic - **Green**: Doxastic - **Red**: Conditional ### Detailed Analysis 1. **Llama 2 7B**: - **Epistemic (Blue)**: ~20% of the bar. - **Doxastic (Green)**: ~30% of the bar. - **Conditional (Red)**: ~50% of the bar (largest segment). 2. **Llama 2 13B**: - **Epistemic (Blue)**: ~30% of the bar. - **Doxastic (Green)**: ~40% of the bar. - **Conditional (Red)**: ~30% of the bar (smallest segment among the three models). 3. **ChatGPT**: - **Epistemic (Blue)**: ~30% of the bar. - **Doxastic (Green)**: ~40% of the bar. - **Conditional (Red)**: ~30% of the bar (matches Llama 2 13B's distribution). ### Key Observations - **Conditional cues dominate** in all models, with Llama 2 7B having the highest proportion (~50%). - **Llama 2 13B** shows the most balanced distribution, with Epistemic and Conditional cues nearly equal (~30% each). - **ChatGPT** has the lowest Conditional proportion (~30%) but the highest Epistemic proportion (~30%) among the three models. - All models allocate the largest segment to **Doxastic cues** (30–40%). ### Interpretation The data suggests that **Conditional hedging** is a prevalent strategy across models, particularly in smaller architectures like Llama 2 7B. The **Llama 2 13B** model exhibits a more diversified cue distribution, potentially reflecting architectural complexity or training objectives. **ChatGPT**'s higher Epistemic proportion may indicate a design emphasis on uncertainty expression. The consistent dominance of Doxastic cues across models implies a shared focus on factual grounding, though the exact mechanisms differ by architecture. </details> Figure 18: Distribution of hedge cues across incorrect explanations. #### A.10.3 E-CARE Concept Drift See Figure 13. #### A.10.4 E-CARE Coherence See Figure 15. #### A.10.5 E-CARE Uncertainty See Figure 16. #### A.10.6 E-CARE Hedge Ratio See Figure 17. #### A.10.7 E-CARE Hedge Distribution See Figure 18. ### A.11 Causal Directionality <details> <summary>extracted/6246183/cause-effect-acc.png Details</summary>  ### Visual Description ## Bar Chart: Model Accuracy Comparison by Task Type ### Overview The chart compares the accuracy percentages of three language models (Llama 2 7B, Llama 2 13B, and ChatGPT) across two task types: "Cause" and "Effect." The data is presented as horizontal bars with color-coded categories (green for Cause, blue for Effect) and includes numerical labels for each bar. ### Components/Axes - **Y-Axis**: Model names (Llama 2 7B, Llama 2 13B, ChatGPT), ordered from top to bottom. - **X-Axis**: Accuracy (%) ranging from 0% to 80%. - **Legend**: Located on the right, with green representing "Cause" and blue representing "Effect." - **Bars**: Horizontal bars grouped by model, with two bars per model (one for Cause, one for Effect). ### Detailed Analysis - **Llama 2 7B**: - Cause: 64% (green bar). - Effect: 72% (blue bar). - **Llama 2 13B**: - Cause: 72% (green bar). - Effect: 72% (blue bar). - **ChatGPT**: - Cause: 71% (green bar). - Effect: 80% (blue bar). ### Key Observations 1. **Effect Accuracy Dominance**: All models show higher accuracy for "Effect" tasks compared to "Cause" tasks, except Llama 2 13B, where both task types have equal accuracy (72%). 2. **ChatGPT Superiority**: ChatGPT achieves the highest accuracy overall (80% for Effect), significantly outperforming the Llama models. 3. **Llama Model Disparity**: Llama 2 7B has the lowest Cause accuracy (64%), while Llama 2 13B matches ChatGPT in Cause accuracy (72%). ### Interpretation The data suggests that language models generally perform better at predicting "Effect" outcomes than "Cause" relationships. ChatGPT demonstrates superior performance across both task types, potentially due to its larger scale or training data. The Llama 2 13B model uniquely balances Cause and Effect accuracy, indicating architectural or training optimizations for parity. The disparity in Llama 2 7B's Cause accuracy highlights potential limitations in smaller models for causal reasoning tasks. These trends underscore the importance of model architecture and scale in handling different reasoning paradigms. </details> Figure 19: Accuracy in predicting the most plausible causes vs effects on COPA. When considering the causal directionality (i.e. cause vs effect), we observed that accuracy tended to differ between LLMs on COPA. In particular, we found both GPT and LLaMA 2 7B to be more accurate in predicting the effects in causal scenarios (see Figure 19). We hypothesize that LLMs may suffer the challenge of causal sufficiency as the space of potential causal explanations can be far greater than the range of effects once an event has been observed. This hypothesis is partly supported by the fact that GPT and LLaMA 2 7B express greater linguistic uncertainty and produce more complex explanations when predicting causes rather than effects. ### A.12 Dataset Details COPA is released under a BSD-2 license and made available for broad research usage with copyright notification restrictions people.ict.usc.edu/~gordon/copa.html. We do not modify or use COPA outside of its intended use which is primarily open-domain commonsense causal reasoning. E-CARE is released under the MIT license and can be used for broad purposes with copyright notification restrictions github.com/Waste-Wood/e-CARE?tab=MIT-1-ov-file#readme. We do not modify or use E-CARE outside of its intended use which is causal reasoning evaluation of language models.