2207.04053v4

# On the Need and Applicability of Causality for Fairness: A Unified Framework for AI Auditing and Legal Analysis Ruta Binkyte ruta.binkyte@gmail.com CISPA, Helmholtz Center for Information Security Ljupcho Grozdanovski grozdanovski@uliege.be University of Liège Sami Zhioua sami.zhioua@gmail.com INRIA, Saclay Ile-de-France ## Abstract As Artificial Intelligence (AI) increasingly influences decisions in critical societal sectors, understanding and establishing causality becomes essential for evaluating the fairness of automated systems. This article explores the significance of causal reasoning in addressing algorithmic discrimination, emphasizing both legal and societal perspectives. By reviewing landmark cases and regulatory frameworks, particularly within the European Union, we illustrate the challenges inherent in proving causal claims when confronted with opaque AI decision-making processes. The discussion outlines practical obstacles and methodological limitations in applying causal inference to real-world fairness scenarios, proposing actionable solutions to enhance transparency, accountability, and fairness in algorithm-driven decisions. keywords: Fairness, Discrimination, Causality, Anti-discrimination law ## 1 Introduction Artificial Intelligence (AI) systems increasingly shape decisions across critical sectors, including finance, employment, healthcare, and justice. While AI promises efficiency and objectivity, its deployment also raises significant concerns about algorithmic discrimination, particularly when automated decisions perpetuate biases or unjustly disadvantage specific groups. Central to addressing these concerns is establishing causality—accurately identifying whether and how discriminatory harm arises from algorithmic processes. This article examines the challenges and opportunities involved in applying principles of causality to ensure fairness in AI-driven decision-making from both legal and technical perspectives. The core objective is to develop a unified analytical framework that integrates causal reasoning with procedural fairness, aiding courts and regulators in effectively addressing algorithmic discrimination. To achieve this, the article first explores foundational legal principles regarding causality and fairness, emphasizing procedural safeguards essential for fair adjudication. It then critically reviews landmark cases involving AI discrimination—such as Cook vs. HSBC, Pickett, Loomis, and Ewert—to illustrate the practical difficulties of proving causation when faced with opaque AI systems. Subsequently, the discussion delves into causal inference methodologies, highlighting their potential for establishing direct or indirect evidence of discrimination through techniques such as causal discovery algorithms, counterfactual analysis, and the but-for test. The article also evaluates recent legislative efforts, particularly the European Union’s AI Act and AI Liability Directive, analyzing their impact on evidentiary standards and procedural fairness in algorithmic discrimination cases. Ultimately, by clearly identifying the limitations and assumptions underlying causal approaches, the article outlines practical considerations and future research directions. The proposed framework thus serves as a comprehensive resource for policymakers, legal practitioners, and AI developers committed to fostering transparency, accountability, and fairness in algorithmic decision-making. <details> <summary>x1.png Details</summary>  ### Visual Description ## Diagram: Causal Analysis for Auditing Algorithmic Fairness ### Overview The diagram outlines a three-part framework for auditing algorithmic fairness through causal analysis. It emphasizes distinguishing discrimination from spurious correlations, analyzing variables and proxies, and establishing legal evidence standards. ### Components/Axes 1. **Sections**: - **Measuring Discrimination** (left): Focuses on distinguishing discrimination from spurious correlations. - **Path Specific Analysis** (center): Examines variables and proxies to identify discrimination. - **Legal Evidence** (right): Establishes causal evidence standards for court proceedings. 2. **Icons**: - **Left**: A clipboard with a magnifying glass and scales of justice (symbolizing scrutiny and fairness). - **Center**: Purple dashed arrows forming a bidirectional loop (representing variable-proxy relationships). - **Right**: A courthouse with scales of justice (symbolizing legal frameworks). 3. **Textual Labels**: - Headings: "Measuring Discrimination," "Path Specific Analysis," "Legal Evidence." - Subheadings: - "Distinguishing discrimination from spurious correlations" - "Distinguishing Explaining variables from Proxy discrimination" - "Establishing causal evidence in court" ### Detailed Analysis - **Measuring Discrimination**: - Text emphasizes isolating discrimination from coincidental patterns (spurious correlations). - Icon reinforces the need for rigorous scrutiny (magnifying glass) and fairness (scales). - **Path Specific Analysis**: - Purple dashed arrows suggest a non-linear, bidirectional relationship between variables and proxy discrimination. - Highlights the complexity of tracing causal pathways in algorithms. - **Legal Evidence**: - Focuses on translating technical findings into court-admissible causal evidence. - Icon of a courthouse underscores the need for legal accountability. ### Key Observations - No numerical data or quantitative trends are present; the diagram is conceptual. - The bidirectional arrows in the center section imply feedback loops between variables and proxies. - All icons use blue outlines, with the center section uniquely using purple for emphasis. ### Interpretation This framework bridges technical auditing and legal accountability: 1. **Technical Rigor**: The left and center sections stress methodological precision in identifying discrimination. 2. **Legal Translation**: The right section ensures findings meet judicial standards, addressing gaps in current AI governance. 3. **Visual Symbolism**: - The scales of justice in both the left and right icons reinforce fairness as a recurring theme. - The purple arrows in the center section visually differentiate the analytical complexity from the other sections. The diagram advocates for a holistic approach to algorithmic fairness, combining statistical analysis with legal frameworks to ensure accountability. </details> Figure 1: Causality is crucial for auditing algorithmic fairness because it accurately identifies whether automated systems produce genuinely discriminatory outcomes or merely reflect correlations; helps to disentangle the paths relating sensitive attribute and the outcome and justifiable dependency from proxy discrimination; causal analysis aligns with causal evidence for establishing liability in court-practice. ### 1.1 Related Work Over the last decade, causal fairness has emerged as a important approach in the quest for equitable AI. One of the most influential works in counterfactual fairness, introduced by [Kusner et al. (2017)]. They formalized the idea that a decision is fair “if it is the same in the actual world and in a counterfactual world where the individual belonged to a different demographic group” [Kusner et al. (2017)]. Building on this, subsequent research refined causal fairness notions by distinguishing fair and unfair pathways of influence. [Nabi and Shpitser (2018)] proposed defining discrimination as the presence of a causal effect of a sensitive attribute on the outcome along certain disallowed paths. Similarly, [Chiappa (2019)] introduced a method to compute path-specific counterfactual fairness, explicitly quantifying the information flow along particular causal paths and ensuring that protected attributes do not impact the outcome via forbidden channels . [Zhang and Bareinboim (2018)] further contributed a causal explanation formula to decompose observed outcome disparities into portions attributable to different causal factors, enabling one to explain and quantify unfairness in decision-making. [Loftus et al. (2018b)] argued for the importance of causal reasoning when designing fair algorithms, pointing out that purely correlation-based fairness criteria may be misleading if the data is biased by historical discrimination. [Plečko et al. (2024)] enumerates which causal assumptions are needed to test for disparate impact versus disparate treatment and connect causal fairness to legal disparate treatment framework. Carey and Wu (2022) discusses the relationship between causality-based fairness notions to legal principles, noting that many fairness criteria in machine learning echo concepts from discrimination law when interpreted causally. However, they do not link causal fairness in ML and court practice in the context of AI auditing. ### Main Contributions This paper makes several key contributions to the study of auditing fairness in AI through a causal inference perspective. Specifically, we: - Establish the connection between causal fairness analysis and legal frameworks. We analyze how causal methods align with judicial practices, particularly within the European legal framework, and discuss the admissibility and role of causal evidence in proving algorithmic discrimination in court. - Consolidate the arguments for causality in fairness evaluation. We provide systematization of knowledge on scenarios where purely statistical approaches can misrepresent discrimination. - Discuss limitations of applying causality for fairness evaluation. We critically examine assumptions such as ignorability, positivity, SUTVA, and path-specific identifiability in fairness contexts. By bridging the gap between causal inference, fairness in AI, and legal accountability, this work provides a comprehensive framework for evaluating and mitigating algorithmic discrimination. ### Paper Structure This paper is structured as follows. In sec:background we introduce making concepts of causality; in sec:measure we discuss when causality is crucial for reliably measuring discrimination; In sec:mediation we lay out the importance of mediation analysis in the context of fairness evaluation; In sec:legal we delve into the concept of causality in European law and court practice, and link to causal AI auditing. Finally, we conclude in sec:conclusion. ## 2 Background on Causality In this work, we primarily adopt the structural probabilistic models framework (Pearl, 2009a) and the potential outcomes framework (Rubin, 2005). Below, we provide a technical preliminaries for causal framework and causal fairness notions. ### 2.1 Causal Graph A causal graph, denoted as $\mathcal{G}=(\mathbf{V},\mathcal{E})$ , is a Directed Acyclic Graph (DAG) consisting of a set of variables or nodes $\mathbf{V}$ and edges $\mathcal{E}$ . Each edge $X\to Y$ signifies a causal relationship, meaning changes in $X$ directly influence $Y$ . Importantly, altering $X$ impacts $Y$ , but modifying $Y$ does not affect $X$ . fig:simpleColliderJobHiring shows three basic DAG structures - confounder, mediator and collider. ### 2.2 Causal Fairness Notions Causal fairness aims to ensure that sensitive attributes, such as race or gender, do not unfairly influence outcomes. Below, we describe key causal fairness notions and their formal definitions. #### 2.2.1 Total Effect (TE) Total Effect (TE) (Pearl, 2009b) is a causal fairness notion that quantifies the overall effect of a sensitive attribute $X$ on an outcome $Y$ . Formally, TE is defined as: $$ TE_{x_{1},x_{0}}(y)=\mathbb{P}(Y=y\mid do(X=x_{1}))-\mathbb{P}(Y=y\mid do(X=x_ {0})), $$ where $do(X=x)$ denotes an intervention that sets $X$ to $x$ . TE measures the causal impact of changing $X$ from $x_{0}$ to $x_{1}$ on $Y$ across all causal paths connecting $X$ to $Y$ . #### 2.2.2 Mediation Analysis: NDE, NIE, and PSE Mediation analysis decomposes the causal effect of $X$ on $Y$ into direct and indirect effects. This is essential for identifying the pathways through which $X$ influences $Y$ . Natural Direct Effect (NDE) (Pearl, 2001): The NDE quantifies the direct effect of $X$ on $Y$ , bypassing any mediators. For a binary variable $X$ with values $x_{0}$ and $x_{1}$ , the NDE is: $$ NDE_{x_{1},x_{0}}(y)=\mathbb{P}(y_{x_{1},\mathbf{Z}_{x_{0}}})-\mathbb{P}(y_{x_ {0}}), $$ where $\mathbf{Z}$ represents the set of mediator variables, and $\mathbb{P}(y_{x_{1},\mathbf{Z}_{x_{0}}})$ is the probability of $Y=y$ if $X$ is set to $x_{1}$ while the mediators are set to values they would take under $X=x_{0}$ . Natural Indirect Effect (NIE) (Pearl, 2001): The NIE captures the influence of $X$ on $Y$ through mediators. It is given by: $$ NIE_{x_{1},x_{0}}(y)=\mathbb{P}(y_{x_{0},\mathbf{Z}_{x_{1}}})-\mathbb{P}(y_{x_ {0}}), $$ where $\mathbb{P}(y_{x_{0},\mathbf{Z}_{x_{1}}})$ represents the probability of $Y=y$ when $X=x_{0}$ but mediators take values they would under $X=x_{1}$ . Path-Specific Effect (PSE) (Pearl, 2009b; Chiappa, 2019; Wu et al., 2019): The PSE isolates the causal effect of $X$ on $Y$ transmitted through a specific path or set of paths $\pi$ . Formally, it is defined as: $$ PSE^{\pi}_{x_{1},x_{0}}(y)=\mathbb{P}(y_{x_{1}\mid_{\pi},x_{0}\mid_{\overline{ \pi}}})-\mathbb{P}(y_{x_{0}}), $$ where $\mathbb{P}(y_{x_{1}\mid_{\pi},x_{0}\mid_{\overline{\pi}}})$ is the probability of $Y=y$ if $X=x_{1}$ along path $\pi$ , while other paths ( $\overline{\pi}$ ) remain unaffected by the intervention. ## 3 When Causality is Needed for Reliably measuring discrimination? Measuring discrimination without accounting for the underlying causal structure of variable relationships can lead to misleading conclusions and biased estimations of discrimination. In extreme cases, such as Simpson’s paradox, this bias can even reverse conclusions—for example, a biased estimation may suggest positive discrimination when, in reality, an unbiased analysis would reveal negative discrimination. Figures 4 - 4 illustrate the three fundamental causal structures that can result in statistical anomalies, making common fairness metrics unreliable. <details> <summary>x2.png Details</summary>  ### Visual Description ## Diagram: Cyclical Relationship Between Political Belief, Job Hiring, and Socio-Economic Status (SES) ### Overview The diagram illustrates a cyclical relationship among three interconnected factors: **Political Belief (A)**, **Selection for Job Hiring (Y)**, and **SES (Socio-Economic Status) (Z)**. Arrows indicate directional influence, forming a closed loop: A → Y → Z → A. ### Components/Axes - **Nodes**: - **A**: Political Belief - **Y**: Selection for Job Hiring - **Z**: SES (Socio-Economic Status) - **Arrows**: - A → Y: Political Belief influences Selection for Job Hiring. - Y → Z: Selection for Job Hiring affects SES. - Z → A: SES reinforces or modifies Political Belief. ### Detailed Analysis - **Labels**: All nodes are explicitly labeled with their respective concepts. - **Flow Direction**: - Political Belief (A) directly impacts Job Hiring (Y). - Job Hiring (Y) determines or correlates with SES (Z). - SES (Z) feeds back into Political Belief (A), completing the cycle. - **No Numerical Data**: The diagram lacks quantitative values, scales, or legends. It focuses on conceptual relationships rather than measurable trends. ### Key Observations 1. **Cyclical Dependency**: The system is self-reinforcing, with no clear starting or ending point. 2. **Bidirectional Influence**: While arrows suggest unidirectional flow, the loop implies mutual reinforcement (e.g., SES may shape Political Belief, which then affects Job Hiring). 3. **No Outliers**: All components are equally weighted in the cycle, with no emphasis on dominance or hierarchy. ### Interpretation This diagram suggests a **systemic feedback loop** where socio-economic factors, political beliefs, and employment opportunities are interdependent. For example: - Political beliefs (e.g., support for labor rights) might influence hiring practices (Y), which in turn affect an individual’s SES (Z). - Higher SES (Z) could then enable access to education or networks that reinforce or shift Political Belief (A), perpetuating the cycle. - The absence of numerical data limits causal strength assessment but highlights the **theoretical interconnectedness** of these factors in shaping social outcomes. This model aligns with sociological theories of structural mobility, where individual agency (Political Belief) and structural barriers (Job Hiring, SES) co-evolve. Further analysis would require empirical data to quantify the strength and directionality of these relationships. </details> Figure 2: Confounder structure. <details> <summary>x3.png Details</summary>  ### Visual Description ## Diagram: Triangular Relationship Model ### Overview The image depicts a triangular diagram illustrating relationships between three variables: **Gender**, **Department**, and **Admission in College**. Arrows indicate directional influences between these nodes, forming a closed loop. ### Components/Axes - **Nodes**: - **A**: Labeled "Gender" (bottom-left vertex). - **M**: Labeled "Department" (top vertex). - **Y**: Labeled "Admission in College" (bottom-right vertex). - **Arrows**: - **A → M**: Gender influences Department. - **M → Y**: Department influences Admission in College. - **Y → A**: Admission in College influences Gender. ### Detailed Analysis - **Textual Labels**: - All nodes are explicitly labeled with their respective variables. - Arrows are unidirectional, emphasizing causality or influence. - **Structure**: - The triangle is equilateral, with equal spacing between nodes. - No numerical values, scales, or legends are present. ### Key Observations - The diagram suggests a **cyclical relationship** where each variable mutually influences the others. - No outliers or anomalies are depicted, as the model is purely structural. ### Interpretation This diagram likely represents a **systemic model** where: 1. **Gender** (A) shapes choices or opportunities in **Department** (M). 2. **Department** (M) determines access to or success in **Admission in College** (Y). 3. **Admission in College** (Y) reinforces or challenges societal perceptions of **Gender** (A), creating feedback loops. The absence of numerical data implies this is a conceptual framework, possibly used to analyze intersectional factors in educational equity or social dynamics. The cyclical nature highlights how systemic biases or structural barriers might perpetuate themselves across these variables. </details> Figure 3: Mediator structure. <details> <summary>x4.png Details</summary>  ### Visual Description ## Diagram: Causal Relationships in Political Belief, Labor Union Activism, and Job Hiring ### Overview The diagram illustrates a causal network with three nodes (A, Y, W) and directional arrows representing relationships. Node A ("Political Belief") influences both Node Y ("Selection for Job Hiring") and Node W ("Labor Union Activism"). Node W also influences Node Y, creating a feedback loop. The diagram uses black arrows and nodes with no numerical data or color coding. ### Components/Axes - **Nodes**: - **A**: Labeled "Political Belief" (leftmost node). - **Y**: Labeled "Selection for Job Hiring" (rightmost node). - **W**: Labeled "Labor Union Activism" (central node, enclosed in a square). - **Arrows**: - **A → Y**: Direct influence from Political Belief to Job Hiring. - **A → W**: Direct influence from Political Belief to Labor Union Activism. - **W → Y**: Feedback influence from Labor Union Activism to Job Hiring. - **No legends, scales, or numerical data** are present. ### Detailed Analysis - **Political Belief (A)** is the primary driver, affecting both **Job Hiring (Y)** and **Labor Union Activism (W)**. - **Labor Union Activism (W)** acts as an intermediary, amplifying or modifying the relationship between Political Belief and Job Hiring. - The diagram suggests a **mediated causal pathway**: Political Belief → Labor Union Activism → Job Hiring, alongside a direct Political Belief → Job Hiring link. ### Key Observations - The feedback loop (W → Y) implies that Labor Union Activism can reinforce or counteract the effects of Political Belief on Job Hiring. - No quantitative values or thresholds are provided, limiting precise measurement of influence strength. - The diagram emphasizes **directionality** (unidirectional arrows) rather than bidirectional relationships. ### Interpretation This diagram highlights how **Political Belief** shapes outcomes through two pathways: directly influencing **Job Hiring** and indirectly via **Labor Union Activism**. The feedback loop suggests that union activism may mediate or alter the impact of political beliefs on hiring practices. For example, strong union activism could either amplify (e.g., through collective bargaining) or dampen (e.g., through resistance to biased practices) the effects of political beliefs on hiring. The absence of numerical data prevents quantification but underscores the conceptual interplay between these factors. The structure aligns with theories of **mediated causality** in social sciences, where intermediate variables (like union activism) moderate direct relationships. </details> Figure 4: Collider structure. ### 3.1 Confounder structure The first scenario in which ignoring the causal structure of the data can lead to an unreliable estimation of discrimination arises from failing to account for a confounding variable. Consider the hypothetical example in Figure 4, which depicts an automated system used to select candidates for job positions. Suppose the system takes two input features: socio-economic status (SES), denoted as $Z$ , and the candidate’s political belief, denoted as $A$ . The outcome, $Y$ , represents whether the candidate is selected for the next stage of hiring (or the probability of selection). The SES influences the hiring outcome because candidates from higher SES backgrounds may have greater access to reputable academic institutions and expensive training programs. Both SES ( $Z$ ) and political belief ( $A$ ) can be either binary (e.g., rich vs. poor for SES, liberal vs. conservative for political belief) or continuous (e.g., the degree of wealth for SES and the extent of conservatism for political belief). In this example, we assume that a candidate’s political belief is influenced solely by their SES. Now, suppose the automated decision system is suspected of being biased against candidates with a particular political belief. The claim is that the system is more likely to select candidates who hold a specific political ideology. However, failing to account for the confounding effect of SES may lead to incorrect conclusions about whether the system is truly biased. A simple approach to assessing the fairness of the automated selection process, represented by $Y$ , with respect to the sensitive attribute $A$ , is to compare the conditional probabilities: $$ \mathbb{P}(Y=1\mid A=0)\quad\text{and}\quad\mathbb{P}(Y=1\mid A=1) $$ This comparison, known as statistical disparity, quantifies the difference in selection rates between the two groups (e.g., conservatives and liberals). However, such an estimation of discrimination is biased due to the confounding effect of $Z$ . Since $Z$ influences both the sensitive variable $A$ and the outcome $Y$ , it introduces a correlation between $A$ and $Y$ that is not causal. In other words, candidates with a high socio-economic status (SES) are more likely to have conservative political beliefs and, at the same time, have a higher probability of being selected due to their access to better academic institutions and training opportunities. This results in an observed correlation in the data: employers will see a greater proportion of candidates with conservative political beliefs and a lower proportion of those with liberal beliefs. However, this correlation is driven by the confounder $Z$ and should not be misinterpreted as discrimination. Most statistical fairness metrics, such as equal opportunity and predictive parity, fail to account for such confounding effects, making them unsuitable for measuring discrimination in the presence of these statistical anomalies. ### 3.2 Mediator structure The second scenario in which failing to account for the causal structure of data leads to unreliable discrimination estimation arises from the presence of one or more mediator variables. The core issue is whether discrimination transmitted through a mediator variable should be considered justifiable or not. Similar to confounding structures, a mediator variable can result in Simpson’s paradox. A well-known example of Simpson’s paradox caused by a mediator structure is the gender bias observed in the 1973 Berkeley graduate admissions study (Bickel et al., 1975; Loftus et al., 2018a). Figure 4 illustrates the causal graph underlying the data, where the sensitive variable ( $A$ ) represents gender, the outcome ( $Y$ ) represents admission to Berkeley graduate programs, and a single mediator variable ( $M$ ) represents the department to which a candidate applied. In 1973, 44% of male applicants were admitted, compared to only 34% of female applicants. At first glance, this appears to indicate a bias against female candidates. However, when the data was analyzed separately by department, acceptance rates were found to be approximately equal across genders. In a simple mediator structure, there are two possible causal paths from $A$ to $Y$ : a direct path, $A\rightarrow Y$ , and an indirect path, $A\rightarrow M\rightarrow Y$ . Comparing the overall admission rates of male and female candidates considers both paths in the discrimination measure. In contrast, comparing admission rates within each department isolates only the direct path, $A\rightarrow Y$ . Consequently, whether or not to account for mediator paths when measuring discrimination can lead to contradictory conclusions, as demonstrated by Simpson’s paradox. ### 3.3 Collider structure A biased estimation of discrimination can also arise due to the presence of a common effect (collider) variable and implicit conditioning on that variable during the data generation process. Using the same hypothetical job selection example, consider the causal graph in Figure 4, where $A$ and $Y$ retain the same meanings as in the previous example. Assume that the data used to train the automated decision system is collected from various sources, but primarily from labor union records. Let $W$ represent a candidate’s labor union activism, which is influenced by both $A$ and $Y$ . On one hand, political belief ( $A$ ) affects whether a candidate becomes an active labor union member, as individuals with liberal political beliefs are more likely to join labor unions. On the other hand, if a candidate is selected (hired), they are more likely to become a labor union member, increasing the likelihood that their case appears in labor union records. Following prior work, a box around a variable ( $W$ ) indicates that data is implicitly conditioned on that variable during the data collection process. Once again, the simple approach of comparing selection rates between the two groups (conservatives and liberals) leads to a biased estimation of discrimination due to the colliding path through $W$ . Intuitively, an individual appears in the collected data either because they hold liberal political beliefs or because they were selected for the job. Candidates who are both liberal and selected remain in the data, but conditioning on labor union activism introduces a non-causal correlation between $A$ and $Y$ . Specifically, the dataset drawn from labor union records includes fewer liberal candidates who were selected for the job compared to conservative candidates. This pattern may misleadingly suggest discrimination against liberal candidates. However, this correlation arises from the collider structure and should not be mistaken for actual discrimination. ## 4 Mediation Analysis for Disentangling Explaining Variables and Proxy Discrimination <details> <summary>x5.png Details</summary>  ### Visual Description ## Diagram: Socio-Economic and Political Belief Interactions ### Overview The diagram illustrates a directed graph representing relationships between socio-economic status (SES), political beliefs, job hiring, community service, and address. Arrows indicate directional influences or connections between these components. ### Components/Axes - **Nodes (Variables):** - **Z**: SES (Socio-Economic Status) – Positioned at the top of the diagram. - **Y**: Selection for Job Hiring – Located in the upper-middle section. - **W**: Community Service – Positioned at the bottom. - **A**: Political Belief – Located on the left side. - **T**: Address – Positioned centrally, connected to W. - **Arrows (Relationships):** - **A → Y**: Political Belief influences Selection for Job Hiring. - **A → W**: Political Belief influences Community Service. - **Y → Z**: Selection for Job Hiring influences SES. - **T → W**: Address influences Community Service. ### Detailed Analysis - **Node Labels and Positions:** - **Z (SES)**: Topmost node, acting as a terminal outcome. - **Y (Job Hiring)**: Middle-right node, directly connected to Z. - **W (Community Service)**: Bottom node, connected to both A and T. - **A (Political Belief)**: Left node, with two outgoing arrows to Y and W. - **T (Address)**: Central node, with one outgoing arrow to W. - **Arrow Directions:** - All arrows point from source nodes (A, Y, T) to target nodes (Y, W, Z), indicating causal or influential relationships. ### Key Observations 1. **Political Belief (A)** is a central driver, influencing both **Job Hiring (Y)** and **Community Service (W)**. 2. **Job Hiring (Y)** directly impacts **SES (Z)**, suggesting a pathway from employment to socio-economic status. 3. **Address (T)** indirectly affects **Community Service (W)**, implying that residential location may influence participation in community activities. 4. **SES (Z)** is the final outcome, receiving input from both **Job Hiring (Y)** and **Community Service (W)**. ### Interpretation The diagram suggests a model where **Political Belief** acts as a foundational factor shaping opportunities in **Job Hiring** and **Community Service**, which in turn influence **SES**. The inclusion of **Address (T)** as a mediator highlights the role of environmental or geographic factors in community engagement. This structure implies that socio-economic outcomes are not solely determined by individual actions (e.g., job hiring) but are also mediated by systemic factors like political ideology and residential context. The absence of feedback loops (e.g., SES influencing political beliefs) suggests a unidirectional model, though real-world dynamics may involve more complex interactions. </details> Figure 5: Causal graph with two mediated paths. In the presence of one or more mediator variables, it is important to determine how much of the observed discrimination is direct and how much is mediated. More specifically, understanding the extent to which discrimination is transmitted through each mediator variable is crucial. Mediation analysis aims to distinguish the different causal pathways through which discrimination propagates and quantify the proportion of discrimination conveyed through each path. Consider a variation of the job hiring example, illustrated in Figure 5, which includes two mediator variables: address ( $T$ ) and community service participation ( $W$ ). In this scenario, there are four distinct causal paths from the sensitive variable $A$ (political belief) to the outcome variable $Y$ (job hiring): - $A\leftarrow Z\rightarrow Y$ : confounding path - $A\rightarrow Y$ : direct path - $A\rightarrow T\rightarrow Y$ : indirect path through $T$ - $A\rightarrow W\rightarrow Y$ : indirect path through $W$ . The first confounding path is non-causal, meaning that any effect transmitted through it should not be considered when estimating discrimination. As discussed in Section 3.1, this spurious effect arises from the way data is generated or collected and, therefore, should not be counted as actual discrimination. The direct path exists whenever there is an edge between $A$ and $Y$ . The effect through $A\rightarrow Y$ is always discriminatory—it can never be justified or considered acceptable discrimination. The two remaining paths are indirect paths passing through mediator variables. Whether discrimination along an indirect path is justifiable depends on the nature of the mediator variable. For instance, in the job hiring example shown in Figure 5, $T$ (home address) acts as a mediator because, on one hand, political inclination may influence where a candidate lives, and on the other hand, hiring decisions may be influenced by a candidate’s address. Similarly, $W$ (community service) is a mediator because a candidate’s political beliefs may impact their level of involvement in community service, while an employer may consider community service records as an indicator of suitability for a given position. Discrimination along the path $A\rightarrow W\rightarrow Y$ can be considered justifiable, as an employer may argue that disparities between candidates with different political beliefs arise from differences in their community service records. However, discrimination through the path $A\rightarrow T\rightarrow Y$ is typically not acceptable, as an employer cannot justify hiring decisions based on candidates’ addresses. In this context, $T$ is referred to as a proxy variable, whereas $W$ is called an explaining variable. If a single causal path consists of two or more mediator variables, the presence of at least one explaining variable among the mediators makes discrimination along that path justifiable and, therefore, acceptable. Causality, through the concepts of intervention and counterfactual reasoning, provides the necessary tools to distinguish between different types of discrimination based on their causal pathways. By intervening on $A$ , all paths that include an incoming edge to $A$ —such as confounding paths between $A$ and $Y$ —are blocked. Discrimination transmitted through causal paths is quantified by the total effect ( $TE$ ) ( eq:TE). In the causal graph depicted in Figure 5, the $TE$ expression captures discrimination along all causal paths except for the non-causal confounding path $A\leftarrow Z\rightarrow Y$ . To distinguish between direct and indirect discrimination, two key expressions can be used: the natural direct effect ( $NDE$ ) (eq:NDE) and the natural indirect effect ( $NIE$ ) (eq:NIE) (Pearl, 2001). Intuitively, $NDE$ is counterfactual quantity because it corresponds to a candidate who is conservative ( $A=1$ ) along the direct path $A\rightarrow Y$ but liberal ( $A=0$ ) along all indirect paths. This isolates the direct effect of $A$ on $Y$ while holding indirect pathways constant. Here, $NIE$ represents the probability of a counterfactual situation where $Y=y^{\scriptscriptstyle+}$ had $A$ been $0 0$ along the direct path while indirect path took the value it would naturally have if $A=1$ . This captures the portion of discrimination that is mediated through indirect paths. Finally, discrimination conveyed through specific indirect paths can be identified using the path-specific effect ( $PSE$ ) (eq:PSE) (Pearl, 2001; Chiappa, 2019). For example, $PSE$ quantifies the extent to which a sensitive attribute influences a decision through the mediator ’address’ compared to the mediator ’community service.’ This has direct implications for fairness assessment. Mediation analyis allows for a more granular analysis of discrimination by isolating the contribution of specific causal pathways, enabling a deeper understanding of the mechanisms through which bias is propagated in automated decision-making systems. ## 5 Uncovering causality through legal evidence: the regulatory approach in the European Union The method of using mediator structures to uncover causation, as discussed in the previous section, is undoubtedly a valuable model for establishing causality in judicial cases involving algorithmic discrimination. However, an important question arises: does procedural law, particularly within the European Union (EU), support such an analysis? As a preliminary observation, it is essential to clarify that in legal contexts, the term causal fairness generally refers to the procedural conditions under which instances of fairness (or unfairness) are causally represented. With this in mind, this section focuses on two critical and interrelated issues: evidence and procedural fairness. In the EU, the Independent High-Level Expert Group on AI, established by the European Commission, defines procedural fairness as the ability to contest and seek effective redress against decisions made by AI systems and their human operators. See HLEG, Ethics Guidelines for Trustworthy AI, available at https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai, p. 13. From a legal perspective, causality is a question of fact, requiring legally established discovery procedures and corresponding reasoning models designed to yield accurate causal representations—i.e., to distinguish genuine causation from a myriad of correlations (positive associations between candidate causes and a harm suffered) (Haack, 2014). However, in adjudicatory contexts, causality is established primarily to serve the purpose of fairness, typically in the form of compensation as a fair remedy for harm suffered. Legal systems committed to the rule of law In the EU, the concept of the rule of law encompasses the following principles: legality, legal certainty, prohibition of arbitrariness by executive powers, independent and impartial courts, effective judicial review (including respect for fundamental rights), and equality before the law. See Communication from the Commission to the European Parliament and the Council, ‘A New EU Framework to Strengthen the Rule of Law,’ COM(2014) 158 final, p. 4. share a commitment to procedural fairness, based on the normative principle that only fairly designed procedures can lead to fair outcomes. In contemporary systems of evidence and judicial remedies, including those within EU law, the parallelism between fair procedures and fair outcomes is epitomized in fair trial safeguards. These procedural entitlements are intended to uphold basic equality (or procedural parity) and ensure the effectiveness of litigants’ participation in dispute resolution (Solum, 2004). This principle applies not only to equal access to judicial remedies but also to equal access to and the ability to present evidence. The fundamental idea is that neither party should be advantaged or disadvantaged in terms of access to the facts necessary to present their case—typically before a court. In summary, causal fairness in law requires accurate—or at the very least, plausible (Rescher, 1980) —evidence of causality, presented under conditions of procedural fairness. The proof of causality in cases of algorithmic discrimination has profoundly challenged longstanding legal postulates. From a procedural fairness perspective, a major issue arises from the opacity—whether relative or total—of AI systems, which renders their decision-making processes inscrutable and obstructs victims’ ability to properly establish and argue causation. One prominent example in this regard is Cook v. HSBC North America, U.S. District Court for the Northern District of Illinois, 21 March 2014, County of Cook v. HSBC North America Holdings Inc et al., 1:2014cv02031. a credit scoring case in which the system used applicants’ places of residence as a relevant variable, ultimately favoring predominantly white neighborhoods while discriminating against ethnic minorities. The association of subtly discriminatory variables—such as zip code and ethnic background—combined with practical difficulties in accessing information about how the AI system weighted different variables, severely undermined fundamental fair trial safeguards, particularly the right to access evidence and courts. To address these challenges, regulators worldwide, including in the EU, have sought to answer two key questions: 1. What evidence must litigants have access to in order to effectively prove causation? 1. Once that evidence is identified, how should legal procedures be (re)designed to ensure victims’ access to it? These questions are central to ensuring that legal systems remain capable of addressing algorithmic discrimination while upholding principles of procedural fairness. ### 5.1 Methods for Establishing Causality in Algorithmic Discrimination Cases Regarding the first question, emerging—though not yet consolidated—global AI liability case law reveals an interesting trend. While numerous judicial instances could be citepd, for the purpose of this article, we highlight three cases that illustrate the ‘new approach’ to proving causation in AI-related disputes. These cases are Pickett, Superior Court of New Jersey (Appellate Division), 2 February 2021, State of New Jersey v. Corey Pickett, Docket N° A-4207-19T4. (which concerns the use of a DNA matching system—TrueAllele—by law enforcement to track down suspects), Loomis, Supreme Court of Wisconsin, 13 July 2016, State of Wisconsin v. Eric L. Loomis, 881 N.W.2d 749 (2016) 2016 WI 68. (which involves COMPAS, a recidivism-prediction system used by courts), and Ewert, Ewert v. Canada, 2018 SCC 30, File N° 37233, 13 June 2018. (which also deals with the use of recidivism-predicting systems by Canadian correctional services). In all three cases, the plaintiffs argued that the automated decisions were inaccurate because they were unfair—that is, they contained unfair biases: gender bias in Pickett and Loomis, and ethnic bias in Ewert. To uncover the bias-inducing variable associations (i.e., the causal link), the plaintiffs requested that the systems be reverse-engineered. However, this proved nearly impossible. For instance, in Pickett, independent experts confirmed that reverse-engineering the system would take up to 8.5 years to complete. See Superior Court of New Jersey (Appellate Division), 2 February 2021, State of New Jersey v. Corey Pickett, Docket N° A-4207-19T4, at 17. The emerging field of mechanistic interpretability offers methods to inspect models, such as LLMs internal representations without full reverse engineering (Bereska and Gavves, 2024). However, the causal relationship between concepts and representations is hard to establish and is still an open problem for the future research (Sharkey et al., 2025; Binkyte et al., 2025). Faced with the practical infeasibility of reverse-engineering, the courts in Pickett (and in Loomis) turned to general expertise as a faute de mieux solution. The absence of direct evidence (through reverse-engineering) that could reveal unfair bias was ‘compensated’ by relying on pre-existing expert assessments of the system’s general functionality. If the majority of experts agreed that a system—such as TrueAllele in Pickett or COMPAS in Loomis —was generally well-performing (i.e., unbiased and therefore accurate), courts were inclined to presume that the system’s decisions in the specific cases under dispute were also unbiased. Thus, the role of experts is to assess the strength of the causal link between sensitive variables and the decision. In the presence of different causal structures (Section 3), which may lead to various types of bias, a zero causal effect would indicate the absence of discrimination. A possible approach to uncovering this causal link is to first identify the causal graph that represents the relationships between variables. This can be achieved by: 1. Using causal discovery techniques to infer an initial causal graph. 1. Consulting domain experts to refine the discovered graph—such as adding or removing causal links and enforcing domain-specific assumptions. Expert input is also essential for clarifying the role of each variable, particularly in classifying mediator variables as either explaining (leading to justifiable discrimination) or proxy (leading to unjustifiable discrimination) variables. This classification is crucial for selecting the appropriate causal fairness metric (Section 4) to assess discrimination. From the perspectives of procedural fairness and the mediator structure model, this trend is open to criticism. First, general expert opinions on a system’s accuracy are not as probative as direct evidence (such as reverse engineering), which could provide highly reliable information on the mediator associations leading to a discriminatory outcome. Second, the inability to prove causation through reliable evidence appears to have led to a peculiar application of the so-called but-for test. In principle, the but-for test employs counterfactual reasoning to determine whether a harm would have occurred if an alleged cause had not been present. For example, in Cook v. HSBC (a credit scoring case), a standard application of this test would involve determining whether the same loan applicants would have been approved had the system not considered their place of residence as a relevant variable. However, the cases cited in this section (particularly Pickett and Loomis) reveal a shift in how the but-for test is applied. In conventional (non-AI-related) disputes, the test seeks to answer a factual question about causal association: would the outcome have been the same (or different) if certain facts (such as address, gender, or age) were absent from the causal structure? In AI-related disputes, however, the but-for test is reframed to address a different causal question: would a human decision relying on AI output have been the same or different if the AI system had not been used at all? In this context, statistical causality tools can be applied to detect discrimination in data reflecting previous hiring or loan-granting practices within the company. If an association between the sensitive attribute and the outcome is identified, one can infer that the decision would remain the same even without algorithmic assistance. This shifts the focus from scrutinizing the AI system (and its designers) to assessing broader company practices. Here, causality analysis helps differentiate between spurious associations, explainable disparities, and actual discrimination. Conversely, if the data used to train the AI system contains ingrained discrimination, compliance with AI design guidelines warrants further examination. Finally, causality tools offer formal mathematical expressions that capture the otherwise intangible concept of counterfactual reasoning (Shpitser and Pearl, 2007, 2008), making them particularly useful for directly verifying the but-for test. This allows us to raise the second issue mentioned above: should systems of evidence include a right to access/to request disclosure of evidence? ### 5.2 Disclosing causal evidence to victims of discrimination Given these practical and theoretical challenges associated with the but-for test and counterfactual reasoning in AI-related discrimination cases, an important question emerges regarding how evidence disclosure mechanisms might facilitate fairer judicial outcomes. In particular, should litigants possess a specific right to access evidence that would allow effective causal analysis? From a procedural fairness perspective, the right to request evidence disclosure is crucial for a victim of algorithmic discrimination, providing at least an opportunity to seek the lifting of the opacity veil that may obscure a causal chain (Grozdanovski, 2021). In the EU, recent regulatory developments appear—at least on the surface—to be moving toward recognizing such a right. The first major step was the introduction of the AI Act, Proposal for a Regulation of the European Parliament and of the Council laying down harmonized rules on Artificial Intelligence (AI Act) and amending certain Union legislative acts, COM(2021) 206 final. a horizontal, cross-sectoral legislation that makes two significant contributions. First, it establishes a four-level taxonomy of AI-related risks: non-high, limited, high, and unacceptable. Second, based on this risk classification, the AI Act introduces a set of technical standards—including transparency, data governance, and risk-mitigation strategies—specifically targeting high-risk AI systems deployed in eight key market sectors. The ‘high-risk’ sectors, listed in Annex III of the AI Act, include employment, education, healthcare, transport, energy, public sector (including asylum, migration, border controls, judiciary, and social security services), defense and security, finance, banking, and insurance. To complement the AI Act and to provide procedural avenues for compensating harm caused by high-risk AI systems, the EU introduced the AI Liability Directive (AILD). Proposal for a Directive of the European Parliament and of the Council on adapting non-contractual civil liability rules to Artificial Intelligence (AI Liability Directive), COM(2022)496 final. This directive establishes an evidentiary framework granting victims the right to request disclosure of evidence. Under the AILD, if the defendant (a programmer or user) refuses to disclose the requested evidence, or if a national or EU court finds that the disclosed evidence is both probative and plausible, the defendant is presumed responsible for the harm (e.g., discrimination) suffered by the claimant. However, it is important to note that the type of evidence a victim may request under the AILD does not include the kind of evidence previously identified as necessary (i.e., expert analysis) in the cases discussed earlier. The AILD permits disclosure only of evidence related to the defendant’s compliance with the technical standards outlined in the AI Act. In other words, defendants would not be required to provide information such as access to the AI system’s code or, where feasible, allow for reverse engineering to support a proper causal analysis. Instead, they would merely be asked to demonstrate compliance with obligations such as ensuring human oversight and control. The rationale behind this limitation is that the AILD operates under the assumption that if harm—such as discrimination—occurs, it is because the AI Act was not fully adhered to. As a result, the AILD effectively narrows the scope of evidentiary debates in AI discrimination cases. Instead of compelling parties to uncover the actual causal structure underlying a discriminatory AI outcome, legal proceedings will primarily focus on identifying the human agent who failed to meet a legally prescribed duty of care. ## 6 Challenges and Opportunities in Using Causality for Fairness While recent regulatory developments such as the AI Act and AI Liability Directive represent significant progress in addressing algorithmic discrimination, their implementation relies fundamentally on the practical feasibility of causal analysis. Thus, understanding the methodological constraints and opportunities of causal inference becomes essential to ensure that regulatory efforts lead to meaningful improvements in fairness. Many causal requirements can, in principle, be satisfied through specific experimental designs—ideally, through random assignment. However, in fairness-related scenarios, such experimental setups are often impractical or ethically infeasible. As a result, discrimination is typically evaluated using observational data. In this section, we outline some key requirements for applying causal inference, particularly those most relevant to fairness applications. ### 6.1 Availability of Causal Graph The effective use of causal inference in fairness applications is shaped by several critical methodological requirements and assumptions. Among these, one of the most significant is the availability of an accurate directed acyclic graph (DAG) representing relationships among variables. The DAG is subject to additional assumptions, including the causal Markov condition, causal faithfulness, and causal sufficiency. Together, these conditions encode the same requirements as SUTVA and ignorability in the potential outcome framework and, therefore, will not be discussed separately. (Pearl, 2009a). Research by (Binkytė-Sadauskienė et al., 2022) highlights significant disagreements in causal fairness estimations due to minor variations in the assumed causal structure. The availability of a DAG is particularly crucial in the presence of collider structures, as conditioning on a collider induces bias in causal effect estimation (Pearl, 1988). Additionally, DAGs are essential for evaluating path-specific effects, which play a key role in distinguishing between redlining and explaining variables in fairness scenarios. Causal structures (or causal graphs) can be obtained through two main approaches: expert consultation or learning from observational data. However, both approaches have limitations. Domain experts may hold differing views or introduce biased assumptions. Learning causal relationships from data, on the other hand, often requires additional assumptions regarding data distribution, functional dependencies, relationships between exogenous unobserved variables, and an informed choice of the learning algorithm. Research by (Binkytė-Sadauskienė et al., 2022) demonstrates how different causal discovery algorithms, when applied to the same dataset, can yield significantly different causal structures. This variability raises concerns about the reliability of causality learned purely from observational data (Guyon et al., 2011). Expert-driven causal graphs, while useful, are not easily scalable and may be prone to human error. However, recent advancements in causal discovery with large language models (LLMs) offer a promising alternative (Kıcıman et al., 2023; Kasetty et al., 2024; Vashishtha et al., 2023). Hybrid approaches that combine LLM-driven insights with statistical causal discovery methods appear to be particularly promising in bridging the gap between expert-driven and data-driven causal inference (Afonja et al., 2024). ### 6.2 Causal Transportability Beyond the accurate identification of causal graphs, another crucial consideration when applying causal inference to fairness is whether causal findings obtained in one context remain valid and generalizable across diverse populations—an issue addressed by the concept of causal transportability. Currently, the application of causal knowledge across populations relies heavily on assumptions about general causal structures that remain relatively stable across different contexts. Examples include physical laws, such as ”altitude causes temperature,” or well-established medical principles, such as ”bacteria cause disease.” However, the principles underlying ethical AI are often deeply intertwined with cultural values and societal norms, which are neither immutable nor universally consistent. These variations present significant challenges in transferring causal insights across different demographic or geographic populations. The framework developed by Pearl and Bareinboim utilizes directed acyclic graphs (DAGs) to adjust causal knowledge for new settings through targeted data collection (Bareinboim and Pearl, 2014; Pearl and Bareinboim, 2011). Building on this, Binkyte et al. (2024) propose an expectation-maximization (EM) approach to adapt causal knowledge for applications in target demographic groups, further enhancing the practicality of causal transportability in real-world scenarios. ### 6.3 Possibility for Intervention In fairness estimation, the sensitive attribute is typically considered the exposure or treatment variable, with the goal of measuring its impact on the outcome. Most definitions of causal effect are based on the notion of intervention or manipulation of a cause variable (exposure) (VanderWeele and Hernán, 2012; Holland, 1986). This presents a challenge when making causal claims about non-manipulable attributes, such as race or gender. Some approaches in the literature suggest shifting the focus from actual manipulation to changes in perception (Rahmattalabi and Xiang, 2022). For instance, instead of altering the gender of candidates to estimate its effect on hiring decisions, researchers could manipulate the employer’s perception of gender. This could be achieved by submitting two otherwise identical résumés with different names or titles indicating different genders. Such an approach aligns with methodologies used in social experiments examining the impact of race or gender on hiring decisions (Neumark, 2018). (VanderWeele and Hernán, 2012) further differentiates immutable sensitive attributes into those that are randomized at birth (e.g., biological sex) and those that are not (e.g., race, social gender). This distinction is important when estimating the causal effect of a sensitive attribute. If the attribute is randomized, its causal effect on an outcome can be estimated by directly comparing exposure levels. For example, the total causal effect of biological sex on an outcome can be estimated by comparing observed differences between men and women (VanderWeele and Hernán, 2012). In contrast, race is not randomly assigned but influenced by ancestral and socio-cultural factors, making causal effect estimation more complex. Even at the biological level, estimating the effect of race requires a thorough understanding of the causal structure of relevant covariates. Such estimation is particularly relevant in medical scenarios, where assuming independence between the sensitive attribute and the outcome (e.g., disease probability) is unreasonable. Similarly, in potential discrimination scenarios, (VanderWeele and Hernán, 2012) and (Rahmattalabi and Xiang, 2022) suggest shifting the focus to the direct effect of the perceived gender or race on decision-making. This perspective aligns with causal fairness approaches that assess discrimination based on how sensitive attributes are perceived and acted upon within decision-making processes. ### 6.4 Causal assumptions The Stable Unit Treatment Value Assumption (SUTVA) (Rubin, 1986) imposes two key requirements: no interference and consistency. The no interference assumption states that the effect of the sensitive attribute on the outcome should not be influenced by interactions between individuals. However, in social sciences research—where interactions and feedback loops are common—this assumption is often challenged, necessitating careful discussion and restricted interpretations of causal estimates (Morgan and Winship, 2015). Since fairness is typically evaluated within a social context, potential interactions must be carefully assessed. For example, in the hiring scenario, a violation of the no interference assumption would occur if hiring individuals from one political spectrum increased the likelihood of favoring candidates with the same political beliefs in future hiring decisions. This scenario is plausible because current employees may prefer candidates who share their political ideology, creating a feedback loop that perpetuates bias. The consistency assumption requires that each treatment level leads to the same potential outcomes (Keele, 2015). In fairness evaluation, the treatment variable is replaced by the sensitive attribute, which is often a socially constructed category such as race or gender. Identifying the causal effect of gender on hiring becomes problematic if gender itself does not have a consistent effect on hiring decisions. For instance, discrimination may not apply uniformly to all women; rather, only women who exhibit a certain level of ”femininity” might experience bias. If this possibility cannot be ruled out, it should be considered in studies aiming for a fine-grained causal analysis. In summary, SUTVA assumptions are likely to be violated in fairness scenarios. However, causal approaches can still be applied if the results are interpreted with caution. Some methods for identifying causal effects despite SUTVA violations are discussed in (Laffers and Mellace, 2020). The ignorability assumption (Rubin, 1986) requires that the sensitive attribute and the outcome be independent, given the observable variables. In other words, no unobserved variables should create a significant link between the sensitive attribute and the outcome. In fairness evaluation, the presence of such a link could indicate that a portion of the observed discrimination is, in fact, a spurious effect induced by a confounding variable. For example, if education is a confounder but is not included in the dataset, its confounding effect cannot be controlled. Consequently, it becomes impossible to estimate the causal effect of political belief on hiring decisions separately from the effect of education. Unobserved confounders are less likely to exist for immutable sensitive attributes such as sex or race, as these attributes generally do not have temporally prior causes. However, dependency between noise terms related to the sensitive attribute and the outcome can still violate ignorability. (Fawkes et al., 2022) highlight the implications of assuming ignorability when using causal counterfactuals. Following their reasoning, consider the case of college admissions (illustrated in Figure 4). Under the ignorability assumption, an average male applicant to a technical profession could be counterfactually replaced with an average female applicant to the same profession. However, due to social expectations tied to gender roles, a woman applying to a technical profession is likely to be more motivated and hardworking than an average male applicant with the same professional aspirations. This example illustrates how violations of ignorability can introduce bias into causal estimations. The positivity assumption (Rubin, 1986) is violated if certain combinations of a sensitive attribute and a covariate have zero probability. Violations of positivity can be either deterministic or random (Westreich and Cole, 2010). For example, positivity would be violated if a specific level of education always corresponded to liberal political beliefs. In this scenario, the violation would likely be random, as it is improbable that education would have a strictly deterministic relationship with political beliefs. In such cases, statistical methods exist for handling analysis under violations of positivity (Westreich and Cole, 2010). However, consider a case where obtaining a Harvard degree is treated as an explanatory mediator between ethnicity and hiring. Due to historical patterns of discrimination and socioeconomic disparities, certain ethnic groups may have near zero probability of obtaining a Harvard degree. This constitutes a deterministic violation of positivity. In such cases, it is essential to reconsider whether possessing a Harvard degree should be an absolute requirement for the job in question, given its potential for exclusion and disparate impact. The identifiability of path-specific effects in the presence of multiple mediators requires the absence of causal links between the mediators (VanderWeele and Vansteelandt, 2014). Evaluating path-specific effects is particularly important for understanding the mechanism through which the sensitive attribute influences the outcome. As discussed earlier (Section 4), the effect of the sensitive attribute can be considered either justifiable or discriminatory depending on the mediating variables along the causal path. However, in fairness scenarios, causal dependencies between mediators are common. For example, consider a case where the relationship between race and hiring decisions is mediated by both social status (e.g., redlining) and education (an explaining variable). It is highly probable that education level is influenced by social status. In such a scenario, the indirect effects of race through social status and education separately are not identifiable, as the mediators are causally dependent. To address this challenge, (VanderWeele and Vansteelandt, 2014) proposed a method that treats multiple mediators jointly. In some cases, this approach can help disentangle and identify individual indirect effects in the presence of causal links between the mediators. Recognizing these methodological challenges underscores the importance of carefully balancing technical rigor and practical applicability. With these considerations in mind, we now summarize our findings and highlight future directions for research. ## 7 Conclusions This article has outlined the critical role of causality in evaluating the fairness of AI-driven decisions. We demonstrated that relying exclusively on statistical tools without causal analysis can lead to significant inaccuracies, such as overestimating, underestimating, or even reversing discrimination effects. Furthermore, we highlighted the essential connections between reliable evidence of algorithmic discrimination, legal practices in courts, and evolving regulatory frameworks, particularly within the European Union. Specifically, we examined how causal reasoning influences legal assessments of discrimination and identified two primary forms of evidence— direct evidence, derived from reverse engineering AI systems, and indirect evidence. Given that direct evidence is often practically unattainable due to AI opacity, the article explores alternative causal analyses through indirect evidence. Two approaches based on but-for counterfactual reasoning were presented: The first approach evaluates the impact of sensitive attributes or redlining mediators in algorithmic decision-making, using test data to investigate causal relationships. The second approach assesses whether decisions would differ if AI systems were not employed at all, placing emphasis on historical decision-making practices within organizations. We also critically discussed the causal assumptions underlying fairness assessments, emphasizing the inherent challenges, such as dependency on accurate causal graphs, practical limitations in conducting robust causal analyses, and potential biases arising from untestable assumptions. ### 7.1 Limitations and Future Research Directions Several important limitations constrain the practical application of causal fairness evaluations. Most of the challenges are related to accurately specifying causal structures, the infeasibility of reverse engineering complex AI systems, and the limitations inherent in observational data. These factors restrict the effectiveness of existing causal inference methodologies. Future research should address these limitations by: - Establishing causality in mechanistic interpretability methods allowing reliable and legally binding conclusions on fairness of model’s internal representations. - Refining regulatory frameworks to better align legal evidentiary standards with technical realities, thereby enhancing procedural fairness. - Exploring hybrid approaches combining expert knowledge, causal discovery algorithms, and insights from large language models (LLMs) to improve the reliability and scalability of causal fairness assessments. - Investigating cross-cultural validity and adaptability of causal fairness concepts, ensuring their global applicability and effectiveness in diverse societal contexts. This work of Ruta Binkyte and Sami Zhioua was supported by the European Research Council (ERC) project HYPATIA under the European Union’s Horizon 2020 research and innovation programme. Grant agreement n. 835294. The work of Ruta Binkyte is funded in part by PriSyn: Representative, synthetic health data with strong privacy guarantees (BMBF), grant No. 16KISAO29K. It is also supported by Integrated Early Warning System for Local Recognition, Prevention, and Control for Epidemic Outbreaks (LOKI / Helmholtz) grant. The work is also partially funded by Medizininformatik-Plattform ”Privatsphären-schutzende Analytik in der Medizin” (PrivateAIM), grant No. 01ZZ2316G, and ELSA – European Lighthouse on Secure and Safe AI funded by the European Union under grant agreement No. 101070617. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or European Commission. Neither the European Union nor the European Commission can be held responsible for them. ## References - Afonja et al. (2024) Tejumade Afonja, Ivaxi Sheth, Ruta Binkyte, Waqar Hanif, Thomas Ulas, Matthias Becker, and Mario Fritz. LLM4GRN: Discovering Causal Gene Regulatory Networks with LLMs–Evaluation through Synthetic Data Generation. arXiv preprint arXiv:2410.15828, 2024. - Bareinboim and Pearl (2014) Elias Bareinboim and Judea Pearl. Transportability from multiple environments with limited experiments: Completeness results. Advances in neural information processing systems, 27, 2014. - Bereska and Gavves (2024) Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for ai safety–a review. arXiv preprint arXiv:2404.14082, 2024. - Bickel et al. (1975) Peter J Bickel, Eugene A Hammel, and J William O’Connell. Sex bias in graduate admissions: Data from berkeley. Science, 187(4175):398–404, 1975. - Binkyte et al. (2024) Ruta Binkyte, Daniele Gorla, and Catuscia Palamidessi. BaBE: Enhancing Fairness via Estimation of Explaining Variables. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1917–1925, 2024. - Binkyte et al. (2025) Ruta Binkyte, Ivaxi Sheth, Zhijing Jin, Muhammad Havaei, Bernhardt Schölkopf, and Mario Fritz. Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models. arXiv preprint arXiv:2502.21123, 2025. - Binkytė-Sadauskienė et al. (2022) Rūta Binkytė-Sadauskienė, Karima Makhlouf, Carlos Pinzón, Sami Zhioua, and Catuscia Palamidessi. Causal Discovery for Fairness. arXiv preprint arXiv:2206.06685, 2022. - Carey and Wu (2022) Alycia N. Carey and Xintao Wu. The causal fairness field guide: perspectives from social and formal sciences. Frontiers in Big Data, 5:892837, 2022. 10.3389/fdata.2022.892837. - Chiappa (2019) Silvia Chiappa. Path-specific counterfactual fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7801–7808, 2019. - Fawkes et al. (2022) Jake Fawkes, Robin Evans, and Dino Sejdinovic. Selection, ignorability and challenges with causal fairness. In Conference on Causal Learning and Reasoning, pages 275–289. PMLR, 2022. - Grozdanovski (2021) Ljupcho Grozdanovski. In search of effectiveness and fairness in proving algorithmic discrimination in eu law. Common Market Law Review, 58(1), 2021. - Guyon et al. (2011) Isabelle Guyon, Constantin Aliferis, Gregory Cooper, André Elisseeff, Jean Philippe Pellet, Peter Spirtes, and Alexander Statnikov. Causality workbench. In Causality in the sciences. Oxford University Press, 2011. - Haack (2014) Susan Haack. Evidence matters: Science, proof, and truth in the law. Cambridge University Press, 2014. - Holland (1986) Paul W Holland. Statistics and causal inference. Journal of the American statistical Association, 81(396):945–960, 1986. - Kasetty et al. (2024) Tejas Kasetty, Divyat Mahajan, Gintare Karolina Dziugaite, Alexandre Drouin, and Dhanya Sridhar. Evaluating interventional reasoning capabilities of large language models. arXiv preprint arXiv:2404.05545, 2024. - Keele (2015) Luke Keele. The statistics of causal inference: A view from political methodology. Political Analysis, 23(3):313–335, 2015. - Kıcıman et al. (2023) Emre Kıcıman, Robert Ness, Amit Sharma, and Chenhao Tan. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050, 2023. - Kusner et al. (2017) Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in neural information processing systems, 30, 2017. - Laffers and Mellace (2020) Lukas Laffers and Giovanni Mellace. Identification of the average treatment effect when sutva is violated. Discussion Papers on Business and Economics, University of Southern Denmark, 3, 2020. - Loftus et al. (2018a) Joshua R Loftus, Chris Russell, Matt J Kusner, and Ricardo Silva. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859, 2018a. - Loftus et al. (2018b) Joshua R Loftus, Chris Russell, Matt J Kusner, and Ricardo Silva. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859, 2018b. - Morgan and Winship (2015) Stephen L Morgan and Christopher Winship. Counterfactuals and causal inference. Cambridge University Press, 2015. - Nabi and Shpitser (2018) Razieh Nabi and Ilya Shpitser. Fair inference on outcomes. In Proceedings of the… AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence, volume 2018, page 1931. NIH Public Access, 2018. - Neumark (2018) David Neumark. Experimental research on labor market discrimination. Journal of Economic Literature, 56(3):799–866, 2018. - Pearl (1988) Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann, 1988. - Pearl (2001) Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pages 411–420, 2001. - Pearl (2009a) Judea Pearl. Causality. Cambridge university press, 2009a. - Pearl (2009b) Judea Pearl. Causality. Cambridge University Press, Cambridge, 2009b. ISBN 978-0-521-89560-6. 10.1017/CBO9780511803161. URL https://www.cambridge.org/core/books/causality/B0046844FAE10CBF274D4ACBDAEB5F5B. - Pearl and Bareinboim (2011) Judea Pearl and Elias Bareinboim. Transportability of causal and statistical relations: A formal approach. In Twenty-fifth AAAI conference on artificial intelligence, 2011. - Plečko et al. (2024) Drago Plečko, Elias Bareinboim, et al. Causal fairness analysis: a causal toolkit for fair machine learning. Foundations and Trends® in Machine Learning, 17(3):304–589, 2024. - Rahmattalabi and Xiang (2022) Aida Rahmattalabi and Alice Xiang. Promises and challenges of causality for ethical machine learning. arXiv preprint arXiv:2201.10683, 2022. - Rescher (1980) Nicholas Rescher. Plausible reasoning: An introduction to the theory and practice of plausibilistic inference. Philosophy and Rhetoric, 13(3), 1980. - Rubin (1986) Donald B Rubin. Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81(396):961–962, 1986. - Rubin (2005) Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322–331, 2005. - Sharkey et al. (2025) Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496, 2025. - Shpitser and Pearl (2007) Ilya Shpitser and Judea Pearl. What counterfactuals can be tested. In 23rd Conference on Uncertainty in Artificial Intelligence, UAI 2007, pages 352–359, 2007. - Shpitser and Pearl (2008) Ilya Shpitser and Judea Pearl. Complete identification methods for the causal hierarchy. Journal of Machine Learning Research, 9(Sep):1941–1979, 2008. - Solum (2004) Lawrence B Solum. Procedural justice. S. CAl. l. reV., 78:181, 2004. - VanderWeele and Vansteelandt (2014) Tyler VanderWeele and Stijn Vansteelandt. Mediation analysis with multiple mediators. Epidemiologic methods, 2(1):95–115, 2014. - VanderWeele and Hernán (2012) Tyler J VanderWeele and Miguel A Hernán. Causal effects and natural laws: towards a conceptualization of causal counterfactuals for nonmanipulable exposures, with application to the effects of race and sex. Causality: statistical perspectives and applications, pages 101–113, 2012. - Vashishtha et al. (2023) Aniket Vashishtha, Abbavaram Gowtham Reddy, Abhinav Kumar, Saketh Bachu, Vineeth N Balasubramanian, and Amit Sharma. Causal inference using llm-guided discovery. arXiv preprint arXiv:2310.15117, 2023. - Westreich and Cole (2010) Daniel Westreich and Stephen R Cole. Invited commentary: positivity in practice. American journal of epidemiology, 171(6):674–677, 2010. - Wu et al. (2019) Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. Pc-fairness: A unified framework for measuring causality-based fairness. In Advances in Neural Information Processing Systems, pages 3404–3414, 2019. - Zhang and Bareinboim (2018) Junzhe Zhang and Elias Bareinboim. Fairness in decision-making—the causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.