# On the Need and Applicability of Causality for Fairness: A Unified Framework for AI Auditing and Legal Analysis
\clearauthor \Name
Ruta Binkyte \Email ruta.binkyte@gmail.com \addr CISPA, Helmholtz Center for Information Security \Name Ljupcho Grozdanovski \Email grozdanovski@uliege.be \addr University of LiĂšge \Name Sami Zhioua \Email sami.zhioua@gmail.com \addr INRIA, Saclay Ile-de-France
Abstract
As Artificial Intelligence (AI) increasingly influences decisions in critical societal sectors, understanding and establishing causality becomes essential for evaluating the fairness of automated systems. This article explores the significance of causal reasoning in addressing algorithmic discrimination, emphasizing both legal and societal perspectives. By reviewing landmark cases and regulatory frameworks, particularly within the European Union, we illustrate the challenges inherent in proving causal claims when confronted with opaque AI decision-making processes. The discussion outlines practical obstacles and methodological limitations in applying causal inference to real-world fairness scenarios, proposing actionable solutions to enhance transparency, accountability, and fairness in algorithm-driven decisions.
keywords: Fairness, Discrimination, Causality, Anti-discrimination law
1 Introduction
Artificial Intelligence (AI) systems increasingly shape decisions across critical sectors, including finance, employment, healthcare, and justice. While AI promises efficiency and objectivity, its deployment also raises significant concerns about algorithmic discrimination, particularly when automated decisions perpetuate biases or unjustly disadvantage specific groups. Central to addressing these concerns is establishing causalityâaccurately identifying whether and how discriminatory harm arises from algorithmic processes.
This article examines the challenges and opportunities involved in applying principles of causality to ensure fairness in AI-driven decision-making from both legal and technical perspectives. The core objective is to develop a unified analytical framework that integrates causal reasoning with procedural fairness, aiding courts and regulators in effectively addressing algorithmic discrimination.
To achieve this, the article first explores foundational legal principles regarding causality and fairness, emphasizing procedural safeguards essential for fair adjudication. It then critically reviews landmark cases involving AI discriminationâsuch as Cook vs. HSBC, Pickett, Loomis, and Ewertâto illustrate the practical difficulties of proving causation when faced with opaque AI systems.
Subsequently, the discussion delves into causal inference methodologies, highlighting their potential for establishing direct or indirect evidence of discrimination through techniques such as causal discovery algorithms, counterfactual analysis, and the but-for test. The article also evaluates recent legislative efforts, particularly the European Unionâs AI Act and AI Liability Directive, analyzing their impact on evidentiary standards and procedural fairness in algorithmic discrimination cases.
Ultimately, by clearly identifying the limitations and assumptions underlying causal approaches, the article outlines practical considerations and future research directions. The proposed framework thus serves as a comprehensive resource for policymakers, legal practitioners, and AI developers committed to fostering transparency, accountability, and fairness in algorithmic decision-making.
<details>
<summary>x1.png Details</summary>
### Visual Description
\n
## Diagram: Causal Analysis for Auditing Algorithmic Fairness
### Overview
The image is a diagram illustrating three key areas within causal analysis for auditing algorithmic fairness: Measuring Discrimination, Path Specific Analysis, and Legal Evidence. Each area is represented by a title, a brief description, and a corresponding icon. The diagram is presented on a light gray background with a darker gray header.
### Components/Axes
The diagram consists of three main sections arranged horizontally.
* **Header:** "Causal Analysis for Auditing Algorithmic Fairness" - positioned at the top-center.
* **Section 1 (Left):** "Measuring Discrimination" with the subtext "Distinguishing discrimination from spurious correlations." Accompanied by an icon of a clipboard with a magnifying glass.
* **Section 2 (Center):** "Path Specific Analysis" with the subtext "Distinguishing Explaining variables from Proxy discrimination." Accompanied by an icon depicting a series of connected arrows.
* **Section 3 (Right):** "Legal Evidence" with the subtext "Establishing causal evidence in court." Accompanied by an icon of a courthouse building.
### Detailed Analysis or Content Details
The diagram does not contain numerical data or axes. It is a conceptual illustration. The icons are symbolic representations of the concepts.
* **Measuring Discrimination:** The clipboard icon suggests data collection and investigation. The magnifying glass implies a detailed examination.
* **Path Specific Analysis:** The arrows indicate a flow or relationship between variables. The dashed lines suggest potential pathways, and the solid arrows indicate confirmed or established relationships.
* **Legal Evidence:** The courthouse icon represents the application of these analyses in a legal context.
### Key Observations
The diagram presents a logical progression from identifying and measuring discrimination to understanding the underlying causal paths and ultimately using this information as legal evidence. The visual layout emphasizes the interconnectedness of these three areas.
### Interpretation
This diagram outlines a framework for auditing algorithmic fairness using causal analysis. It suggests that simply identifying discriminatory outcomes is insufficient; it's crucial to understand *why* those outcomes occur. The "Path Specific Analysis" section highlights the importance of distinguishing between variables that genuinely explain the outcome and those that are merely correlated with it (proxy discrimination). Finally, the diagram emphasizes the practical application of this analysis in legal settings, where establishing causality is essential for demonstrating unfairness. The diagram is a high-level overview and does not provide specific methodologies or techniques for conducting these analyses. It serves as a conceptual roadmap for a more detailed investigation into algorithmic fairness.
</details>
Figure 1: Causality is crucial for auditing algorithmic fairness because it accurately identifies whether automated systems produce genuinely discriminatory outcomes or merely reflect correlations; helps to disentangle the paths relating sensitive attribute and the outcome and justifiable dependency from proxy discrimination; causal analysis aligns with causal evidence for establishing liability in court-practice.
1.1 Related Work
Over the last decade, causal fairness has emerged as a important approach in the quest for equitable AI. One of the most influential works in counterfactual fairness, introduced by [Kusner et al. (2017)]. They formalized the idea that a decision is fair âif it is the same in the actual world and in a counterfactual world where the individual belonged to a different demographic groupâ [Kusner et al. (2017)]. Building on this, subsequent research refined causal fairness notions by distinguishing fair and unfair pathways of influence. [Nabi and Shpitser (2018)] proposed defining discrimination as the presence of a causal effect of a sensitive attribute on the outcome along certain disallowed paths. Similarly, [Chiappa (2019)] introduced a method to compute path-specific counterfactual fairness, explicitly quantifying the information flow along particular causal paths and ensuring that protected attributes do not impact the outcome via forbidden channels . [Zhang and Bareinboim (2018)] further contributed a causal explanation formula to decompose observed outcome disparities into portions attributable to different causal factors, enabling one to explain and quantify unfairness in decision-making. [Loftus et al. (2018b)] argued for the importance of causal reasoning when designing fair algorithms, pointing out that purely correlation-based fairness criteria may be misleading if the data is biased by historical discrimination. [PleÄko et al. (2024)] enumerates which causal assumptions are needed to test for disparate impact versus disparate treatment and connect causal fairness to legal disparate treatment framework. Carey and Wu (2022) discusses the relationship between causality-based fairness notions to legal principles, noting that many fairness criteria in machine learning echo concepts from discrimination law when interpreted causally. However, they do not link causal fairness in ML and court practice in the context of AI auditing.
Main Contributions
This paper makes several key contributions to the study of auditing fairness in AI through a causal inference perspective. Specifically, we:
- Establish the connection between causal fairness analysis and legal frameworks. We analyze how causal methods align with judicial practices, particularly within the European legal framework, and discuss the admissibility and role of causal evidence in proving algorithmic discrimination in court.
- Consolidate the arguments for causality in fairness evaluation. We provide systematization of knowledge on scenarios where purely statistical approaches can misrepresent discrimination.
- Discuss limitations of applying causality for fairness evaluation. We critically examine assumptions such as ignorability, positivity, SUTVA, and path-specific identifiability in fairness contexts.
By bridging the gap between causal inference, fairness in AI, and legal accountability, this work provides a comprehensive framework for evaluating and mitigating algorithmic discrimination.
Paper Structure
This paper is structured as follows. In \autoref sec:background we introduce making concepts of causality; in \autoref sec:measure we discuss when causality is crucial for reliably measuring discrimination; In \autoref sec:mediation we lay out the importance of mediation analysis in the context of fairness evaluation; In \autoref sec:legal we delve into the concept of causality in European law and court practice, and link to causal AI auditing. Finally, we conclude in \autoref sec:conclusion.
2 Background on Causality
In this work, we primarily adopt the structural probabilistic models framework (Pearl, 2009a) and the potential outcomes framework (Rubin, 2005). Below, we provide a technical preliminaries for causal framework and causal fairness notions.
2.1 Causal Graph
A causal graph, denoted as $\mathcal{G}=(\mathbf{V},\mathcal{E})$ , is a Directed Acyclic Graph (DAG) consisting of a set of variables or nodes $\mathbf{V}$ and edges $\mathcal{E}$ . Each edge $Xâ Y$ signifies a causal relationship, meaning changes in $X$ directly influence $Y$ . Importantly, altering $X$ impacts $Y$ , but modifying $Y$ does not affect $X$ . \autoref fig:simpleColliderJobHiring shows three basic DAG structures - confounder, mediator and collider.
2.2 Causal Fairness Notions
Causal fairness aims to ensure that sensitive attributes, such as race or gender, do not unfairly influence outcomes. Below, we describe key causal fairness notions and their formal definitions.
2.2.1 Total Effect (TE)
Total Effect (TE) (Pearl, 2009b) is a causal fairness notion that quantifies the overall effect of a sensitive attribute $X$ on an outcome $Y$ . Formally, TE is defined as:
$$
TE_{x_{1},x_{0}}(y)=\mathbb{P}(Y=y\mid do(X=x_{1}))-\mathbb{P}(Y=y\mid do(X=x_%
{0})),
$$
where $do(X=x)$ denotes an intervention that sets $X$ to $x$ . TE measures the causal impact of changing $X$ from $x_{0}$ to $x_{1}$ on $Y$ across all causal paths connecting $X$ to $Y$ .
2.2.2 Mediation Analysis: NDE, NIE, and PSE
Mediation analysis decomposes the causal effect of $X$ on $Y$ into direct and indirect effects. This is essential for identifying the pathways through which $X$ influences $Y$ .
Natural Direct Effect (NDE) (Pearl, 2001): The NDE quantifies the direct effect of $X$ on $Y$ , bypassing any mediators. For a binary variable $X$ with values $x_{0}$ and $x_{1}$ , the NDE is:
$$
NDE_{x_{1},x_{0}}(y)=\mathbb{P}(y_{x_{1},\mathbf{Z}_{x_{0}}})-\mathbb{P}(y_{x_%
{0}}),
$$
where $\mathbf{Z}$ represents the set of mediator variables, and $\mathbb{P}(y_{x_{1},\mathbf{Z}_{x_{0}}})$ is the probability of $Y=y$ if $X$ is set to $x_{1}$ while the mediators are set to values they would take under $X=x_{0}$ .
Natural Indirect Effect (NIE) (Pearl, 2001): The NIE captures the influence of $X$ on $Y$ through mediators. It is given by:
$$
NIE_{x_{1},x_{0}}(y)=\mathbb{P}(y_{x_{0},\mathbf{Z}_{x_{1}}})-\mathbb{P}(y_{x_%
{0}}),
$$
where $\mathbb{P}(y_{x_{0},\mathbf{Z}_{x_{1}}})$ represents the probability of $Y=y$ when $X=x_{0}$ but mediators take values they would under $X=x_{1}$ .
Path-Specific Effect (PSE) (Pearl, 2009b; Chiappa, 2019; Wu et al., 2019): The PSE isolates the causal effect of $X$ on $Y$ transmitted through a specific path or set of paths $\pi$ . Formally, it is defined as:
$$
PSE^{\pi}_{x_{1},x_{0}}(y)=\mathbb{P}(y_{x_{1}\mid_{\pi},x_{0}\mid_{\overline{%
\pi}}})-\mathbb{P}(y_{x_{0}}),
$$
where $\mathbb{P}(y_{x_{1}\mid_{\pi},x_{0}\mid_{\overline{\pi}}})$ is the probability of $Y=y$ if $X=x_{1}$ along path $\pi$ , while other paths ( $\overline{\pi}$ ) remain unaffected by the intervention.
3 When Causality is Needed for Reliably measuring discrimination?
Measuring discrimination without accounting for the underlying causal structure of variable relationships can lead to misleading conclusions and biased estimations of discrimination. In extreme cases, such as Simpsonâs paradox, this bias can even reverse conclusionsâfor example, a biased estimation may suggest positive discrimination when, in reality, an unbiased analysis would reveal negative discrimination.
Figures 4 - 4 illustrate the three fundamental causal structures that can result in statistical anomalies, making common fairness metrics unreliable.
<details>
<summary>x2.png Details</summary>
### Visual Description
\n
## Diagram: Causal Relationship Model
### Overview
The image depicts a diagram illustrating a causal relationship model between three variables: Political Belief, Socio-Economic Status (SES), and Selection for Job Hiring. The diagram uses a triangular structure with arrows indicating the direction of influence between the variables.
### Components/Axes
The diagram consists of three nodes, each representing a variable:
* **A:** Political Belief
* **Z:** Socio-Economic Status (SES)
* **Y:** Selection for Job Hiring
Arrows connect the nodes, indicating causal relationships. The diagram does not have traditional axes or scales.
### Detailed Analysis or Content Details
The diagram shows the following relationships:
* An arrow points from 'A' (Political Belief) to 'Z' (SES).
* An arrow points from 'Z' (SES) to 'Y' (Selection for Job Hiring).
* An arrow points from 'A' (Political Belief) to 'Y' (Selection for Job Hiring).
The nodes are positioned as vertices of a triangle. 'A' is at the bottom-left, 'Z' is at the top-center, and 'Y' is at the bottom-right. The text labels are positioned directly below or above their respective nodes.
### Key Observations
The diagram suggests a model where both Political Belief and Socio-Economic Status influence Selection for Job Hiring. Additionally, Political Belief appears to influence Socio-Economic Status. There are no numerical values or quantitative data presented in the diagram.
### Interpretation
The diagram illustrates a potential model of how political beliefs and socio-economic status might interact to influence job hiring decisions. It suggests that political beliefs can directly impact hiring decisions, but also indirectly through their influence on socio-economic status. This model could be used to explore potential biases in hiring practices. The diagram is a conceptual representation and does not provide any data to support or refute the relationships depicted. It is a theoretical framework for investigation. The absence of quantitative data means the strength or magnitude of these relationships is unknown. The diagram implies a complex interplay of factors, rather than a simple linear relationship.
</details>
Figure 2: Confounder structure.
<details>
<summary>x3.png Details</summary>
### Visual Description
\n
## Diagram: Causal Relationship Model
### Overview
The image depicts a directed acyclic graph illustrating a causal relationship model. It shows three variables: Gender, Department, and Admission in College, and their interrelationships. The diagram uses circles to represent variables and arrows to represent causal influences.
### Components/Axes
The diagram consists of three nodes labeled as follows:
* **A:** Gender
* **M:** Department
* **Y:** Admission in College
Arrows indicate the direction of influence between these variables. There are no axes or scales present.
### Detailed Analysis or Content Details
The diagram shows the following relationships:
1. Gender (A) influences Admission in College (Y). An arrow points from A to Y.
2. Department (M) influences Admission in College (Y). An arrow points from M to Y.
3. Gender (A) influences Department (M). An arrow points from A to M.
There are no numerical values or quantitative data presented in the diagram. It is a purely qualitative representation of hypothesized relationships.
### Key Observations
The diagram suggests a model where both gender and department influence admission to college, and gender also influences the choice of department. The model does not indicate any feedback loops or reciprocal causation.
### Interpretation
This diagram represents a simplified causal model, likely used in statistical modeling or research design. It posits that gender is a factor influencing both the department a student chooses and their chances of admission to college. The department chosen also influences admission. This type of diagram is often used to visualize assumptions about causal relationships before conducting statistical analysis, such as path analysis or structural equation modeling. The absence of numerical data suggests this is a conceptual model rather than a presentation of empirical findings. The model implies that understanding gender and department is crucial for understanding admission outcomes. It does not, however, suggest the *strength* of these relationships, only their direction.
</details>
Figure 3: Mediator structure.
<details>
<summary>x4.png Details</summary>
### Visual Description
\n
## Diagram: Causal Relationship Model
### Overview
The image depicts a causal relationship diagram with three variables: Political Belief (A), Selection for Job Hiring (Y), and Labor Union Activism (W). Arrows indicate the direction of influence between these variables.
### Components/Axes
The diagram consists of three labeled nodes represented by circles:
* **A:** Political Belief - positioned on the top-left.
* **Y:** Selection for Job Hiring - positioned on the top-right.
* **W:** Labor Union Activism - positioned at the bottom-center.
Arrows connect these nodes, indicating causal relationships. The arrows are bidirectional between A and W, and unidirectional from A to Y and from W to Y.
### Detailed Analysis or Content Details
* **A -> Y:** An arrow points from "Political Belief" (A) to "Selection for Job Hiring" (Y). This suggests that political belief influences job hiring selection.
* **A <-> W:** A bidirectional arrow connects "Political Belief" (A) and "Labor Union Activism" (W). This indicates a reciprocal relationship, where political belief influences labor union activism, and vice versa.
* **W -> Y:** An arrow points from "Labor Union Activism" (W) to "Selection for Job Hiring" (Y). This suggests that labor union activism influences job hiring selection.
### Key Observations
The diagram highlights the interconnectedness of political beliefs, labor union activism, and job hiring practices. The bidirectional relationship between political belief and labor union activism suggests a complex interplay between these two factors.
### Interpretation
The diagram suggests a model where political beliefs can directly influence job hiring selections, and also indirectly through their impact on labor union activism. Labor union activism, in turn, also influences job hiring selections. This model could be used to explore potential biases in hiring practices based on political affiliation or union involvement. The reciprocal relationship between political belief and labor union activism suggests that individuals' political views may shape their engagement in union activities, and their involvement in unions may, in turn, reinforce or modify their political beliefs. The diagram does not provide any quantitative data or specific details about the strength or nature of these relationships; it simply illustrates a proposed causal structure.
</details>
Figure 4: Collider structure.
3.1 Confounder structure
The first scenario in which ignoring the causal structure of the data can lead to an unreliable estimation of discrimination arises from failing to account for a confounding variable. Consider the hypothetical example in Figure 4, which depicts an automated system used to select candidates for job positions.
Suppose the system takes two input features: socio-economic status (SES), denoted as $Z$ , and the candidateâs political belief, denoted as $A$ . The outcome, $Y$ , represents whether the candidate is selected for the next stage of hiring (or the probability of selection). The SES influences the hiring outcome because candidates from higher SES backgrounds may have greater access to reputable academic institutions and expensive training programs.
Both SES ( $Z$ ) and political belief ( $A$ ) can be either binary (e.g., rich vs. poor for SES, liberal vs. conservative for political belief) or continuous (e.g., the degree of wealth for SES and the extent of conservatism for political belief). In this example, we assume that a candidateâs political belief is influenced solely by their SES.
Now, suppose the automated decision system is suspected of being biased against candidates with a particular political belief. The claim is that the system is more likely to select candidates who hold a specific political ideology. However, failing to account for the confounding effect of SES may lead to incorrect conclusions about whether the system is truly biased.
A simple approach to assessing the fairness of the automated selection process, represented by $Y$ , with respect to the sensitive attribute $A$ , is to compare the conditional probabilities:
$$
\mathbb{P}(Y=1\mid A=0)\quad\text{and}\quad\mathbb{P}(Y=1\mid A=1)
$$
This comparison, known as statistical disparity, quantifies the difference in selection rates between the two groups (e.g., conservatives and liberals). However, such an estimation of discrimination is biased due to the confounding effect of $Z$ .
Since $Z$ influences both the sensitive variable $A$ and the outcome $Y$ , it introduces a correlation between $A$ and $Y$ that is not causal. In other words, candidates with a high socio-economic status (SES) are more likely to have conservative political beliefs and, at the same time, have a higher probability of being selected due to their access to better academic institutions and training opportunities. This results in an observed correlation in the data: employers will see a greater proportion of candidates with conservative political beliefs and a lower proportion of those with liberal beliefs. However, this correlation is driven by the confounder $Z$ and should not be misinterpreted as discrimination.
Most statistical fairness metrics, such as equal opportunity and predictive parity, fail to account for such confounding effects, making them unsuitable for measuring discrimination in the presence of these statistical anomalies.
3.2 Mediator structure
The second scenario in which failing to account for the causal structure of data leads to unreliable discrimination estimation arises from the presence of one or more mediator variables. The core issue is whether discrimination transmitted through a mediator variable should be considered justifiable or not. Similar to confounding structures, a mediator variable can result in Simpsonâs paradox.
A well-known example of Simpsonâs paradox caused by a mediator structure is the gender bias observed in the 1973 Berkeley graduate admissions study (Bickel et al., 1975; Loftus et al., 2018a). Figure 4 illustrates the causal graph underlying the data, where the sensitive variable ( $A$ ) represents gender, the outcome ( $Y$ ) represents admission to Berkeley graduate programs, and a single mediator variable ( $M$ ) represents the department to which a candidate applied.
In 1973, 44% of male applicants were admitted, compared to only 34% of female applicants. At first glance, this appears to indicate a bias against female candidates. However, when the data was analyzed separately by department, acceptance rates were found to be approximately equal across genders.
In a simple mediator structure, there are two possible causal paths from $A$ to $Y$ : a direct path, $Aâ Y$ , and an indirect path, $Aâ Mâ Y$ . Comparing the overall admission rates of male and female candidates considers both paths in the discrimination measure. In contrast, comparing admission rates within each department isolates only the direct path, $Aâ Y$ . Consequently, whether or not to account for mediator paths when measuring discrimination can lead to contradictory conclusions, as demonstrated by Simpsonâs paradox.
3.3 Collider structure
A biased estimation of discrimination can also arise due to the presence of a common effect (collider) variable and implicit conditioning on that variable during the data generation process. Using the same hypothetical job selection example, consider the causal graph in Figure 4, where $A$ and $Y$ retain the same meanings as in the previous example.
Assume that the data used to train the automated decision system is collected from various sources, but primarily from labor union records. Let $W$ represent a candidateâs labor union activism, which is influenced by both $A$ and $Y$ . On one hand, political belief ( $A$ ) affects whether a candidate becomes an active labor union member, as individuals with liberal political beliefs are more likely to join labor unions. On the other hand, if a candidate is selected (hired), they are more likely to become a labor union member, increasing the likelihood that their case appears in labor union records. Following prior work, a box around a variable ( $W$ ) indicates that data is implicitly conditioned on that variable during the data collection process.
Once again, the simple approach of comparing selection rates between the two groups (conservatives and liberals) leads to a biased estimation of discrimination due to the colliding path through $W$ . Intuitively, an individual appears in the collected data either because they hold liberal political beliefs or because they were selected for the job. Candidates who are both liberal and selected remain in the data, but conditioning on labor union activism introduces a non-causal correlation between $A$ and $Y$ . Specifically, the dataset drawn from labor union records includes fewer liberal candidates who were selected for the job compared to conservative candidates. This pattern may misleadingly suggest discrimination against liberal candidates. However, this correlation arises from the collider structure and should not be mistaken for actual discrimination.
4 Mediation Analysis for Disentangling Explaining Variables and Proxy Discrimination
<details>
<summary>x5.png Details</summary>
### Visual Description
\n
## Diagram: Causal Model of Job Hiring
### Overview
The image depicts a causal diagram illustrating relationships between several variables related to job hiring. The diagram uses nodes (circles) to represent variables and directed arrows to indicate causal relationships. There are no numerical values or scales present; it's a qualitative representation of hypothesized influences.
### Components/Axes
The diagram consists of five variables represented by labeled nodes:
* **A:** Political Belief
* **Z:** SES (Socio-Economic Status)
* **Y:** Selection for Job Hiring
* **T:** Address
* **W:** Community Service
Arrows indicate the following relationships:
* A -> Z
* A -> Y
* Z -> Y
* A -> T
* T -> W
* T -> Y
* W -> Y
### Detailed Analysis or Content Details
The diagram shows a network of causal relationships.
* **Political Belief (A)** influences both **Socio-Economic Status (Z)** and **Selection for Job Hiring (Y)** directly. It also influences **Address (T)**.
* **Socio-Economic Status (Z)** directly influences **Selection for Job Hiring (Y)**.
* **Address (T)** influences both **Community Service (W)** and **Selection for Job Hiring (Y)**.
* **Community Service (W)** directly influences **Selection for Job Hiring (Y)**.
There are no quantitative values associated with these relationships. The diagram only indicates the *presence* of a causal link, not the *strength* of the link.
### Key Observations
The diagram suggests that job hiring decisions are influenced by a complex interplay of factors, including political beliefs, socio-economic status, address, and community service. The variable "Address" appears to be a mediating factor, connecting political beliefs to community service and job hiring. The variable "Selection for Job Hiring" is the ultimate outcome, influenced by multiple upstream variables.
### Interpretation
This diagram represents a theoretical model of how various social factors might influence job hiring outcomes. It suggests that hiring is not solely based on qualifications but is also affected by broader societal factors like political beliefs and socio-economic status. The inclusion of "Address" and "Community Service" suggests that geographic location and civic engagement may also play a role.
The model could be used to investigate potential biases in the hiring process. For example, if individuals with certain political beliefs are systematically disadvantaged in the hiring process, this diagram provides a framework for understanding how that might occur. The model also highlights the potential for interventions aimed at promoting fairness and equity in hiring.
The diagram is a simplified representation of reality. It does not account for other potentially relevant factors, such as education, experience, or skills. It also assumes that the causal relationships are unidirectional, which may not always be the case. However, it provides a useful starting point for understanding the complex factors that influence job hiring outcomes.
</details>
Figure 5: Causal graph with two mediated paths.
In the presence of one or more mediator variables, it is important to determine how much of the observed discrimination is direct and how much is mediated. More specifically, understanding the extent to which discrimination is transmitted through each mediator variable is crucial. Mediation analysis aims to distinguish the different causal pathways through which discrimination propagates and quantify the proportion of discrimination conveyed through each path.
Consider a variation of the job hiring example, illustrated in Figure 5, which includes two mediator variables: address ( $T$ ) and community service participation ( $W$ ). In this scenario, there are four distinct causal paths from the sensitive variable $A$ (political belief) to the outcome variable $Y$ (job hiring):
- $Aâ Zâ Y$ : confounding path
- $Aâ Y$ : direct path
- $Aâ Tâ Y$ : indirect path through $T$
- $Aâ Wâ Y$ : indirect path through $W$ .
The first confounding path is non-causal, meaning that any effect transmitted through it should not be considered when estimating discrimination. As discussed in Section 3.1, this spurious effect arises from the way data is generated or collected and, therefore, should not be counted as actual discrimination.
The direct path exists whenever there is an edge between $A$ and $Y$ . The effect through $Aâ Y$ is always discriminatoryâit can never be justified or considered acceptable discrimination.
The two remaining paths are indirect paths passing through mediator variables. Whether discrimination along an indirect path is justifiable depends on the nature of the mediator variable. For instance, in the job hiring example shown in Figure 5, $T$ (home address) acts as a mediator because, on one hand, political inclination may influence where a candidate lives, and on the other hand, hiring decisions may be influenced by a candidateâs address. Similarly, $W$ (community service) is a mediator because a candidateâs political beliefs may impact their level of involvement in community service, while an employer may consider community service records as an indicator of suitability for a given position.
Discrimination along the path $Aâ Wâ Y$ can be considered justifiable, as an employer may argue that disparities between candidates with different political beliefs arise from differences in their community service records. However, discrimination through the path $Aâ Tâ Y$ is typically not acceptable, as an employer cannot justify hiring decisions based on candidatesâ addresses. In this context, $T$ is referred to as a proxy variable, whereas $W$ is called an explaining variable. If a single causal path consists of two or more mediator variables, the presence of at least one explaining variable among the mediators makes discrimination along that path justifiable and, therefore, acceptable.
Causality, through the concepts of intervention and counterfactual reasoning, provides the necessary tools to distinguish between different types of discrimination based on their causal pathways. By intervening on $A$ , all paths that include an incoming edge to $A$ âsuch as confounding paths between $A$ and $Y$ âare blocked. Discrimination transmitted through causal paths is quantified by the total effect ( $TE$ ) ( \autoref eq:TE). In the causal graph depicted in Figure 5, the $TE$ expression captures discrimination along all causal paths except for the non-causal confounding path $Aâ Zâ Y$ .
To distinguish between direct and indirect discrimination, two key expressions can be used: the natural direct effect ( $NDE$ ) (\autoref eq:NDE) and the natural indirect effect ( $NIE$ ) (\autoref eq:NIE) (Pearl, 2001). Intuitively, $NDE$ is counterfactual quantity because it corresponds to a candidate who is conservative ( $A=1$ ) along the direct path $Aâ Y$ but liberal ( $A=0$ ) along all indirect paths. This isolates the direct effect of $A$ on $Y$ while holding indirect pathways constant.
Here, $NIE$ represents the probability of a counterfactual situation where $Y=y^{\scriptscriptstyle+}$ had $A$ been $0 0$ along the direct path while indirect path took the value it would naturally have if $A=1$ . This captures the portion of discrimination that is mediated through indirect paths.
Finally, discrimination conveyed through specific indirect paths can be identified using the path-specific effect ( $PSE$ ) (\autoref eq:PSE) (Pearl, 2001; Chiappa, 2019). For example, $PSE$ quantifies the extent to which a sensitive attribute influences a decision through the mediator âaddressâ compared to the mediator âcommunity service.â This has direct implications for fairness assessment.
Mediation analyis allows for a more granular analysis of discrimination by isolating the contribution of specific causal pathways, enabling a deeper understanding of the mechanisms through which bias is propagated in automated decision-making systems.
5 Uncovering causality through legal evidence: the regulatory approach in the European Union
The method of using mediator structures to uncover causation, as discussed in the previous section, is undoubtedly a valuable model for establishing causality in judicial cases involving algorithmic discrimination. However, an important question arises: does procedural law, particularly within the European Union (EU), support such an analysis?
As a preliminary observation, it is essential to clarify that in legal contexts, the term causal fairness generally refers to the procedural conditions under which instances of fairness (or unfairness) are causally represented. With this in mind, this section focuses on two critical and interrelated issues: evidence and procedural fairness. In the EU, the Independent High-Level Expert Group on AI, established by the European Commission, defines procedural fairness as the ability to contest and seek effective redress against decisions made by AI systems and their human operators. See HLEG, Ethics Guidelines for Trustworthy AI, available at \url https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai, p. 13.
From a legal perspective, causality is a question of fact, requiring legally established discovery procedures and corresponding reasoning models designed to yield accurate causal representationsâi.e., to distinguish genuine causation from a myriad of correlations (positive associations between candidate causes and a harm suffered) (Haack, 2014).
However, in adjudicatory contexts, causality is established primarily to serve the purpose of fairness, typically in the form of compensation as a fair remedy for harm suffered. Legal systems committed to the rule of law In the EU, the concept of the rule of law encompasses the following principles: legality, legal certainty, prohibition of arbitrariness by executive powers, independent and impartial courts, effective judicial review (including respect for fundamental rights), and equality before the law. See Communication from the Commission to the European Parliament and the Council, âA New EU Framework to Strengthen the Rule of Law,â COM(2014) 158 final, p. 4. share a commitment to procedural fairness, based on the normative principle that only fairly designed procedures can lead to fair outcomes.
In contemporary systems of evidence and judicial remedies, including those within EU law, the parallelism between fair procedures and fair outcomes is epitomized in fair trial safeguards. These procedural entitlements are intended to uphold basic equality (or procedural parity) and ensure the effectiveness of litigantsâ participation in dispute resolution (Solum, 2004). This principle applies not only to equal access to judicial remedies but also to equal access to and the ability to present evidence. The fundamental idea is that neither party should be advantaged or disadvantaged in terms of access to the facts necessary to present their caseâtypically before a court.
In summary, causal fairness in law requires accurateâor at the very least, plausible (Rescher, 1980) âevidence of causality, presented under conditions of procedural fairness.
The proof of causality in cases of algorithmic discrimination has profoundly challenged longstanding legal postulates. From a procedural fairness perspective, a major issue arises from the opacityâwhether relative or totalâof AI systems, which renders their decision-making processes inscrutable and obstructs victimsâ ability to properly establish and argue causation.
One prominent example in this regard is Cook v. HSBC North America, U.S. District Court for the Northern District of Illinois, 21 March 2014, County of Cook v. HSBC North America Holdings Inc et al., 1:2014cv02031. a credit scoring case in which the system used applicantsâ places of residence as a relevant variable, ultimately favoring predominantly white neighborhoods while discriminating against ethnic minorities. The association of subtly discriminatory variablesâsuch as zip code and ethnic backgroundâcombined with practical difficulties in accessing information about how the AI system weighted different variables, severely undermined fundamental fair trial safeguards, particularly the right to access evidence and courts.
To address these challenges, regulators worldwide, including in the EU, have sought to answer two key questions:
1. What evidence must litigants have access to in order to effectively prove causation?
1. Once that evidence is identified, how should legal procedures be (re)designed to ensure victimsâ access to it?
These questions are central to ensuring that legal systems remain capable of addressing algorithmic discrimination while upholding principles of procedural fairness.
5.1 Methods for Establishing Causality in Algorithmic Discrimination Cases
Regarding the first question, emergingâthough not yet consolidatedâglobal AI liability case law reveals an interesting trend. While numerous judicial instances could be citepd, for the purpose of this article, we highlight three cases that illustrate the ânew approachâ to proving causation in AI-related disputes. These cases are Pickett, Superior Court of New Jersey (Appellate Division), 2 February 2021, State of New Jersey v. Corey Pickett, Docket N° A-4207-19T4. (which concerns the use of a DNA matching systemâTrueAlleleâby law enforcement to track down suspects), Loomis, Supreme Court of Wisconsin, 13 July 2016, State of Wisconsin v. Eric L. Loomis, 881 N.W.2d 749 (2016) 2016 WI 68. (which involves COMPAS, a recidivism-prediction system used by courts), and Ewert, Ewert v. Canada, 2018 SCC 30, File N° 37233, 13 June 2018. (which also deals with the use of recidivism-predicting systems by Canadian correctional services).
In all three cases, the plaintiffs argued that the automated decisions were inaccurate because they were unfairâthat is, they contained unfair biases: gender bias in Pickett and Loomis, and ethnic bias in Ewert. To uncover the bias-inducing variable associations (i.e., the causal link), the plaintiffs requested that the systems be reverse-engineered. However, this proved nearly impossible. For instance, in Pickett, independent experts confirmed that reverse-engineering the system would take up to 8.5 years to complete. See Superior Court of New Jersey (Appellate Division), 2 February 2021, State of New Jersey v. Corey Pickett, Docket N° A-4207-19T4, at 17. The emerging field of mechanistic interpretability offers methods to inspect models, such as LLMs internal representations without full reverse engineering (Bereska and Gavves, 2024). However, the causal relationship between concepts and representations is hard to establish and is still an open problem for the future research (Sharkey et al., 2025; Binkyte et al., 2025).
Faced with the practical infeasibility of reverse-engineering, the courts in Pickett (and in Loomis) turned to general expertise as a faute de mieux solution. The absence of direct evidence (through reverse-engineering) that could reveal unfair bias was âcompensatedâ by relying on pre-existing expert assessments of the systemâs general functionality. If the majority of experts agreed that a systemâsuch as TrueAllele in Pickett or COMPAS in Loomis âwas generally well-performing (i.e., unbiased and therefore accurate), courts were inclined to presume that the systemâs decisions in the specific cases under dispute were also unbiased.
Thus, the role of experts is to assess the strength of the causal link between sensitive variables and the decision. In the presence of different causal structures (Section 3), which may lead to various types of bias, a zero causal effect would indicate the absence of discrimination. A possible approach to uncovering this causal link is to first identify the causal graph that represents the relationships between variables. This can be achieved by:
1. Using causal discovery techniques to infer an initial causal graph.
1. Consulting domain experts to refine the discovered graphâsuch as adding or removing causal links and enforcing domain-specific assumptions.
Expert input is also essential for clarifying the role of each variable, particularly in classifying mediator variables as either explaining (leading to justifiable discrimination) or proxy (leading to unjustifiable discrimination) variables. This classification is crucial for selecting the appropriate causal fairness metric (Section 4) to assess discrimination.
From the perspectives of procedural fairness and the mediator structure model, this trend is open to criticism. First, general expert opinions on a systemâs accuracy are not as probative as direct evidence (such as reverse engineering), which could provide highly reliable information on the mediator associations leading to a discriminatory outcome. Second, the inability to prove causation through reliable evidence appears to have led to a peculiar application of the so-called but-for test.
In principle, the but-for test employs counterfactual reasoning to determine whether a harm would have occurred if an alleged cause had not been present. For example, in Cook v. HSBC (a credit scoring case), a standard application of this test would involve determining whether the same loan applicants would have been approved had the system not considered their place of residence as a relevant variable.
However, the cases cited in this section (particularly Pickett and Loomis) reveal a shift in how the but-for test is applied. In conventional (non-AI-related) disputes, the test seeks to answer a factual question about causal association: would the outcome have been the same (or different) if certain facts (such as address, gender, or age) were absent from the causal structure? In AI-related disputes, however, the but-for test is reframed to address a different causal question: would a human decision relying on AI output have been the same or different if the AI system had not been used at all?
In this context, statistical causality tools can be applied to detect discrimination in data reflecting previous hiring or loan-granting practices within the company. If an association between the sensitive attribute and the outcome is identified, one can infer that the decision would remain the same even without algorithmic assistance. This shifts the focus from scrutinizing the AI system (and its designers) to assessing broader company practices. Here, causality analysis helps differentiate between spurious associations, explainable disparities, and actual discrimination.
Conversely, if the data used to train the AI system contains ingrained discrimination, compliance with AI design guidelines warrants further examination. Finally, causality tools offer formal mathematical expressions that capture the otherwise intangible concept of counterfactual reasoning (Shpitser and Pearl, 2007, 2008), making them particularly useful for directly verifying the but-for test.
This allows us to raise the second issue mentioned above: should systems of evidence include a right to access/to request disclosure of evidence?
5.2 Disclosing causal evidence to victims of discrimination
Given these practical and theoretical challenges associated with the but-for test and counterfactual reasoning in AI-related discrimination cases, an important question emerges regarding how evidence disclosure mechanisms might facilitate fairer judicial outcomes. In particular, should litigants possess a specific right to access evidence that would allow effective causal analysis? From a procedural fairness perspective, the right to request evidence disclosure is crucial for a victim of algorithmic discrimination, providing at least an opportunity to seek the lifting of the opacity veil that may obscure a causal chain (Grozdanovski, 2021). In the EU, recent regulatory developments appearâat least on the surfaceâto be moving toward recognizing such a right.
The first major step was the introduction of the AI Act, Proposal for a Regulation of the European Parliament and of the Council laying down harmonized rules on Artificial Intelligence (AI Act) and amending certain Union legislative acts, COM(2021) 206 final. a horizontal, cross-sectoral legislation that makes two significant contributions. First, it establishes a four-level taxonomy of AI-related risks: non-high, limited, high, and unacceptable. Second, based on this risk classification, the AI Act introduces a set of technical standardsâincluding transparency, data governance, and risk-mitigation strategiesâspecifically targeting high-risk AI systems deployed in eight key market sectors. The âhigh-riskâ sectors, listed in Annex III of the AI Act, include employment, education, healthcare, transport, energy, public sector (including asylum, migration, border controls, judiciary, and social security services), defense and security, finance, banking, and insurance.
To complement the AI Act and to provide procedural avenues for compensating harm caused by high-risk AI systems, the EU introduced the AI Liability Directive (AILD). Proposal for a Directive of the European Parliament and of the Council on adapting non-contractual civil liability rules to Artificial Intelligence (AI Liability Directive), COM(2022)496 final. This directive establishes an evidentiary framework granting victims the right to request disclosure of evidence. Under the AILD, if the defendant (a programmer or user) refuses to disclose the requested evidence, or if a national or EU court finds that the disclosed evidence is both probative and plausible, the defendant is presumed responsible for the harm (e.g., discrimination) suffered by the claimant.
However, it is important to note that the type of evidence a victim may request under the AILD does not include the kind of evidence previously identified as necessary (i.e., expert analysis) in the cases discussed earlier. The AILD permits disclosure only of evidence related to the defendantâs compliance with the technical standards outlined in the AI Act. In other words, defendants would not be required to provide information such as access to the AI systemâs code or, where feasible, allow for reverse engineering to support a proper causal analysis. Instead, they would merely be asked to demonstrate compliance with obligations such as ensuring human oversight and control.
The rationale behind this limitation is that the AILD operates under the assumption that if harmâsuch as discriminationâoccurs, it is because the AI Act was not fully adhered to. As a result, the AILD effectively narrows the scope of evidentiary debates in AI discrimination cases. Instead of compelling parties to uncover the actual causal structure underlying a discriminatory AI outcome, legal proceedings will primarily focus on identifying the human agent who failed to meet a legally prescribed duty of care.
6 Challenges and Opportunities in Using Causality for Fairness
While recent regulatory developments such as the AI Act and AI Liability Directive represent significant progress in addressing algorithmic discrimination, their implementation relies fundamentally on the practical feasibility of causal analysis. Thus, understanding the methodological constraints and opportunities of causal inference becomes essential to ensure that regulatory efforts lead to meaningful improvements in fairness. Many causal requirements can, in principle, be satisfied through specific experimental designsâideally, through random assignment. However, in fairness-related scenarios, such experimental setups are often impractical or ethically infeasible. As a result, discrimination is typically evaluated using observational data.
In this section, we outline some key requirements for applying causal inference, particularly those most relevant to fairness applications.
6.1 Availability of Causal Graph
The effective use of causal inference in fairness applications is shaped by several critical methodological requirements and assumptions. Among these, one of the most significant is the availability of an accurate directed acyclic graph (DAG) representing relationships among variables. The DAG is subject to additional assumptions, including the causal Markov condition, causal faithfulness, and causal sufficiency. Together, these conditions encode the same requirements as SUTVA and ignorability in the potential outcome framework and, therefore, will not be discussed separately. (Pearl, 2009a). Research by (BinkytÄ-SadauskienÄ et al., 2022) highlights significant disagreements in causal fairness estimations due to minor variations in the assumed causal structure.
The availability of a DAG is particularly crucial in the presence of collider structures, as conditioning on a collider induces bias in causal effect estimation (Pearl, 1988). Additionally, DAGs are essential for evaluating path-specific effects, which play a key role in distinguishing between redlining and explaining variables in fairness scenarios.
Causal structures (or causal graphs) can be obtained through two main approaches: expert consultation or learning from observational data. However, both approaches have limitations. Domain experts may hold differing views or introduce biased assumptions.
Learning causal relationships from data, on the other hand, often requires additional assumptions regarding data distribution, functional dependencies, relationships between exogenous unobserved variables, and an informed choice of the learning algorithm. Research by (BinkytÄ-SadauskienÄ et al., 2022) demonstrates how different causal discovery algorithms, when applied to the same dataset, can yield significantly different causal structures. This variability raises concerns about the reliability of causality learned purely from observational data (Guyon et al., 2011).
Expert-driven causal graphs, while useful, are not easily scalable and may be prone to human error. However, recent advancements in causal discovery with large language models (LLMs) offer a promising alternative (Kıcıman et al., 2023; Kasetty et al., 2024; Vashishtha et al., 2023). Hybrid approaches that combine LLM-driven insights with statistical causal discovery methods appear to be particularly promising in bridging the gap between expert-driven and data-driven causal inference (Afonja et al., 2024).
6.2 Causal Transportability
Beyond the accurate identification of causal graphs, another crucial consideration when applying causal inference to fairness is whether causal findings obtained in one context remain valid and generalizable across diverse populationsâan issue addressed by the concept of causal transportability. Currently, the application of causal knowledge across populations relies heavily on assumptions about general causal structures that remain relatively stable across different contexts. Examples include physical laws, such as âaltitude causes temperature,â or well-established medical principles, such as âbacteria cause disease.â
However, the principles underlying ethical AI are often deeply intertwined with cultural values and societal norms, which are neither immutable nor universally consistent. These variations present significant challenges in transferring causal insights across different demographic or geographic populations.
The framework developed by Pearl and Bareinboim utilizes directed acyclic graphs (DAGs) to adjust causal knowledge for new settings through targeted data collection (Bareinboim and Pearl, 2014; Pearl and Bareinboim, 2011). Building on this, Binkyte et al. (2024) propose an expectation-maximization (EM) approach to adapt causal knowledge for applications in target demographic groups, further enhancing the practicality of causal transportability in real-world scenarios.
6.3 Possibility for Intervention
In fairness estimation, the sensitive attribute is typically considered the exposure or treatment variable, with the goal of measuring its impact on the outcome.
Most definitions of causal effect are based on the notion of intervention or manipulation of a cause variable (exposure) (VanderWeele and HernĂĄn, 2012; Holland, 1986). This presents a challenge when making causal claims about non-manipulable attributes, such as race or gender.
Some approaches in the literature suggest shifting the focus from actual manipulation to changes in perception (Rahmattalabi and Xiang, 2022). For instance, instead of altering the gender of candidates to estimate its effect on hiring decisions, researchers could manipulate the employerâs perception of gender. This could be achieved by submitting two otherwise identical rĂ©sumĂ©s with different names or titles indicating different genders. Such an approach aligns with methodologies used in social experiments examining the impact of race or gender on hiring decisions (Neumark, 2018).
(VanderWeele and HernĂĄn, 2012) further differentiates immutable sensitive attributes into those that are randomized at birth (e.g., biological sex) and those that are not (e.g., race, social gender). This distinction is important when estimating the causal effect of a sensitive attribute. If the attribute is randomized, its causal effect on an outcome can be estimated by directly comparing exposure levels. For example, the total causal effect of biological sex on an outcome can be estimated by comparing observed differences between men and women (VanderWeele and HernĂĄn, 2012).
In contrast, race is not randomly assigned but influenced by ancestral and socio-cultural factors, making causal effect estimation more complex. Even at the biological level, estimating the effect of race requires a thorough understanding of the causal structure of relevant covariates. Such estimation is particularly relevant in medical scenarios, where assuming independence between the sensitive attribute and the outcome (e.g., disease probability) is unreasonable.
Similarly, in potential discrimination scenarios, (VanderWeele and HernĂĄn, 2012) and (Rahmattalabi and Xiang, 2022) suggest shifting the focus to the direct effect of the perceived gender or race on decision-making. This perspective aligns with causal fairness approaches that assess discrimination based on how sensitive attributes are perceived and acted upon within decision-making processes.
6.4 Causal assumptions
The Stable Unit Treatment Value Assumption (SUTVA) (Rubin, 1986) imposes two key requirements: no interference and consistency.
The no interference assumption states that the effect of the sensitive attribute on the outcome should not be influenced by interactions between individuals. However, in social sciences researchâwhere interactions and feedback loops are commonâthis assumption is often challenged, necessitating careful discussion and restricted interpretations of causal estimates (Morgan and Winship, 2015).
Since fairness is typically evaluated within a social context, potential interactions must be carefully assessed. For example, in the hiring scenario, a violation of the no interference assumption would occur if hiring individuals from one political spectrum increased the likelihood of favoring candidates with the same political beliefs in future hiring decisions. This scenario is plausible because current employees may prefer candidates who share their political ideology, creating a feedback loop that perpetuates bias.
The consistency assumption requires that each treatment level leads to the same potential outcomes (Keele, 2015). In fairness evaluation, the treatment variable is replaced by the sensitive attribute, which is often a socially constructed category such as race or gender. Identifying the causal effect of gender on hiring becomes problematic if gender itself does not have a consistent effect on hiring decisions. For instance, discrimination may not apply uniformly to all women; rather, only women who exhibit a certain level of âfemininityâ might experience bias. If this possibility cannot be ruled out, it should be considered in studies aiming for a fine-grained causal analysis.
In summary, SUTVA assumptions are likely to be violated in fairness scenarios. However, causal approaches can still be applied if the results are interpreted with caution. Some methods for identifying causal effects despite SUTVA violations are discussed in (Laffers and Mellace, 2020).
The ignorability assumption (Rubin, 1986) requires that the sensitive attribute and the outcome be independent, given the observable variables. In other words, no unobserved variables should create a significant link between the sensitive attribute and the outcome.
In fairness evaluation, the presence of such a link could indicate that a portion of the observed discrimination is, in fact, a spurious effect induced by a confounding variable. For example, if education is a confounder but is not included in the dataset, its confounding effect cannot be controlled. Consequently, it becomes impossible to estimate the causal effect of political belief on hiring decisions separately from the effect of education.
Unobserved confounders are less likely to exist for immutable sensitive attributes such as sex or race, as these attributes generally do not have temporally prior causes. However, dependency between noise terms related to the sensitive attribute and the outcome can still violate ignorability.
(Fawkes et al., 2022) highlight the implications of assuming ignorability when using causal counterfactuals. Following their reasoning, consider the case of college admissions (illustrated in Figure 4). Under the ignorability assumption, an average male applicant to a technical profession could be counterfactually replaced with an average female applicant to the same profession. However, due to social expectations tied to gender roles, a woman applying to a technical profession is likely to be more motivated and hardworking than an average male applicant with the same professional aspirations. This example illustrates how violations of ignorability can introduce bias into causal estimations.
The positivity assumption (Rubin, 1986) is violated if certain combinations of a sensitive attribute and a covariate have zero probability. Violations of positivity can be either deterministic or random (Westreich and Cole, 2010).
For example, positivity would be violated if a specific level of education always corresponded to liberal political beliefs. In this scenario, the violation would likely be random, as it is improbable that education would have a strictly deterministic relationship with political beliefs. In such cases, statistical methods exist for handling analysis under violations of positivity (Westreich and Cole, 2010).
However, consider a case where obtaining a Harvard degree is treated as an explanatory mediator between ethnicity and hiring. Due to historical patterns of discrimination and socioeconomic disparities, certain ethnic groups may have near zero probability of obtaining a Harvard degree. This constitutes a deterministic violation of positivity. In such cases, it is essential to reconsider whether possessing a Harvard degree should be an absolute requirement for the job in question, given its potential for exclusion and disparate impact.
The identifiability of path-specific effects in the presence of multiple mediators requires the absence of causal links between the mediators (VanderWeele and Vansteelandt, 2014). Evaluating path-specific effects is particularly important for understanding the mechanism through which the sensitive attribute influences the outcome. As discussed earlier (Section 4), the effect of the sensitive attribute can be considered either justifiable or discriminatory depending on the mediating variables along the causal path.
However, in fairness scenarios, causal dependencies between mediators are common. For example, consider a case where the relationship between race and hiring decisions is mediated by both social status (e.g., redlining) and education (an explaining variable). It is highly probable that education level is influenced by social status. In such a scenario, the indirect effects of race through social status and education separately are not identifiable, as the mediators are causally dependent.
To address this challenge, (VanderWeele and Vansteelandt, 2014) proposed a method that treats multiple mediators jointly. In some cases, this approach can help disentangle and identify individual indirect effects in the presence of causal links between the mediators.
Recognizing these methodological challenges underscores the importance of carefully balancing technical rigor and practical applicability. With these considerations in mind, we now summarize our findings and highlight future directions for research.
7 Conclusions
This article has outlined the critical role of causality in evaluating the fairness of AI-driven decisions. We demonstrated that relying exclusively on statistical tools without causal analysis can lead to significant inaccuracies, such as overestimating, underestimating, or even reversing discrimination effects. Furthermore, we highlighted the essential connections between reliable evidence of algorithmic discrimination, legal practices in courts, and evolving regulatory frameworks, particularly within the European Union.
Specifically, we examined how causal reasoning influences legal assessments of discrimination and identified two primary forms of evidenceâ direct evidence, derived from reverse engineering AI systems, and indirect evidence. Given that direct evidence is often practically unattainable due to AI opacity, the article explores alternative causal analyses through indirect evidence. Two approaches based on but-for counterfactual reasoning were presented:
The first approach evaluates the impact of sensitive attributes or redlining mediators in algorithmic decision-making, using test data to investigate causal relationships.
The second approach assesses whether decisions would differ if AI systems were not employed at all, placing emphasis on historical decision-making practices within organizations.
We also critically discussed the causal assumptions underlying fairness assessments, emphasizing the inherent challenges, such as dependency on accurate causal graphs, practical limitations in conducting robust causal analyses, and potential biases arising from untestable assumptions.
7.1 Limitations and Future Research Directions
Several important limitations constrain the practical application of causal fairness evaluations. Most of the challenges are related to accurately specifying causal structures, the infeasibility of reverse engineering complex AI systems, and the limitations inherent in observational data. These factors restrict the effectiveness of existing causal inference methodologies. Future research should address these limitations by:
- Establishing causality in mechanistic interpretability methods allowing reliable and legally binding conclusions on fairness of modelâs internal representations.
- Refining regulatory frameworks to better align legal evidentiary standards with technical realities, thereby enhancing procedural fairness.
- Exploring hybrid approaches combining expert knowledge, causal discovery algorithms, and insights from large language models (LLMs) to improve the reliability and scalability of causal fairness assessments.
- Investigating cross-cultural validity and adaptability of causal fairness concepts, ensuring their global applicability and effectiveness in diverse societal contexts.
\acks
This work of Ruta Binkyte and Sami Zhioua was supported by the European Research Council (ERC) project HYPATIA under the European Unionâs Horizon 2020 research and innovation programme. Grant agreement n. 835294. The work of Ruta Binkyte is funded in part by PriSyn: Representative, synthetic health data with strong privacy guarantees (BMBF), grant No. 16KISAO29K. It is also supported by Integrated Early Warning System for Local Recognition, Prevention, and Control for Epidemic Outbreaks (LOKI / Helmholtz) grant. The work is also partially funded by Medizininformatik-Plattform âPrivatsphĂ€ren-schutzende Analytik in der Medizinâ (PrivateAIM), grant No. 01ZZ2316G, and ELSA â European Lighthouse on Secure and Safe AI funded by the European Union under grant agreement No. 101070617. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or European Commission. Neither the European Union nor the European Commission can be held responsible for them.
References
- Afonja et al. (2024) Tejumade Afonja, Ivaxi Sheth, Ruta Binkyte, Waqar Hanif, Thomas Ulas, Matthias Becker, and Mario Fritz. LLM4GRN: Discovering Causal Gene Regulatory Networks with LLMsâEvaluation through Synthetic Data Generation. arXiv preprint arXiv:2410.15828, 2024.
- Bareinboim and Pearl (2014) Elias Bareinboim and Judea Pearl. Transportability from multiple environments with limited experiments: Completeness results. Advances in neural information processing systems, 27, 2014.
- Bereska and Gavves (2024) Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for ai safetyâa review. arXiv preprint arXiv:2404.14082, 2024.
- Bickel et al. (1975) Peter J Bickel, Eugene A Hammel, and J William OâConnell. Sex bias in graduate admissions: Data from berkeley. Science, 187(4175):398â404, 1975.
- Binkyte et al. (2024) Ruta Binkyte, Daniele Gorla, and Catuscia Palamidessi. BaBE: Enhancing Fairness via Estimation of Explaining Variables. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1917â1925, 2024.
- Binkyte et al. (2025) Ruta Binkyte, Ivaxi Sheth, Zhijing Jin, Muhammad Havaei, Bernhardt Schölkopf, and Mario Fritz. Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models. arXiv preprint arXiv:2502.21123, 2025.
- BinkytÄ-SadauskienÄ et al. (2022) RĆ«ta BinkytÄ-SadauskienÄ, Karima Makhlouf, Carlos PinzĂłn, Sami Zhioua, and Catuscia Palamidessi. Causal Discovery for Fairness. arXiv preprint arXiv:2206.06685, 2022.
- Carey and Wu (2022) Alycia N. Carey and Xintao Wu. The causal fairness field guide: perspectives from social and formal sciences. Frontiers in Big Data, 5:892837, 2022. 10.3389/fdata.2022.892837.
- Chiappa (2019) Silvia Chiappa. Path-specific counterfactual fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7801â7808, 2019.
- Fawkes et al. (2022) Jake Fawkes, Robin Evans, and Dino Sejdinovic. Selection, ignorability and challenges with causal fairness. In Conference on Causal Learning and Reasoning, pages 275â289. PMLR, 2022.
- Grozdanovski (2021) Ljupcho Grozdanovski. In search of effectiveness and fairness in proving algorithmic discrimination in eu law. Common Market Law Review, 58(1), 2021.
- Guyon et al. (2011) Isabelle Guyon, Constantin Aliferis, Gregory Cooper, André Elisseeff, Jean Philippe Pellet, Peter Spirtes, and Alexander Statnikov. Causality workbench. In Causality in the sciences. Oxford University Press, 2011.
- Haack (2014) Susan Haack. Evidence matters: Science, proof, and truth in the law. Cambridge University Press, 2014.
- Holland (1986) Paul W Holland. Statistics and causal inference. Journal of the American statistical Association, 81(396):945â960, 1986.
- Kasetty et al. (2024) Tejas Kasetty, Divyat Mahajan, Gintare Karolina Dziugaite, Alexandre Drouin, and Dhanya Sridhar. Evaluating interventional reasoning capabilities of large language models. arXiv preprint arXiv:2404.05545, 2024.
- Keele (2015) Luke Keele. The statistics of causal inference: A view from political methodology. Political Analysis, 23(3):313â335, 2015.
- Kıcıman et al. (2023) Emre Kıcıman, Robert Ness, Amit Sharma, and Chenhao Tan. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050, 2023.
- Kusner et al. (2017) Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in neural information processing systems, 30, 2017.
- Laffers and Mellace (2020) Lukas Laffers and Giovanni Mellace. Identification of the average treatment effect when sutva is violated. Discussion Papers on Business and Economics, University of Southern Denmark, 3, 2020.
- Loftus et al. (2018a) Joshua R Loftus, Chris Russell, Matt J Kusner, and Ricardo Silva. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859, 2018a.
- Loftus et al. (2018b) Joshua R Loftus, Chris Russell, Matt J Kusner, and Ricardo Silva. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859, 2018b.
- Morgan and Winship (2015) Stephen L Morgan and Christopher Winship. Counterfactuals and causal inference. Cambridge University Press, 2015.
- Nabi and Shpitser (2018) Razieh Nabi and Ilya Shpitser. Fair inference on outcomes. In Proceedings of the⊠AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence, volume 2018, page 1931. NIH Public Access, 2018.
- Neumark (2018) David Neumark. Experimental research on labor market discrimination. Journal of Economic Literature, 56(3):799â866, 2018.
- Pearl (1988) Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann, 1988.
- Pearl (2001) Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pages 411â420, 2001.
- Pearl (2009a) Judea Pearl. Causality. Cambridge university press, 2009a.
- Pearl (2009b) Judea Pearl. Causality. Cambridge University Press, Cambridge, 2009b. ISBN 978-0-521-89560-6. 10.1017/CBO9780511803161. URL \url https://www.cambridge.org/core/books/causality/B0046844FAE10CBF274D4ACBDAEB5F5B.
- Pearl and Bareinboim (2011) Judea Pearl and Elias Bareinboim. Transportability of causal and statistical relations: A formal approach. In Twenty-fifth AAAI conference on artificial intelligence, 2011.
- PleÄko et al. (2024) Drago PleÄko, Elias Bareinboim, et al. Causal fairness analysis: a causal toolkit for fair machine learning. Foundations and TrendsÂź in Machine Learning, 17(3):304â589, 2024.
- Rahmattalabi and Xiang (2022) Aida Rahmattalabi and Alice Xiang. Promises and challenges of causality for ethical machine learning. arXiv preprint arXiv:2201.10683, 2022.
- Rescher (1980) Nicholas Rescher. Plausible reasoning: An introduction to the theory and practice of plausibilistic inference. Philosophy and Rhetoric, 13(3), 1980.
- Rubin (1986) Donald B Rubin. Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81(396):961â962, 1986.
- Rubin (2005) Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322â331, 2005.
- Sharkey et al. (2025) Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496, 2025.
- Shpitser and Pearl (2007) Ilya Shpitser and Judea Pearl. What counterfactuals can be tested. In 23rd Conference on Uncertainty in Artificial Intelligence, UAI 2007, pages 352â359, 2007.
- Shpitser and Pearl (2008) Ilya Shpitser and Judea Pearl. Complete identification methods for the causal hierarchy. Journal of Machine Learning Research, 9(Sep):1941â1979, 2008.
- Solum (2004) Lawrence B Solum. Procedural justice. S. CAl. l. reV., 78:181, 2004.
- VanderWeele and Vansteelandt (2014) Tyler VanderWeele and Stijn Vansteelandt. Mediation analysis with multiple mediators. Epidemiologic methods, 2(1):95â115, 2014.
- VanderWeele and HernĂĄn (2012) Tyler J VanderWeele and Miguel A HernĂĄn. Causal effects and natural laws: towards a conceptualization of causal counterfactuals for nonmanipulable exposures, with application to the effects of race and sex. Causality: statistical perspectives and applications, pages 101â113, 2012.
- Vashishtha et al. (2023) Aniket Vashishtha, Abbavaram Gowtham Reddy, Abhinav Kumar, Saketh Bachu, Vineeth N Balasubramanian, and Amit Sharma. Causal inference using llm-guided discovery. arXiv preprint arXiv:2310.15117, 2023.
- Westreich and Cole (2010) Daniel Westreich and Stephen R Cole. Invited commentary: positivity in practice. American journal of epidemiology, 171(6):674â677, 2010.
- Wu et al. (2019) Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong. Pc-fairness: A unified framework for measuring causality-based fairness. In Advances in Neural Information Processing Systems, pages 3404â3414, 2019.
- Zhang and Bareinboim (2018) Junzhe Zhang and Elias Bareinboim. Fairness in decision-makingâthe causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.