## Box Plot: Causal Effect (ATE) under Different Scenarios ### Overview The image presents a series of box plots comparing the Causal Effect (ATE) under six different scenarios: Biased, Direct-Effect, Indirect-Effect, Fair Observable, Fair Unobservable, and Fair Additive Noise. Each scenario displays the distribution of the ATE for four different methods: FairPFN, EGR, Unaware, and Unfair. The average rank of each method is also provided in the legend. ### Components/Axes * **Y-axis:** Causal Effect (ATE), ranging from -0.5 to 0.75 with increments of 0.25. * **X-axis:** Implicitly represents the four methods (FairPFN, EGR, Unaware, Unfair) within each scenario. * **Box Plots:** Represent the distribution of ATE for each method within each scenario. * **Titles:** Each plot is titled with a scenario name (e.g., "1. Biased"). * **Legend (Bottom):** * FairPFN (Pink): Avg. Rank (ATE) = 1.88/4 * EGR (Purple): Avg. Rank (ATE) = 2.11/4 * Unaware (Orange): Avg. Rank (ATE) = 2.16/4 * Unfair (Blue): Avg. Rank (ATE) = 3.42/4 ### Detailed Analysis **1. Biased:** * Unfair (Blue): The median is slightly above 0, with a box extending from approximately 0 to 0.25. Outliers extend up to approximately 0.75 and down to -0.25. * Unaware (Orange): The median is slightly below 0, with a box extending from approximately -0.05 to 0.2. Outliers extend up to approximately 0.5 and down to -0.25. * EGR (Purple): The median is slightly above 0, with a box extending from approximately -0.1 to 0.1. Outliers extend up to approximately 0.25 and down to -0.25. * FairPFN (Pink): The median is approximately 0, with a very small box. Outliers are clustered around 0. **2. Direct-Effect:** * Unfair (Blue): The median is approximately 0.25, with a box extending from approximately 0.1 to 0.5. * Unaware (Orange): Not present in this scenario. * EGR (Purple): The median is approximately 0, with a very small box. * FairPFN (Pink): Not present in this scenario. **3. Indirect-Effect:** * Unfair (Blue): The median is approximately 0.2, with a box extending from approximately 0 to 0.25. Outliers extend up to approximately 0.75 and down to -0.25. * Unaware (Orange): The median is approximately 0.1, with a box extending from approximately 0 to 0.25. Outliers extend up to approximately 0.5 and down to -0.25. * EGR (Purple): The median is approximately 0, with a box extending from approximately -0.1 to 0.1. Outliers extend up to approximately 0.25 and down to -0.25. * FairPFN (Pink): The median is approximately 0, with a very small box. Outliers are clustered around 0. **4. Fair Observable:** * Unfair (Blue): The median is approximately 0.2, with a box extending from approximately 0 to 0.25. Outliers extend up to approximately 0.75 and down to -0.1. * Unaware (Orange): The median is approximately 0.05, with a box extending from approximately 0 to 0.1. Outliers extend up to approximately 0.25 and down to -0.1. * EGR (Purple): The median is approximately 0, with a box extending from approximately -0.05 to 0.05. Outliers extend up to approximately 0.25 and down to -0.25. * FairPFN (Pink): The median is approximately 0, with a very small box. Outliers are clustered around 0. **5. Fair Unobservable:** * Unfair (Blue): The median is approximately 0.2, with a box extending from approximately 0 to 0.25. Outliers extend up to approximately 0.75 and down to -0.1. * Unaware (Orange): The median is approximately 0.05, with a box extending from approximately 0 to 0.1. Outliers extend up to approximately 0.25 and down to -0.1. * EGR (Purple): The median is approximately 0, with a box extending from approximately -0.05 to 0.05. Outliers extend up to approximately 0.25 and down to -0.25. * FairPFN (Pink): The median is approximately 0, with a very small box. Outliers are clustered around 0. **6. Fair Additive Noise:** * Unfair (Blue): The median is approximately 0.2, with a box extending from approximately 0 to 0.25. Outliers extend up to approximately 0.75 and down to -0.1. * Unaware (Orange): The median is approximately 0.05, with a box extending from approximately 0 to 0.1. Outliers extend up to approximately 0.25 and down to -0.1. * EGR (Purple): The median is approximately 0, with a box extending from approximately -0.05 to 0.05. Outliers extend up to approximately 0.25 and down to -0.25. * FairPFN (Pink): The median is approximately 0, with a very small box. Outliers are clustered around 0. ### Key Observations * The "Unfair" method (blue) generally has a higher median ATE compared to the other methods across most scenarios. * The "FairPFN" method (pink) consistently has a median ATE close to 0 with a very small box, indicating a more concentrated distribution around 0. * The "Direct-Effect" scenario (plot 2) only shows data for the "Unfair" and "EGR" methods. * The average rank of the methods, as indicated in the legend, suggests that FairPFN performs best on average (1.88/4), while Unfair performs worst (3.42/4). ### Interpretation The box plots illustrate the performance of different methods in estimating the Causal Effect (ATE) under various conditions. The "Unfair" method tends to overestimate the ATE, while the "FairPFN" method provides estimates closer to zero. The "Direct-Effect" scenario highlights a specific case where only the "Unfair" and "EGR" methods are applicable or relevant. The average rank values provide a summary of the overall performance of each method across all scenarios, suggesting that "FairPFN" is the most reliable in terms of achieving a lower (better) rank. The spread of the box plots and the presence of outliers indicate the variability in the ATE estimates for each method under different scenarios.

\n ## Box Plots: Causal Effect Analysis under Different Fairness Constraints ### Overview The image presents six box plots, each representing the distribution of causal effects (ATE - Average Treatment Effect) under different fairness scenarios. Each plot compares four different algorithms: FairPFN, EGR, Unaware, and Unfair. The x-axis represents the average rank of each algorithm, and the y-axis represents the causal effect. A horizontal gray dashed line at y=0 serves as a reference point. ### Components/Axes * **Y-axis:** "Causal Effect (ATE)" ranging from -0.5 to 0.75. * **X-axis:** "Avg. Rank (ATE)" with values 1, 2, 3, and 4. * **Titles:** Each subplot is titled with a fairness scenario: "1. Biased", "2. Direct-Effect", "3. Indirect-Effect", "4. Fair Observable", "5. Fair Unobservable", "6. Fair Additive Noise". * **Legend:** Located at the bottom center of the image. * FairPFN: Purple, labeled "1.88/4" * EGR: Green, labeled "2.11/4" * Unaware: Orange, labeled "2.16/4" * Unfair: Blue, labeled "3.42/4" ### Detailed Analysis Each subplot displays box plots for the four algorithms. The box plots show the median, quartiles, and outliers of the causal effect distribution. **1. Biased:** * FairPFN (Purple): Median around 0.1, IQR from approximately 0 to 0.25. Several outliers above 0.5. * EGR (Green): Median around 0.2, IQR from approximately 0 to 0.3. * Unaware (Orange): Median around 0, IQR from approximately -0.1 to 0.15. * Unfair (Blue): Median around 0.25, IQR from approximately 0.1 to 0.4. **2. Direct-Effect:** * FairPFN (Purple): Median around 0.25, IQR from approximately 0.1 to 0.4. * EGR (Green): Median around 0.25, IQR from approximately 0.1 to 0.4. * Unaware (Orange): Median around 0, IQR from approximately -0.1 to 0.1. * Unfair (Blue): Median around 0.25, IQR from approximately 0.1 to 0.4. **3. Indirect-Effect:** * FairPFN (Purple): Median around 0.2, IQR from approximately 0 to 0.3. * EGR (Green): Median around 0.2, IQR from approximately 0 to 0.3. * Unaware (Orange): Median around 0, IQR from approximately -0.1 to 0.1. * Unfair (Blue): Median around 0.2, IQR from approximately 0 to 0.3. **4. Fair Observable:** * FairPFN (Purple): Median around 0.25, IQR from approximately 0.1 to 0.4. * EGR (Green): Median around 0.2, IQR from approximately 0 to 0.3. * Unaware (Orange): Median around 0, IQR from approximately -0.1 to 0.1. * Unfair (Blue): Median around 0.2, IQR from approximately 0 to 0.3. **5. Fair Unobservable:** * FairPFN (Purple): Median around 0.2, IQR from approximately 0 to 0.3. * EGR (Green): Median around 0.2, IQR from approximately 0 to 0.3. * Unaware (Orange): Median around 0, IQR from approximately -0.1 to 0.1. * Unfair (Blue): Median around 0.2, IQR from approximately 0 to 0.3. **6. Fair Additive Noise:** * FairPFN (Purple): Median around 0.2, IQR from approximately 0 to 0.3. * EGR (Green): Median around 0.2, IQR from approximately 0 to 0.3. * Unaware (Orange): Median around 0, IQR from approximately -0.1 to 0.1. * Unfair (Blue): Median around 0.2, IQR from approximately 0 to 0.3. ### Key Observations * The "Unfair" algorithm consistently exhibits a higher median causal effect than the other algorithms across most scenarios. * The "Unaware" algorithm generally has a median causal effect close to zero. * FairPFN and EGR show similar distributions in most scenarios. * The average rank values in the legend indicate that FairPFN performs best (lowest rank) on average, followed by EGR and Unaware, with Unfair performing worst. * Outliers are present in several box plots, particularly for FairPFN, suggesting variability in the causal effect. ### Interpretation The data suggests that the "Unfair" algorithm consistently produces a higher causal effect, potentially indicating a bias in its predictions. The "Unaware" algorithm, which does not consider fairness constraints, tends to have a neutral causal effect. FairPFN and EGR, designed with fairness in mind, achieve comparable performance and generally exhibit lower causal effects than the "Unfair" algorithm. The average rank values confirm that FairPFN is the best-performing algorithm overall, followed by EGR. The presence of outliers suggests that the causal effect can vary significantly depending on the specific data instance. The different fairness scenarios (Biased, Direct-Effect, etc.) highlight the importance of considering different types of fairness constraints when evaluating and comparing algorithms. The consistent performance of FairPFN and EGR across these scenarios suggests their robustness to different fairness challenges.

## Box Plot Series: Causal Effect (ATE) Across Six Fairness Scenarios ### Overview The image displays a 2x3 grid of six box plots, each visualizing the distribution of "Causal Effect (ATE)" for four different methods under distinct experimental conditions. The plots are titled to indicate the scenario being tested. A shared legend at the bottom identifies the four methods and provides a summary performance metric. ### Components/Axes * **Chart Type:** Box and whisker plots with overlaid data points (jitter). * **Y-Axis (All Plots):** Labeled **"Causal Effect (ATE)"**. The scale ranges from -0.5 to 0.75, with major gridlines at intervals of 0.25 (-0.5, -0.25, 0, 0.25, 0.5, 0.75). * **X-Axis (All Plots):** Represents four categorical methods. The categories are not labeled on the axis but are defined by color in the legend. * **Legend (Bottom Center):** Contains the title **"Avg. Rank (ATE)"** and defines the four methods with associated colors and a numerical rank (lower is better): * **Pink:** `FairPFN: 1.88/4` * **Purple:** `EGR: 2.11/4` * **Orange:** `Unaware: 2.16/4` * **Blue:** `Unfair: 3.42/4` * **Subplot Titles (Top of each plot):** 1. **Biased** (Top Left) 2. **Direct-Effect** (Top Center) 3. **Indirect-Effect** (Top Right) 4. **Fair Observable** (Bottom Left) 5. **Fair Unobservable** (Bottom Center) 6. **Fair Additive Noise** (Bottom Right) ### Detailed Analysis **General Structure per Plot:** Each subplot contains four box plots, one for each method (Blue, Orange, Purple, Pink from left to right). The box represents the interquartile range (IQR), the line inside is the median, whiskers extend to 1.5*IQR, and circles represent individual data points/outliers. **Plot-by-Plot Analysis:** 1. **Biased:** * **Unfair (Blue):** Highest median (~0.05), largest IQR (box from ~0 to ~0.2), and widest overall range (whiskers from ~-0.15 to ~0.45). Many high-value outliers up to ~0.75. * **Unaware (Orange):** Median near 0, smaller IQR than Blue, range ~-0.05 to ~0.3. * **EGR (Purple):** Median slightly below 0, IQR similar to Orange, but with notable low-value outliers down to ~-0.5. * **FairPFN (Pink):** Median at 0, very compact IQR, range ~-0.1 to ~0.1. Tightest distribution. 2. **Direct-Effect:** * **Unfair (Blue):** Dominates the plot. Median ~0.15, large IQR (box from ~0.05 to ~0.35), whiskers from ~-0.1 to ~0.65. * **Unaware (Orange), EGR (Purple), FairPFN (Pink):** All are extremely compressed around 0. Their boxes are nearly flat lines, indicating near-zero variance and median. Minor outliers exist within ±0.1. 3. **Indirect-Effect:** * **Unfair (Blue):** Similar pattern to "Biased" plot. Median ~0.05, IQR ~0 to ~0.2, outliers up to ~0.75. * **Unaware (Orange):** Median ~0, IQR ~0 to ~0.1. * **EGR (Purple):** Median ~0, IQR ~0 to ~0.1, with low outliers to ~-0.4. * **FairPFN (Pink):** Very tight distribution around 0. 4. **Fair Observable:** * **Unfair (Blue):** Median ~0.15, IQR ~0.05 to ~0.3. * **Unaware (Orange):** Median ~0, very compact. * **EGR (Purple):** Median ~0, compact but with low outliers to ~-0.4. * **FairPFN (Pink):** Extremely tight around 0. 5. **Fair Unobservable:** * **Unfair (Blue):** Median ~0.2, IQR ~0.05 to ~0.35, whiskers to ~0.7. * **Unaware (Orange):** Median ~0.05, small IQR. * **EGR (Purple):** Median ~0, small IQR, low outliers. * **FairPFN (Pink):** Tight around 0. 6. **Fair Additive Noise:** * **Unfair (Blue):** Median ~0.15, IQR ~0.05 to ~0.3. * **Unaware (Orange):** Median ~0, small IQR. * **EGR (Purple):** Median ~0, small IQR, low outliers. * **FairPFN (Pink):** Tight around 0. ### Key Observations 1. **Consistent Hierarchy:** Across all six scenarios, the **Unfair (Blue)** method consistently shows the highest median causal effect (ATE) and the greatest variance (widest box and whiskers). **FairPFN (Pink)** consistently shows a median at or very near zero with the smallest variance. 2. **Scenario Impact:** The "Direct-Effect" scenario shows the most dramatic suppression of effect for the three fair/unaware methods (Orange, Purple, Pink), compressing them to near-zero variance. The "Biased" and "Indirect-Effect" scenarios show the most pronounced high-value outliers for the Unfair method. 3. **Method Comparison:** The **Unaware (Orange)** and **EGR (Purple)** methods generally perform similarly, with medians near zero. EGR exhibits a recurring pattern of negative outliers (low ATE values) in several plots (Biased, Indirect-Effect, Fair Observable). 4. **Legend Rank Correlation:** The visual performance aligns with the "Avg. Rank" in the legend. FairPFN (rank 1.88) is visually the best (lowest, tightest ATE). Unfair (rank 3.42) is visually the worst (highest, most variable ATE). Unaware and EGR are in the middle and close in rank (2.16 vs. 2.11), reflecting their similar visual performance. ### Interpretation This figure evaluates how different algorithmic approaches (FairPFN, EGR, Unaware) perform in estimating or mitigating **causal effects** (specifically, Average Treatment Effect - ATE) compared to an **Unfair** baseline, across various data-generating scenarios related to fairness. * **What the data suggests:** The "Unfair" method, which likely does not account for fairness constraints, results in substantial and variable estimated causal effects. In contrast, the methods designed for fairness (FairPFN, EGR) or that are simply unaware of sensitive attributes (Unaware) successfully drive the estimated ATE towards zero. This implies these methods are effective at removing or neutralizing the measured causal influence of a treatment, which in a fairness context often corresponds to a sensitive attribute like race or gender. * **How elements relate:** The six scenarios (Biased, Direct/Indirect Effect, Fair Observable/Unobservable/Noise) test the robustness of the methods under different assumptions about how bias or fairness is embedded in the data. The consistent pattern across plots indicates the core finding is robust: fairness-aware methods suppress the measured causal effect. * **Notable patterns/anomalies:** * The extreme compression in the "Direct-Effect" plot suggests that when the causal pathway is direct, the fairness interventions (and even the unaware method) are exceptionally effective at nullifying the measured effect. * The negative outliers for EGR are an anomaly, suggesting that in some runs, this method may over-correct, leading to a negative estimated ATE. * The high-value outliers for the Unfair method in "Biased" and "Indirect-Effect" scenarios indicate that under those data conditions, the lack of fairness constraints can lead to very large estimated causal disparities. **In summary, the visualization provides strong evidence that the FairPFN method (and to a lesser extent EGR and Unaware) consistently and effectively minimizes the estimated average causal effect of a treatment across a variety of fairness-related data scenarios, outperforming an unfair baseline.**

## Box Plot Chart: Causal Effect Analysis Across Model Conditions ### Overview The image presents six box plots arranged in two rows (three per row) comparing causal effect distributions across different model conditions. Each plot visualizes the distribution of Average Treatment Effect (ATE) values under specific scenarios, with color-coded categories representing model fairness and performance metrics. ### Components/Axes - **Y-Axis**: "Causal Effect (ATE)" with range -0.5 to 0.75 - **X-Axis**: Unlabeled categorical axis with six conditions: 1. Biased 2. Direct-Effect 3. Indirect-Effect 4. Fair Observable 5. Fair Unobservable 6. Fair Additive Noise - **Legend** (bottom-center): - Pink: FairPFN: 1.88/4 - Purple: EGR: 2.11/4 - Orange: Unaware: 2.16/4 - Blue: Unfair: 3.42/4 ### Detailed Analysis 1. **Biased Condition** (Top-left): - Blue (Unfair) box dominates with median ~0.2, IQR 0.1-0.3 - Pink (FairPFN) median ~0.05, IQR -0.1 to 0.2 - Orange (Unaware) median ~0.0, IQR -0.1 to 0.1 - Purple (EGR) median ~0.0, IQR -0.1 to 0.1 2. **Direct-Effect Condition** (Top-center): - Blue (Unfair) median ~0.3, IQR 0.15-0.45 - Other categories cluster near 0 with narrower IQRs 3. **Indirect-Effect Condition** (Top-right): - Blue (Unfair) median ~0.2, IQR 0.05-0.35 - Purple (EGR) shows slight positive skew - Orange (Unaware) median ~0.0, IQR -0.05 to 0.05 4. **Fair Observable** (Bottom-left): - Blue (Unfair) median ~0.2, IQR 0.1-0.3 - Pink (FairPFN) median ~0.05, IQR -0.05 to 0.15 5. **Fair Unobservable** (Bottom-center): - Blue (Unfair) median ~0.25, IQR 0.15-0.4 - Purple (EGR) median ~0.05, IQR -0.05 to 0.15 6. **Fair Additive Noise** (Bottom-right): - Blue (Unfair) median ~0.2, IQR 0.1-0.3 - Pink (FairPFN) median ~0.05, IQR -0.05 to 0.15 ### Key Observations 1. **Unfair Condition Dominance**: Blue (Unfair) boxes consistently show highest medians across all conditions, with values ranging from 0.05 to 0.3 2. **Fair Model Variability**: Pink (FairPFN) and Purple (EGR) categories show similar performance patterns, with medians clustered near 0 3. **Statistical Significance**: Orange (Unaware) category demonstrates near-zero effects in most conditions, suggesting baseline performance 4. **Outlier Patterns**: Circular outliers appear in all plots, with highest frequency in "Biased" and "Direct-Effect" conditions 5. **Rank Metrics**: Legend values (e.g., 3.42/4 for Unfair) indicate average ranking positions, with lower values representing better performance ### Interpretation The data reveals systematic performance disparities between model conditions: - **Unfair models** (blue) consistently demonstrate stronger causal effects across all scenarios, suggesting potential bias amplification - **Fair models** (pink/purple) show more balanced performance, with effects clustering near zero - The "Fair Additive Noise" condition mirrors "Fair Observable" patterns, indicating similar robustness mechanisms - The Unfair condition's higher average rank (3.42/4) compared to FairPFN (1.88/4) quantitatively confirms its inferior performance - Outlier distributions suggest potential data quality issues or model instability in extreme cases This analysis highlights critical tradeoffs between model fairness and causal effect strength, with implications for ethical AI development and deployment strategies.