## Box Plot: Performance Comparison of mTPI2 vs mTPI ### Overview The image presents a box plot comparing the performance difference between two methods, mTPI2 and mTPI, across varying target toxic probabilities. The plot is divided into two sections, labeled "Reliability" and "Safety," each displaying box plots for the performance difference at target toxic probabilities of 0.1, 0.15/0.16, 0.2, 0.25, and 0.3/0.33. ### Components/Axes * **Title:** Performance Comparison: mTPI2 - mTPI * **X-axis:** Target toxic probability, with values 0.1, 0.15/0.16, 0.2, 0.25, and 0.3/0.33. * **Y-axis:** Performance Difference, ranging from -0.2 to 0.2. * **Sections:** The plot is divided into two sections, labeled "Reliability" (left) and "Safety" (right). * **Reference Line:** A horizontal dotted red line is present at Performance Difference = 0.0. ### Detailed Analysis **Reliability Section:** * **Target toxic probability 0.1:** The box plot is centered slightly below 0.0. The box extends from approximately -0.05 to 0.01. There are several outliers both above and below the box. * **Target toxic probability 0.15/0.16:** The box plot is centered around 0.0. The box extends from approximately -0.04 to 0.02. There are several outliers both above and below the box. * **Target toxic probability 0.2:** The box plot is centered around 0.0. The box extends from approximately -0.05 to 0.03. There are several outliers both above and below the box. * **Target toxic probability 0.25:** The box plot is centered slightly above 0.0. The box extends from approximately -0.03 to 0.04. There are several outliers both above and below the box. * **Target toxic probability 0.3/0.33:** The box plot is centered around 0.0. The box extends from approximately -0.01 to 0.03. There are several outliers both above and below the box. **Safety Section:** * **Target toxic probability 0.1:** The box plot is centered slightly above 0.0. The box extends from approximately 0.0 to 0.06. There are several outliers both above and below the box. * **Target toxic probability 0.15/0.16:** The box plot is centered above 0.0. The box extends from approximately 0.01 to 0.07. There are several outliers both above and below the box. * **Target toxic probability 0.2:** The box plot is centered above 0.0. The box extends from approximately 0.02 to 0.08. There are several outliers both above and below the box. * **Target toxic probability 0.25:** The box plot is centered around 0.0. The box extends from approximately -0.01 to 0.06. There are several outliers both above and below the box. * **Target toxic probability 0.3/0.33:** The box plot is centered around 0.0. The box extends from approximately -0.01 to 0.01. There are several outliers both above and below the box. ### Key Observations * The performance difference between mTPI2 and mTPI varies with the target toxic probability. * In the "Reliability" section, the median performance difference is generally close to zero for all target toxic probabilities. * In the "Safety" section, the median performance difference tends to be slightly above zero for lower target toxic probabilities (0.1, 0.15/0.16, 0.2) and closer to zero for higher target toxic probabilities (0.25, 0.3/0.33). * The presence of outliers indicates variability in the performance difference for both reliability and safety. * The box plots in the "Safety" section appear to have a higher median performance difference for lower target toxic probabilities compared to the "Reliability" section. ### Interpretation The box plot suggests that mTPI2 and mTPI have varying performance differences depending on the target toxic probability and whether the metric is "Reliability" or "Safety." The "Safety" section shows a tendency for mTPI2 to outperform mTPI at lower target toxic probabilities, while the "Reliability" section shows performance differences centered around zero. The outliers indicate that the performance difference can be highly variable in certain cases. The dotted red line at zero serves as a reference point to easily visualize whether mTPI2 is performing better or worse than mTPI. The data suggests that the choice between mTPI2 and mTPI might depend on the specific target toxic probability and the desired balance between reliability and safety.

## Boxplot: Performance Comparison: mTPI2 - mTPI ### Overview The image presents a series of boxplots comparing the performance difference between mTPI2 and mTPI across different target toxic probability levels. The plot is divided into two sections: "Reliability" on the left and "Safety" on the right. Each section contains boxplots for five different target toxic probability values. ### Components/Axes * **Title:** Performance Comparison: mTPI2 - mTPI * **X-axis:** Target toxic probability. Markers are: 0.1, 0.15/0.16, 0.2, 0.25, 0.3/0.33 * **Y-axis:** Performance Difference. Scale ranges from approximately -0.2 to 0.2. * **Sections:** Reliability (left) and Safety (right). These are indicated by text labels at the top of each section. * **Boxplots:** Each boxplot represents the distribution of performance differences for a specific target toxic probability. ### Detailed Analysis The boxplots show the median, quartiles, and outliers for the performance difference. **Reliability Section:** * **0.1:** The boxplot is centered around approximately 0.01. The interquartile range (IQR) is roughly between -0.02 and 0.04. There are several outliers below -0.15. * **0.15/0.16:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.03. There are a few outliers below -0.15. * **0.2:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.03. There are a few outliers below -0.15. * **0.25:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.04. There are a few outliers below -0.15. * **0.3/0.33:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.04. There are several outliers below -0.15. **Safety Section:** * **0.1:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.03. There are a few outliers below -0.1. * **0.15/0.16:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.03. There are a few outliers below -0.1. * **0.2:** The boxplot is centered around approximately 0.01. The IQR is roughly between -0.03 and 0.04. There are a few outliers below -0.1. * **0.25:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.04. There are a few outliers below -0.1. * **0.3/0.33:** The boxplot is centered around approximately 0.00. The IQR is roughly between -0.03 and 0.04. There are a few outliers below -0.1. ### Key Observations * In the Reliability section, the performance difference tends to be slightly positive for the lowest target toxic probability (0.1), but is generally around zero for the other probabilities. There are more outliers in the Reliability section, particularly at the lower probabilities. * In the Safety section, the performance difference is consistently around zero across all target toxic probabilities. * The spread of the data (as indicated by the IQR and whiskers) is similar across most target toxic probabilities within each section. * Outliers are present in both sections, suggesting some instances where the performance difference is significantly different from the typical range. ### Interpretation The data suggests that mTPI2 and mTPI perform similarly across most target toxic probability levels, as indicated by the performance differences being centered around zero. However, there is some evidence that mTPI2 may perform slightly better than mTPI at lower target toxic probabilities in terms of Reliability. The presence of outliers indicates that there are cases where the performance difference is substantial, potentially due to specific input characteristics or edge cases. The Safety section shows a very consistent performance difference of approximately zero, suggesting that the two models perform equally well in terms of safety across all tested probabilities. The larger number of outliers in the Reliability section suggests that this metric is more sensitive to variations in input data or model behavior. The boxplot visualization effectively communicates the distribution of performance differences and highlights potential areas of concern or improvement.

## Box Plot Comparison: Performance Difference (mTPI2 - mTPI) ### Overview The image is a faceted box plot comparing the performance difference between two methods, mTPI2 and mTPI, across two metrics: **Reliability** and **Safety**. The comparison is made at different levels of a "Target toxic probability." The plot shows the distribution of the performance difference (mTPI2 minus mTPI) for each category. A positive value indicates mTPI2 performed better, while a negative value indicates mTPI performed better. ### Components/Axes * **Main Title:** "Performance Comparison: mTPI2 - mTPI" * **Facet Panels (Top Labels):** * Left Panel: "Reliability" * Right Panel: "Safety" * **Y-Axis:** * **Label:** "Performance Difference" * **Scale:** Linear, ranging from approximately -0.2 to +0.2. * **Key Reference Line:** A horizontal red dashed line at y = 0.0, indicating no performance difference. * **X-Axis:** * **Label:** "Target toxic probability" * **Categories (Tick Labels):** The same set of categories appears under both panels: * `0.1` * `0.15/0.16` * `0.2` * `0.25` * `0.30/0.33` * **Plot Elements:** Standard box-and-whisker plots. Each box represents the interquartile range (IQR, 25th to 75th percentile). The horizontal line within each box is the median. Whiskers extend to the most extreme data points within 1.5 * IQR. Individual points beyond the whiskers are outliers. ### Detailed Analysis #### **Left Panel: Reliability** * **Trend:** The median performance difference for Reliability hovers very close to the zero line across all target toxic probability categories. There is no strong, consistent directional trend (upward or downward) in the medians as the target probability increases. * **Data Points (Approximate Medians & Spread):** * `0.1`: Median ≈ 0.0. Box spans from ~ -0.02 to +0.01. Whiskers extend from ~ -0.05 to +0.05. Several outliers above +0.05, with one near +0.1. * `0.15/0.16`: Median ≈ 0.0. Box spans from ~ -0.04 to +0.01. Whiskers extend from ~ -0.12 to +0.08. Notable outliers below -0.15. * `0.2`: Median ≈ 0.0. Box spans from ~ -0.03 to +0.01. Whiskers extend from ~ -0.05 to +0.05. **This category has the most extreme outliers**, with points reaching down to ~ -0.2 and up to +0.2. * `0.25`: Median ≈ 0.0. Box spans from ~ -0.01 to +0.02. Whiskers extend from ~ -0.05 to +0.05. Outliers present both above and below. * `0.30/0.33`: Median ≈ 0.0. Box spans from ~ -0.01 to +0.02. Whiskers extend from ~ -0.05 to +0.05. Outliers present, with a cluster below -0.1. #### **Right Panel: Safety** * **Trend:** The median performance difference for Safety is consistently **positive** for the first four categories (`0.1` to `0.25`), indicating mTPI2 generally outperforms mTPI on this metric. There is a clear **downward trend** in the median difference as the target toxic probability increases, approaching zero at the highest probability (`0.30/0.33`). * **Data Points (Approximate Medians & Spread):** * `0.1`: Median ≈ +0.02. Box spans from ~ 0.0 to +0.04. Upper whisker extends to ~ +0.09. * `0.15/0.16`: Median ≈ +0.02. Box spans from ~ 0.0 to +0.06. Upper whisker extends to ~ +0.13. * `0.2`: Median ≈ +0.025 (highest median in the plot). Box spans from ~ 0.0 to +0.07. Upper whisker extends to ~ +0.12. * `0.25`: Median ≈ +0.01. Box spans from ~ 0.0 to +0.03. Upper whisker extends to ~ +0.05. Outliers present above +0.1. * `0.30/0.33`: Median ≈ 0.0. Box is very compressed around zero. Whiskers are short. A dense vertical line of outliers extends from ~ -0.15 to +0.13, indicating high variance but no consistent advantage for either method. ### Key Observations 1. **Metric-Dependent Performance:** The performance difference is highly dependent on the metric. mTPI2 shows a clear, though diminishing, advantage in **Safety** for lower target toxic probabilities, but shows no consistent advantage or disadvantage in **Reliability**. 2. **Safety Advantage Erodes with Higher Toxicity:** The positive median difference in the Safety panel decreases as the "Target toxic probability" increases, vanishing at the `0.30/0.33` level. 3. **High Variance at Specific Points:** The `0.2` category in the Reliability panel shows extreme variance, with outliers spanning almost the entire y-axis range. The `0.30/0.33` category in the Safety panel also shows very high variance (many outliers) despite a median near zero. 4. **Consistent Zero-Line for Reliability:** The Reliability medians are remarkably stable at zero across all conditions, suggesting the two methods are statistically indistinguishable on this metric regardless of the target toxicity. ### Interpretation This analysis suggests a **trade-off or context-dependent superiority** between the mTPI2 and mTPI methods. * **For Safety-critical applications** where the target toxic probability is low to moderate (0.1 to 0.25), **mTPI2 is the preferable method**, as it demonstrates a consistent, positive performance difference. This could mean it better avoids toxic outputs or adheres to safety constraints under those conditions. * **For Reliability,** there is **no evidence to prefer one method over the other** based on this data. Their performance distributions are centered on zero difference. * **The convergence at high target toxicity (`0.30/0.33`)** is notable. It implies that when the system is explicitly tuned to accept a higher probability of toxic outcomes, the algorithmic differences between mTPI2 and mTPI become negligible for both Safety and Reliability. The high variance here suggests outcomes become more unpredictable for both methods. * **The extreme outliers at `0.2` for Reliability** warrant investigation. This specific toxicity target may represent a "knife-edge" condition where the methods' behaviors diverge dramatically in rare cases, leading to both significant failures (large negative differences) and significant successes (large positive differences). **In summary:** Choose mTPI2 over mTPI for safer operation at lower toxicity thresholds. For pure reliability or at high toxicity thresholds, the methods are functionally equivalent. The choice of target toxic probability is a critical parameter that influences not just the absolute performance, but the *relative* performance of these two algorithms.

## Box Plot: Performance Comparison: mTPI2 - mTPI ### Overview The image presents a comparative box plot analysis of performance differences between two models (mTPI2 and mTPI) across varying target toxic probabilities. The data is split into two categories: **Reliability** (left) and **Safety** (right). Performance differences are measured on a scale from -0.2 to 0.2, with a red dashed line at 0.0 indicating no difference. The x-axis categorizes data by target toxic probability ranges (0.1, 0.15/0.16, 0.2, 0.25, 0.3/0.33). --- ### Components/Axes - **X-Axis**: Target toxic probability (categorical bins: 0.1, 0.15/0.16, 0.2, 0.25, 0.3/0.33). - **Y-Axis**: Performance Difference (continuous scale from -0.2 to 0.2). - **Legend**: - **Reliability**: Gray (left section). - **Safety**: Black (right section). - **Red Dashed Line**: Baseline at 0.0 performance difference. --- ### Detailed Analysis #### Reliability (Left Section) - **0.1**: Median ~-0.05, interquartile range (IQR) spans -0.05 to -0.01. Outliers at ~-0.15 and ~0.05. - **0.15/0.16**: Median ~-0.03, IQR spans -0.08 to -0.01. Outliers at ~-0.18 and ~0.02. - **0.2**: Median ~-0.02, IQR spans -0.06 to 0.01. Outliers at ~-0.12 and ~0.08. - **0.25**: Median ~-0.01, IQR spans -0.04 to 0.02. Outliers at ~-0.15 and ~0.1. - **0.3/0.33**: Median ~-0.01, IQR spans -0.03 to 0.01. Outliers at ~-0.18 and ~0.05. #### Safety (Right Section) - **0.1**: Median ~0.03, IQR spans 0.01 to 0.05. Outliers at ~-0.05 and ~0.12. - **0.15/0.16**: Median ~0.05, IQR spans 0.03 to 0.08. Outliers at ~-0.02 and ~0.15. - **0.2**: Median ~0.06, IQR spans 0.04 to 0.1. Outliers at ~-0.03 and ~0.18. - **0.25**: Median ~0.04, IQR spans 0.02 to 0.07. Outliers at ~-0.05 and ~0.12. - **0.3/0.33**: Median ~0.02, IQR spans 0.01 to 0.04. Outliers at ~-0.08 and ~0.15. --- ### Key Observations 1. **Reliability Trends**: - Performance differences are predominantly negative (below 0.0) across all target toxic probabilities. - Median values cluster near -0.01 to -0.05, with increasing variability (wider IQRs) at higher toxicity targets (e.g., 0.25 and 0.3/0.33). - Outliers suggest occasional positive performance differences, but these are rare. 2. **Safety Trends**: - Performance differences are predominantly positive (above 0.0) across all ranges. - Median values increase slightly with higher toxicity targets (e.g., 0.2 has the highest median at ~0.06). - Safety shows greater variability at higher toxicity targets (e.g., 0.25 and 0.3/0.33 have wider IQRs). 3. **Outliers**: - Reliability outliers below -0.15 and above 0.05 are sparse but notable. - Safety outliers below -0.05 and above 0.15 indicate rare deviations from the trend. --- ### Interpretation - **Model Behavior**: - mTPI2 underperforms mTPI in **Reliability** across most scenarios, with performance differences worsening as target toxic probability increases. This suggests mTPI2 may struggle with consistency in high-stakes or ambiguous contexts. - In **Safety**, mTPI2 generally outperforms mTPI, with performance differences peaking at 0.2. However, the widening IQRs at higher toxicity targets imply increased unpredictability in extreme cases. - **Design Implications**: - The divergence in trends between Reliability and Safety highlights a trade-off: optimizing for safety may come at the cost of reliability, and vice versa. - Outliers in both categories warrant investigation to determine if they represent edge cases or systemic issues in specific toxicity ranges. - **Limitations**: - The box plots do not specify the direction of performance differences (e.g., whether mTPI2 is faster, more accurate, or less resource-intensive than mTPI). - The absence of statistical significance markers (e.g., p-values) makes it unclear whether differences are meaningful or due to random variation. --- ### Spatial Grounding - **Legend**: Top-center, clearly labeling Reliability (gray) and Safety (black). - **Red Dashed Line**: Horizontally centered at 0.0, dividing positive and negative performance differences. - **Box Plots**: Aligned vertically under each x-axis category, with Reliability on the left and Safety on the right. --- ### Content Details - **Reliability**: - Median values: -0.05 (0.1), -0.03 (0.15/0.16), -0.02 (0.2), -0.01 (0.25), -0.01 (0.3/0.33). - Outliers: -0.15, -0.18, -0.12, -0.15, -0.18 (negative); 0.05, 0.02, 0.08, 0.1, 0.05 (positive). - **Safety**: - Median values: 0.03 (0.1), 0.05 (0.15/0.16), 0.06 (0.2), 0.04 (0.25), 0.02 (0.3/0.33). - Outliers: -0.05, -0.02, -0.03, -0.05, -0.08 (negative); 0.12, 0.15, 0.18, 0.12, 0.15 (positive). --- ### Final Notes The data underscores a critical tension between Reliability and Safety in model performance. While mTPI2 shows promise in Safety, its Reliability lags behind mTPI, particularly in high-toxicity scenarios. Further analysis is needed to quantify the trade-offs and explore mitigation strategies.