## Bar Chart: ROAR Performance Across Attack Scenarios ### Overview The image presents a series of four bar charts, arranged in a 2x2 grid, comparing the performance of three different ROAR metrics (ROAR_lp, ROAR_pm, and ROAR_co) across three attack scenarios (Chrome, CAPEC-22, and T1550.001) under different attack types: Backdoor-Vulnerability, Backdoor-Mitigation, Targeted-Vulnerability, and Targeted-Mitigation. The charts use two y-axes: MRR@K (Mean Reciprocal Rank at K) on the top and HIT@5K (Hit Rate at 5K) on the bottom. ### Components/Axes * **X-axis:** Represents the attack scenarios: Chrome, CAPEC-22, and T1550.001. * **Y-axis (Left):** MRR@K, ranging from 0.00 to 1.00. * **Y-axis (Right):** HIT@5K, ranging from -1.00 to 1.00. * **Legend:** Located at the top-right of each chart, indicating the ROAR metrics: * ROAR_lp (Light Green) * ROAR_pm (Medium Green) * ROAR_co (Dark Green) * **Chart Titles:** Located below each chart, identifying the attack type: * (a) Backdoor-Vulnerability * (b) Backdoor-Mitigation * (c) Targeted-Vulnerability * (d) Targeted-Mitigation ### Detailed Analysis **Chart (a) - Backdoor-Vulnerability:** * **Chrome:** * ROAR_lp: Approximately 0.57 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K * ROAR_pm: Approximately 0.45 (±0.02) MRR, approximately 0.30 (±0.02) HIT@5K * ROAR_co: Approximately 0.33 (±0.02) MRR, approximately 0.41 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.24 (±0.02) MRR, approximately 0.52 (±0.02) HIT@5K * ROAR_pm: Approximately 0.18 (±0.02) MRR, approximately 0.66 (±0.02) HIT@5K * ROAR_co: Approximately 0.16 (±0.02) MRR, approximately 0.68 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.37 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_pm: Approximately 0.33 (±0.02) MRR, approximately 0.53 (±0.02) HIT@5K * ROAR_co: Approximately 0.29 (±0.02) MRR, approximately 0.66 (±0.02) HIT@5K **Chart (b) - Backdoor-Mitigation:** * **Chrome:** * ROAR_lp: Approximately 0.64 (±0.02) MRR, approximately 0.32 (±0.02) HIT@5K * ROAR_pm: Approximately 0.68 (±0.02) MRR, approximately 0.33 (±0.02) HIT@5K * ROAR_co: Approximately 0.45 (±0.02) MRR, approximately 0.52 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.32 (±0.02) MRR, approximately 0.58 (±0.02) HIT@5K * ROAR_pm: Approximately 0.21 (±0.02) MRR, approximately 0.66 (±0.02) HIT@5K * ROAR_co: Approximately 0.41 (±0.02) MRR, approximately 0.46 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.38 (±0.02) MRR, approximately 0.40 (±0.02) HIT@5K * ROAR_pm: Approximately 0.34 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_co: Approximately 0.26 (±0.02) MRR, approximately 0.52 (±0.02) HIT@5K **Chart (c) - Targeted-Vulnerability:** * **Chrome:** * ROAR_lp: Approximately 0.86 (±0.02) MRR, approximately 0.70 (±0.02) HIT@5K * ROAR_pm: Approximately 0.74 (±0.02) MRR, approximately 0.51 (±0.02) HIT@5K * ROAR_co: Approximately 0.45 (±0.02) MRR, approximately 0.56 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.52 (±0.02) MRR, approximately 0.13 (±0.02) HIT@5K * ROAR_pm: Approximately 0.45 (±0.02) MRR, approximately 0.07 (±0.02) HIT@5K * ROAR_co: Approximately 0.21 (±0.02) MRR, approximately 0.14 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.50 (±0.02) MRR, approximately 0.30 (±0.02) HIT@5K * ROAR_pm: Approximately 0.43 (±0.02) MRR, approximately 0.24 (±0.02) HIT@5K * ROAR_co: Approximately 0.24 (±0.02) MRR, approximately 0.10 (±0.02) HIT@5K **Chart (d) - Targeted-Mitigation:** * **Chrome:** * ROAR_lp: Approximately 0.66 (±0.02) MRR, approximately 0.49 (±0.02) HIT@5K * ROAR_pm: Approximately 0.62 (±0.02) MRR, approximately 0.41 (±0.02) HIT@5K * ROAR_co: Approximately 0.49 (±0.02) MRR, approximately 0.30 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.52 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_pm: Approximately 0.49 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_co: Approximately 0.41 (±0.02) MRR, approximately 0.38 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.11 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K * ROAR_pm: Approximately 0.06 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K * ROAR_co: Approximately 0.06 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K ### Key Observations * ROAR_lp generally performs best in terms of MRR@K across most scenarios. * ROAR_co often shows the highest HIT@5K values in the Backdoor-Vulnerability and Backdoor-Mitigation scenarios. * Performance varies significantly depending on the attack scenario. Chrome consistently shows higher MRR@K values than CAPEC-22 and T1550.001. * Targeted-Mitigation consistently shows the lowest performance across all ROAR metrics, particularly for T1550.001. ### Interpretation The charts demonstrate the effectiveness of different ROAR metrics in detecting attacks across various scenarios. The varying performance suggests that the optimal ROAR metric depends on the specific attack type and target. The higher MRR@K values for ROAR_lp indicate its strength in ranking relevant results higher, while the higher HIT@5K values for ROAR_co in certain scenarios suggest its ability to identify a broader range of relevant results. The consistently poor performance in the Targeted-Mitigation scenario, especially for T1550.001, highlights a potential weakness in detecting attacks when mitigation strategies are in place, or a limitation of the ROAR metrics in this specific context. The differences in performance across attack scenarios (Chrome, CAPEC-22, T1550.001) suggest that the characteristics of each attack influence the effectiveness of the ROAR metrics. Further investigation is needed to understand the underlying reasons for these differences and to develop more robust detection strategies.