## Bar Chart: ROAR Performance Comparison ### Overview The image presents four bar charts comparing the performance of three different methods (Chrome, CAPEC-22, and T1550.001) under different scenarios: Backdoor-Vulnerability, Backdoor-Mitigation, Targeted-Vulnerability, and Targeted-Mitigation. The performance is measured using two metrics: MRR (Mean Reciprocal Rank) and HIT@5. Three different ROAR variants (ROARkp, ROARqm, and ROARco) are compared. ### Components/Axes * **Chart Titles:** * (a) Backdoor-Vulnerability * (b) Backdoor-Mitigation * (c) Targeted-Vulnerability * (d) Targeted-Mitigation * **X-axis:** Categorical, representing the methods: Chrome, CAPEC-22, T1550.001. * **Y-axis (Left):** * Top: MRR(↑), ranging from 0.00 to 1.00. The arrow indicates higher is better. * Bottom: HIT@5(↑), ranging from 1.00 to -1.00. The arrow indicates higher is better. * **Y-axis (Right):** * Top: MRR(↓), ranging from 1.00 to 0.00. The arrow indicates lower is better. * Bottom: HIT@5(↓), ranging from 1.00 to -1.00. The arrow indicates lower is better. * **Legend:** Located at the top of the image. * ROARkp: Light Green, with diagonal lines. * ROARqm: Dark Green, with diagonal lines. * ROARco: Dark Green, with dotted pattern. ### Detailed Analysis #### (a) Backdoor-Vulnerability * **MRR(↑):** * Chrome: ROARkp = 0.35, ROARqm = 0.51, ROARco = 0.57 * CAPEC-22: ROARkp = 0.24, ROARqm = 0.33, ROARco = 0.45 * T1550.001: ROARkp = 0.18, ROARqm = 0.28, ROARco = 0.30 * **HIT@5(↑):** * Chrome: ROARkp = 0.50, ROARqm = 0.58, ROARco = 0.66 * CAPEC-22: ROARkp = 0.31, ROARqm = 0.40, ROARco = 0.52 * T1550.001: ROARkp = 0.16, ROARqm = 0.42, ROARco = 0.44 #### (b) Backdoor-Mitigation * **MRR(↑):** * Chrome: ROARkp = 0.37, ROARqm = 0.64, ROARco = 0.68 * CAPEC-22: ROARkp = 0.32, ROARqm = 0.45, ROARco = 0.53 * T1550.001: ROARkp = 0.21, ROARqm = 0.38, ROARco = 0.41 * **HIT@5(↑):** * Chrome: ROARkp = 0.55, ROARqm = 0.66, ROARco = 0.68 * CAPEC-22: ROARkp = 0.33, ROARqm = 0.52, ROARco = 0.58 * T1550.001: ROARkp = 0.22, ROARqm = 0.44, ROARco = 0.46 #### (c) Targeted-Vulnerability * **MRR(↑):** * Chrome: ROARkp = 0.33, ROARqm = 0.74, ROARco = 0.86 * CAPEC-22: ROARkp = 0.21, ROARqm = 0.45, ROARco = 0.52 * T1550.001: ROARkp = 0.14, ROARqm = 0.50, ROARco = 0.59 * **HIT@5(↑):** * Chrome: ROARkp = 0.26, ROARqm = 0.76, ROARco = 0.92 * CAPEC-22: ROARkp = 0.13, ROARqm = 0.51, ROARco = 0.56 * T1550.001: ROARkp = 0.07, ROARqm = 0.53, ROARco = 0.58 #### (d) Targeted-Mitigation * **MRR(↑):** * Chrome: ROARkp = 0.43, ROARqm = 0.62, ROARco = 0.66 * CAPEC-22: ROARkp = 0.24, ROARqm = 0.45, ROARco = 0.44 * T1550.001: ROARkp = 0.10, ROARqm = 0.49, ROARco = 0.41 * **HIT@5(↑):** * Chrome: ROARkp = 0.30, ROARqm = 0.80, ROARco = 0.85 * CAPEC-22: ROARkp = 0.11, ROARqm = 0.52, ROARco = 0.49 * T1550.001: ROARkp = 0.06, ROARqm = 0.44, ROARco = 0.39 ### Key Observations * ROARco consistently outperforms ROARqm and ROARkp across all scenarios and methods. * ROARqm generally outperforms ROARkp. * Chrome generally shows the highest performance, especially in Targeted-Vulnerability and Targeted-Mitigation scenarios. * T1550.001 often has the lowest performance. * The Targeted-Vulnerability scenario shows the largest performance differences between the ROAR variants, particularly for Chrome. ### Interpretation The data suggests that ROARco is the most effective variant for both vulnerability detection and mitigation, followed by ROARqm. Chrome appears to be the most robust method overall, while T1550.001 may require further optimization. The significant performance differences in the Targeted-Vulnerability scenario highlight the importance of choosing the right ROAR variant for specific tasks. The consistent trend of ROARco outperforming the other variants indicates that the features it utilizes are more relevant for the tasks being evaluated.

## Bar Chart: ROAR Performance Across Attack Scenarios ### Overview The image presents a series of four bar charts, arranged in a 2x2 grid, comparing the performance of three different ROAR metrics (ROAR_lp, ROAR_pm, and ROAR_co) across three attack scenarios (Chrome, CAPEC-22, and T1550.001) under different attack types: Backdoor-Vulnerability, Backdoor-Mitigation, Targeted-Vulnerability, and Targeted-Mitigation. The charts use two y-axes: MRR@K (Mean Reciprocal Rank at K) on the top and HIT@5K (Hit Rate at 5K) on the bottom. ### Components/Axes * **X-axis:** Represents the attack scenarios: Chrome, CAPEC-22, and T1550.001. * **Y-axis (Left):** MRR@K, ranging from 0.00 to 1.00. * **Y-axis (Right):** HIT@5K, ranging from -1.00 to 1.00. * **Legend:** Located at the top-right of each chart, indicating the ROAR metrics: * ROAR_lp (Light Green) * ROAR_pm (Medium Green) * ROAR_co (Dark Green) * **Chart Titles:** Located below each chart, identifying the attack type: * (a) Backdoor-Vulnerability * (b) Backdoor-Mitigation * (c) Targeted-Vulnerability * (d) Targeted-Mitigation ### Detailed Analysis **Chart (a) - Backdoor-Vulnerability:** * **Chrome:** * ROAR_lp: Approximately 0.57 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K * ROAR_pm: Approximately 0.45 (±0.02) MRR, approximately 0.30 (±0.02) HIT@5K * ROAR_co: Approximately 0.33 (±0.02) MRR, approximately 0.41 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.24 (±0.02) MRR, approximately 0.52 (±0.02) HIT@5K * ROAR_pm: Approximately 0.18 (±0.02) MRR, approximately 0.66 (±0.02) HIT@5K * ROAR_co: Approximately 0.16 (±0.02) MRR, approximately 0.68 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.37 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_pm: Approximately 0.33 (±0.02) MRR, approximately 0.53 (±0.02) HIT@5K * ROAR_co: Approximately 0.29 (±0.02) MRR, approximately 0.66 (±0.02) HIT@5K **Chart (b) - Backdoor-Mitigation:** * **Chrome:** * ROAR_lp: Approximately 0.64 (±0.02) MRR, approximately 0.32 (±0.02) HIT@5K * ROAR_pm: Approximately 0.68 (±0.02) MRR, approximately 0.33 (±0.02) HIT@5K * ROAR_co: Approximately 0.45 (±0.02) MRR, approximately 0.52 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.32 (±0.02) MRR, approximately 0.58 (±0.02) HIT@5K * ROAR_pm: Approximately 0.21 (±0.02) MRR, approximately 0.66 (±0.02) HIT@5K * ROAR_co: Approximately 0.41 (±0.02) MRR, approximately 0.46 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.38 (±0.02) MRR, approximately 0.40 (±0.02) HIT@5K * ROAR_pm: Approximately 0.34 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_co: Approximately 0.26 (±0.02) MRR, approximately 0.52 (±0.02) HIT@5K **Chart (c) - Targeted-Vulnerability:** * **Chrome:** * ROAR_lp: Approximately 0.86 (±0.02) MRR, approximately 0.70 (±0.02) HIT@5K * ROAR_pm: Approximately 0.74 (±0.02) MRR, approximately 0.51 (±0.02) HIT@5K * ROAR_co: Approximately 0.45 (±0.02) MRR, approximately 0.56 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.52 (±0.02) MRR, approximately 0.13 (±0.02) HIT@5K * ROAR_pm: Approximately 0.45 (±0.02) MRR, approximately 0.07 (±0.02) HIT@5K * ROAR_co: Approximately 0.21 (±0.02) MRR, approximately 0.14 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.50 (±0.02) MRR, approximately 0.30 (±0.02) HIT@5K * ROAR_pm: Approximately 0.43 (±0.02) MRR, approximately 0.24 (±0.02) HIT@5K * ROAR_co: Approximately 0.24 (±0.02) MRR, approximately 0.10 (±0.02) HIT@5K **Chart (d) - Targeted-Mitigation:** * **Chrome:** * ROAR_lp: Approximately 0.66 (±0.02) MRR, approximately 0.49 (±0.02) HIT@5K * ROAR_pm: Approximately 0.62 (±0.02) MRR, approximately 0.41 (±0.02) HIT@5K * ROAR_co: Approximately 0.49 (±0.02) MRR, approximately 0.30 (±0.02) HIT@5K * **CAPEC-22:** * ROAR_lp: Approximately 0.52 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_pm: Approximately 0.49 (±0.02) MRR, approximately 0.44 (±0.02) HIT@5K * ROAR_co: Approximately 0.41 (±0.02) MRR, approximately 0.38 (±0.02) HIT@5K * **T1550.001:** * ROAR_lp: Approximately 0.11 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K * ROAR_pm: Approximately 0.06 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K * ROAR_co: Approximately 0.06 (±0.02) MRR, approximately 0.06 (±0.02) HIT@5K ### Key Observations * ROAR_lp generally performs best in terms of MRR@K across most scenarios. * ROAR_co often shows the highest HIT@5K values in the Backdoor-Vulnerability and Backdoor-Mitigation scenarios. * Performance varies significantly depending on the attack scenario. Chrome consistently shows higher MRR@K values than CAPEC-22 and T1550.001. * Targeted-Mitigation consistently shows the lowest performance across all ROAR metrics, particularly for T1550.001. ### Interpretation The charts demonstrate the effectiveness of different ROAR metrics in detecting attacks across various scenarios. The varying performance suggests that the optimal ROAR metric depends on the specific attack type and target. The higher MRR@K values for ROAR_lp indicate its strength in ranking relevant results higher, while the higher HIT@5K values for ROAR_co in certain scenarios suggest its ability to identify a broader range of relevant results. The consistently poor performance in the Targeted-Mitigation scenario, especially for T1550.001, highlights a potential weakness in detecting attacks when mitigation strategies are in place, or a limitation of the ROAR metrics in this specific context. The differences in performance across attack scenarios (Chrome, CAPEC-22, T1550.001) suggest that the characteristics of each attack influence the effectiveness of the ROAR metrics. Further investigation is needed to understand the underlying reasons for these differences and to develop more robust detection strategies.

## Grouped Bar Chart: MRR and HIT@5 for Backdoor/Targeted Vulnerability/Mitigation ### Overview The image is a grouped bar chart with two subplots (top: **MRR** (Mean Reciprocal Rank), bottom: **HIT@5** (Hit Rate at 5)) across four scenarios: - (a) Backdoor-Vulnerability - (b) Backdoor-Mitigation - (c) Targeted-Vulnerability - (d) Targeted-Mitigation Each scenario compares three methods (ROARₖₚ, ROARₙₘ, ROARₙₒ) across three categories: *Chrome*, *CAPEC-22*, and *T1550.001*. ### Components/Axes - **Y-Axes**: - Top (MRR): Range `0.00–1.00` (higher values = better MRR). - Bottom (HIT@5): Range `0.00–1.00` (inverted: lower on the axis = higher HIT@5). - **X-Axis Categories**: *Chrome*, *CAPEC-22*, *T1550.001* (repeated for each scenario). - **Legend**: - ROARₖₚ: Light green, diagonal stripes. - ROARₙₘ: Medium green, solid. - ROARₙₒ: Dark green, dotted. - **Scenarios**: Labeled below each subplot (a–d). ### Detailed Analysis (Subplot by Subplot) #### (a) Backdoor-Vulnerability - **MRR (Top)**: - *Chrome*: ROARₖₚ=0.35, ROARₙₘ=0.51, ROARₙₒ=0.57 (increasing trend: kp < qm < co). - *CAPEC-22*: ROARₖₚ=0.24, ROARₙₘ=0.33, ROARₙₒ=0.45 (increasing trend). - *T1550.001*: ROARₖₚ=0.18, ROARₙₘ=0.28, ROARₙₒ=0.30 (increasing trend). - **HIT@5 (Bottom)**: - *Chrome*: ROARₖₚ=0.50, ROARₙₘ=0.58, ROARₙₒ=0.66 (increasing trend: kp < qm < co). - *CAPEC-22*: ROARₖₚ=0.31, ROARₙₘ=0.40, ROARₙₒ=0.52 (increasing trend). - *T1550.001*: ROARₖₚ=0.16, ROARₙₘ=0.42, ROARₙₒ=0.44 (increasing trend). #### (b) Backdoor-Mitigation - **MRR (Top)**: - *Chrome*: ROARₖₚ=0.37, ROARₙₘ=0.64, ROARₙₒ=0.68 (increasing trend). - *CAPEC-22*: ROARₖₚ=0.32, ROARₙₘ=0.45, ROARₙₒ=0.53 (increasing trend). - *T1550.001*: ROARₖₚ=0.21, ROARₙₘ=0.38, ROARₙₒ=0.41 (increasing trend). - **HIT@5 (Bottom)**: - *Chrome*: ROARₖₚ=0.55, ROARₙₘ=0.66, ROARₙₒ=0.68 (increasing trend). - *CAPEC-22*: ROARₖₚ=0.33, ROARₙₘ=0.52, ROARₙₒ=0.58 (increasing trend). - *T1550.001*: ROARₖₚ=0.22, ROARₙₘ=0.44, ROARₙₒ=0.46 (increasing trend). #### (c) Targeted-Vulnerability - **MRR (Top)**: - *Chrome*: ROARₖₚ=0.33, ROARₙₘ=0.74, ROARₙₒ=0.86 (large jump for qm/co). - *CAPEC-22*: ROARₖₚ=0.21, ROARₙₘ=0.45, ROARₙₒ=0.52 (increasing trend). - *T1550.001*: ROARₖₚ=0.14, ROARₙₘ=0.50, ROARₙₒ=0.59 (large jump for qm/co). - **HIT@5 (Bottom)**: - *Chrome*: ROARₖₚ=0.26, ROARₙₘ=0.76, ROARₙₒ=0.92 (large jump for qm/co). - *CAPEC-22*: ROARₖₚ=0.13, ROARₙₘ=0.51, ROARₙₒ=0.56 (increasing trend). - *T1550.001*: ROARₖₚ=0.07, ROARₙₘ=0.53, ROARₙₒ=0.58 (large jump for qm/co). #### (d) Targeted-Mitigation - **MRR (Top)**: - *Chrome*: ROARₖₚ=0.43, ROARₙₘ=0.62, ROARₙₒ=0.66 (increasing trend). - *CAPEC-22*: ROARₖₚ=0.24, ROARₙₘ=0.45, ROARₙₒ=0.44 (slight decrease for co). - *T1550.001*: ROARₖₚ=0.10, ROARₙₘ=0.49, ROARₙₒ=0.41 (decrease for co). - **HIT@5 (Bottom)**: - *Chrome*: ROARₖₚ=0.30, ROARₙₘ=0.80, ROARₙₒ=0.85 (increasing trend). - *CAPEC-22*: ROARₖₚ=0.11, ROARₙₘ=0.52, ROARₙₒ=0.49 (slight decrease for co). - *T1550.001*: ROARₖₚ=0.06, ROARₙₘ=0.44, ROARₙₒ=0.39 (decrease for co). ### Key Observations - **Method Hierarchy**: ROARₙₒ consistently outperforms ROARₙₘ, which outperforms ROARₖₚ (increasing trend across methods). - **Scenario Sensitivity**: Targeted scenarios (c, d) show larger performance gaps between methods (e.g., *Chrome* in (c) MRR: 0.33 vs 0.86) than Backdoor scenarios (a, b). - **Category Differences**: *Chrome* often has higher MRR/HIT@5 than *CAPEC-22* and *T1550.001*, especially in Targeted scenarios. - **Anomalies**: In (d) Targeted-Mitigation, *CAPEC-22* and *T1550.001* show a slight decrease in ROARₙₒ performance (deviating from the usual increasing trend). ### Interpretation The chart evaluates three methods for vulnerability detection/mitigation (Backdoor vs Targeted) across three categories. ROARₙₒ’s consistent superiority suggests it is more effective for these tasks. Targeted scenarios amplify performance differences, indicating the methods are more sensitive to targeted attacks. *Chrome*’s higher performance may reflect its relevance or complexity in the dataset. The anomalies in (d) suggest ROARₙₒ’s effectiveness varies by category in targeted mitigation, warranting further investigation. (Note: All values are approximate, based on visual extraction from the chart.)

## Bar Chart: Comparative Analysis of ROAR Methods Across Vulnerability/Mitigation Scenarios ### Overview The image presents four grouped bar charts comparing three methods (ROAR_kp, ROAR_qm, ROAR_co) across four categories: Backdoor-Vulnerability, Backdoor-Mitigation, Targeted-Vulnerability, and Targeted-Mitigation. Each category contains three subcategories (Chrome, CAPEC-22, T1550.001). The y-axis measures two metrics: MRR (↑) and HIT@5 (↓), with values ranging from 0.00 to 1.00. ### Components/Axes - **X-axis**: Categories (a-d) with subcategories (Chrome, CAPEC-22, T1550.001) - **Y-axis**: - MRR (↑): Top scale (0.00–1.00) - HIT@5 (↓): Bottom scale (0.00–1.00) - **Legend**: - Top-right position - ROAR_kp: Diagonal stripes (blue) - ROAR_qm: Crosshatch (green) - ROAR_co: Dots (dark green) - **Color Patterns**: - Blue (ROAR_kp) and green (ROAR_qm/ROAR_co) with distinct hatching ### Detailed Analysis #### (a) Backdoor-Vulnerability - **Chrome**: - ROAR_kp: 0.51 (MRR), 0.58 (HIT@5) - ROAR_qm: 0.35 (MRR), 0.50 (HIT@5) - ROAR_co: 0.57 (MRR), 0.66 (HIT@5) - **CAPEC-22**: - ROAR_kp: 0.33 (MRR), 0.40 (HIT@5) - ROAR_qm: 0.24 (MRR), 0.31 (HIT@5) - ROAR_co: 0.45 (MRR), 0.52 (HIT@5) - **T1550.001**: - ROAR_kp: 0.28 (MRR), 0.42 (HIT@5) - ROAR_qm: 0.18 (MRR), 0.16 (HIT@5) - ROAR_co: 0.30 (MRR), 0.44 (HIT@5) #### (b) Backdoor-Mitigation - **Chrome**: - ROAR_kp: 0.64 (MRR), 0.68 (HIT@5) - ROAR_qm: 0.37 (MRR), 0.55 (HIT@5) - ROAR_co: 0.68 (MRR), 0.68 (HIT@5) - **CAPEC-22**: - ROAR_kp: 0.45 (MRR), 0.52 (HIT@5) - ROAR_qm: 0.32 (MRR), 0.33 (HIT@5) - ROAR_co: 0.53 (MRR), 0.58 (HIT@5) - **T1550.001**: - ROAR_kp: 0.44 (MRR), 0.46 (HIT@5) - ROAR_qm: 0.21 (MRR), 0.22 (HIT@5) - ROAR_co: 0.41 (MRR), 0.46 (HIT@5) #### (c) Targeted-Vulnerability - **Chrome**: - ROAR_kp: 0.74 (MRR), 0.76 (HIT@5) - ROAR_qm: 0.33 (MRR), 0.26 (HIT@5) - ROAR_co: 0.86 (MRR), 0.92 (HIT@5) - **CAPEC-22**: - ROAR_kp: 0.45 (MRR), 0.56 (HIT@5) - ROAR_qm: 0.21 (MRR), 0.13 (HIT@5) - ROAR_co: 0.59 (MRR), 0.58 (HIT@5) - **T1550.001**: - ROAR_kp: 0.50 (MRR), 0.53 (HIT@5) - ROAR_qm: 0.14 (MRR), 0.07 (HIT@5) - ROAR_co: 0.58 (MRR), 0.58 (HIT@5) #### (d) Targeted-Mitigation - **Chrome**: - ROAR_kp: 0.62 (MRR), 0.80 (HIT@5) - ROAR_qm: 0.43 (MRR), 0.30 (HIT@5) - ROAR_co: 0.66 (MRR), 0.85 (HIT@5) - **CAPEC-22**: - ROAR_kp: 0.49 (MRR), 0.44 (HIT@5) - ROAR_qm: 0.24 (MRR), 0.11 (HIT@5) - ROAR_co: 0.44 (MRR), 0.49 (HIT@5) - **T1550.001**: - ROAR_kp: 0.41 (MRR), 0.39 (HIT@5) - ROAR_qm: 0.10 (MRR), 0.06 (HIT@5) - ROAR_co: 0.41 (MRR), 0.39 (HIT@5) ### Key Observations 1. **ROAR_co Dominance**: - Consistently highest MRR across all categories (e.g., 0.86 in Targeted-Vulnerability Chrome) - Strong HIT@5 performance in Backdoor-Mitigation (0.68) and Targeted-Mitigation (0.85 Chrome) 2. **ROAR_kp Trade-offs**: - High MRR in Targeted-Vulnerability (0.74 Chrome) but lower HIT@5 (0.76) - Poorest HIT@5 in Backdoor-Mitigation CAPEC-22 (0.33) 3. **ROAR_qm Weaknesses**: - Lowest MRR in Backdoor-Mitigation T1550.001 (0.21) - Extremely low HIT@5 in Targeted-Vulnerability T1550.001 (0.07) 4. **Category-Specific Trends**: - Targeted-Mitigation shows highest MRR values overall (0.62–0.66) - Backdoor-Vulnerability has most balanced performance (MRR/HIT@5 ratios <1.2) ### Interpretation The data reveals method-specific trade-offs between MRR (precision) and HIT@5 (recall). ROAR_co excels in MRR but shows diminishing returns in HIT@5 for complex scenarios (e.g., Targeted-Mitigation T1550.001). ROAR_kp demonstrates situational strength in high-stakes categories (Targeted-Vulnerability) but underperforms in mitigation tasks. The consistent HIT@5 values in ROAR_co across subcategories suggest robustness in recall, while ROAR_qm's erratic performance indicates potential instability in complex environments. These findings imply that method selection should prioritize context-specific requirements: ROAR_co for precision-critical applications, ROAR_kp for recall-sensitive scenarios, and ROAR_qm for balanced but less extreme use cases.