\n ## Chart: Property Verification Performance vs. Computation Time ### Overview The image presents two line charts comparing the performance of different methods (interval, planetsym-feasible, planet-feasible, planet-opt) in verifying properties. The charts show the percentage of properties verified as a function of computation time, for two different datasets: CollisionDetection and ACAS. ### Components/Axes * **X-axis (both charts):** Computation time (in seconds), displayed on a logarithmic scale from 10⁰ to 10³. * **Y-axis (both charts):** % of properties verified, ranging from 0 to 100. * **Chart (a):** CollisionDetection Dataset * **Chart (b):** ACAS Dataset * **Legend (both charts):** * Blue Line: interval * Orange Line: planetsym-feasible * Green Line: planet-feasible * Red Line: planet-opt ### Detailed Analysis or Content Details **Chart (a): CollisionDetection Dataset** * **interval (Blue):** Starts at approximately 0% verified at 10⁰ seconds, rises sharply to approximately 80% verified at 10¹ seconds, and plateaus around 85% verified for computation times greater than 10¹ seconds. * **planetsym-feasible (Orange):** Starts at approximately 0% verified at 10⁰ seconds, rises gradually to approximately 60% verified at 10² seconds, and plateaus around 65% verified for computation times greater than 10² seconds. * **planet-feasible (Green):** Starts at approximately 0% verified at 10⁰ seconds, rises sharply to approximately 85% verified at 10¹ seconds, and plateaus around 90% verified for computation times greater than 10¹ seconds. * **planet-opt (Red):** Starts at approximately 0% verified at 10⁰ seconds, rises very sharply to approximately 100% verified at 10¹ seconds, and remains at 100% for all subsequent computation times. **Chart (b): ACAS Dataset** * **interval (Blue):** Starts at approximately 0% verified at 10⁰ seconds, rises slowly to approximately 20% verified at 10² seconds, and continues to rise gradually, reaching approximately 45% verified at 10³ seconds. * **planetsym-feasible (Orange):** Starts at approximately 0% verified at 10⁰ seconds, rises to approximately 20% verified at 10¹ seconds, and plateaus around 45% verified for computation times greater than 10² seconds. * **planet-feasible (Green):** Starts at approximately 0% verified at 10⁰ seconds, rises to approximately 40% verified at 10¹ seconds, and plateaus around 50% verified for computation times greater than 10² seconds. * **planet-opt (Red):** Starts at approximately 0% verified at 10⁰ seconds, rises to approximately 40% verified at 10¹ seconds, and plateaus around 50% verified for computation times greater than 10² seconds. ### Key Observations * For the CollisionDetection dataset, `planet-opt` consistently outperforms all other methods, achieving 100% verification at a relatively low computation time (10¹ seconds). `planet-feasible` also performs well, reaching approximately 90% verification at the same computation time. * For the ACAS dataset, the performance of all methods is significantly lower than for the CollisionDetection dataset. `planet-opt` and `planet-feasible` achieve the highest verification rates, plateauing around 50%. The `interval` method shows the slowest performance. * The logarithmic scale of the x-axis highlights the rapid initial gains in property verification for most methods. ### Interpretation The charts demonstrate the trade-off between computation time and property verification accuracy for different methods on two distinct datasets. The `planet-opt` method appears to be the most efficient for the CollisionDetection dataset, achieving complete verification quickly. However, its performance is comparable to other methods on the ACAS dataset, suggesting that the effectiveness of each method is dataset-dependent. The significant difference in performance between the two datasets indicates that the complexity of the properties being verified, or the characteristics of the datasets themselves, play a crucial role in determining the suitability of each method. The ACAS dataset appears to be more challenging to verify, requiring significantly more computation time to achieve comparable levels of verification. The slow performance of the `interval` method on the ACAS dataset suggests that it may not be well-suited for this type of problem. The plateauing of the verification rates for all methods on the ACAS dataset indicates that there may be inherent limitations in their ability to verify all properties, or that the remaining properties are significantly more difficult to verify.