\n ## Chart: Cumulative Solving + Checking Time ### Overview The image presents a line chart illustrating the cumulative solving and checking time for three different solver/checker combinations across a varying number of benchmarks. The y-axis represents time in seconds on a logarithmic scale, while the x-axis represents the number of benchmarks. The chart aims to compare the performance of these solvers as the number of benchmarks increases. ### Components/Axes * **Title:** "Cumulative solving + checking time" (centered at the top) * **X-axis Label:** "Number of benchmarks" (bottom-center) * **Y-axis Label:** "Time (s)" (left-center) - Logarithmic scale. * **Legend:** Located in the bottom-right corner. * "Duper" (Blue line) * "cvc5+Lean-SMT" (Orange line) * "veriT+Sledgehammer" (Green line) * **Gridlines:** Dashed horizontal and vertical lines providing a visual reference. * **Y-axis Scale:** Logarithmic, with markers at 10^-2, 10^-1, 10⁰, 10¹, 10², 10³, 10⁴. ### Detailed Analysis The chart displays three distinct curves, each representing a solver/checker combination. * **Duper (Blue Line):** The blue line starts at approximately 0.1 seconds at 0 benchmarks and rapidly increases, reaching approximately 10⁴ seconds (10,000 seconds) at 2500 benchmarks. The curve initially shows a steep slope, which gradually flattens as the number of benchmarks increases. * At 100 benchmarks: ~10¹ seconds (10 seconds) * At 500 benchmarks: ~10² seconds (100 seconds) * At 1000 benchmarks: ~300 seconds * At 1500 benchmarks: ~600 seconds * At 2000 benchmarks: ~1500 seconds * At 2500 benchmarks: ~10,000 seconds * **cvc5+Lean-SMT (Orange Line):** The orange line starts at approximately 0.01 seconds at 0 benchmarks and increases more gradually than the blue line. It reaches approximately 500 seconds at 2500 benchmarks. The slope is consistently less steep than the blue line. * At 100 benchmarks: ~0.5 seconds * At 500 benchmarks: ~20 seconds * At 1000 benchmarks: ~70 seconds * At 1500 benchmarks: ~150 seconds * At 2000 benchmarks: ~250 seconds * At 2500 benchmarks: ~500 seconds * **veriT+Sledgehammer (Green Line):** The green line starts at approximately 0.001 seconds at 0 benchmarks and exhibits the slowest growth among the three. It reaches approximately 100 seconds at 2500 benchmarks. The slope is consistently the least steep. * At 100 benchmarks: ~0.1 seconds * At 500 benchmarks: ~1 seconds * At 1000 benchmarks: ~5 seconds * At 1500 benchmarks: ~15 seconds * At 2000 benchmarks: ~30 seconds * At 2500 benchmarks: ~100 seconds ### Key Observations * Duper consistently exhibits the highest cumulative solving and checking time across all benchmark counts. * cvc5+Lean-SMT performs better than Duper, with significantly lower cumulative times. * veriT+Sledgehammer demonstrates the best performance, with the lowest cumulative times throughout the entire range of benchmarks. * The logarithmic scale emphasizes the differences in performance, particularly at higher benchmark counts. * All three lines show diminishing returns, meaning the increase in time slows down as the number of benchmarks increases. ### Interpretation The chart demonstrates a clear performance hierarchy among the three solver/checker combinations. veriT+Sledgehammer is the most efficient, followed by cvc5+Lean-SMT, and then Duper. This suggests that veriT+Sledgehammer scales better with an increasing number of benchmarks. The logarithmic scale highlights the substantial difference in performance, especially as the problem size (number of benchmarks) grows. The diminishing returns observed in all three curves indicate that the complexity of solving each additional benchmark increases, but veriT+Sledgehammer handles this complexity more effectively. This data is valuable for selecting the appropriate solver/checker for a given task, considering the expected number of benchmarks and the acceptable solving time. The chart suggests that for a large number of benchmarks, veriT+Sledgehammer would be the preferred choice.