## Code Comparison: LLM vs. LLM + APOLLO ### Overview The image presents a side-by-side comparison of code snippets, likely from a formal verification system like Lean, used to prove a mathematical theorem. The left section, labeled "LLM," shows a proof attempt using a Large Language Model. The right section, labeled "LLM + APOLLO," shows a proof attempt using a Large Language Model augmented with the APOLLO system. The code includes imports, theorem definitions, and proof steps. The goal is to demonstrate how APOLLO enhances the proof process. ### Components/Axes * **Headers:** * Left: "LLM" (Red background) * Right: "LLM + APOLLO" (Green background) * **Code Blocks:** Two blocks of code, one under each header, containing Lean code. * **Annotations:** "#1" is annotated in the right margin of each code block. * The code uses mathematical notation and Lean-specific syntax. ### Detailed Analysis or ### Content Details **Left Code Block (LLM):** * Imports: `import Mathlib`, `import Aesop` * Sets option `maxHeartbeats` to 0. * Opens `BigOperators Real Nat Topology Rat`. * Defines a theorem `mathd_algebra_293_llm` for `x : NNReal` (non-negative real number). * The theorem states: `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x)` * Proof starts with `:= by`. * `have h1`: Introduces a hypothesis stating `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63*x))`. * `rw`: Applies rewrite rules using `Real.sqrt_mul (by positivity)` twice. * `rw [h1]`: Rewrites using hypothesis `h1`. * `have h2`: Introduces another hypothesis stating `(60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x)`. * `ring_nf`: Simplifies using ring normal form. * `<;> simp [x_mul_x]`: Simplifies using the `x_mul_x` simplification rule. * `<;> linarith (show (0: R) ≤ x from by positivity]`: Attempts to prove a linear inequality. * `rw [h2]`: Rewrites using hypothesis `h2`. * `rw [Real.sqrt_mul (by positivity)]`: Applies rewrite rule. * `rw [Real.sqrt_sq (by simp [x])]`: Applies rewrite rule. * `all_goals positivity`: Attempts to prove all remaining goals are positivity conditions. * The last three lines are highlighted in red, indicating a potential failure or area of concern. **Right Code Block (LLM + APOLLO):** * Imports: `import Mathlib`, `import Aesop` * Sets option `maxHeartbeats` to 0. * Opens `BigOperators Real Nat Topology Rat`. * Defines a theorem `mathd_algebra_293_apollo` for `x : NNReal`. * The theorem states: `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x)` * Proof starts with `:= by`. * `have h1`: Introduces a hypothesis stating `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63 * x))`. * `rw`: Applies rewrite rules using `Real.sqrt_mul (by positivity)` twice. * `rw [h1]`: Rewrites using hypothesis `h1`. * `have h2`: Introduces another hypothesis stating `(60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x)`. * `ring_nf`: Simplifies using ring normal form. * `<;> simp [x_mul_x]`: Simplifies using the `x_mul_x` simplification rule. * `<;> linarith (show (0: R) ≤ x from by positivity]`: Attempts to prove a linear inequality. * `try norm_cast; try norm_num; try simp_all; try ring_nf at * ; try native_decide ; try linarith; try nlinarith`: Applies a series of simplification tactics. * `have h1: √(60: R) = 2 * √15 := by ...`: Introduces and proves a series of hypotheses about square roots of constants. * `have h2': √(12: R) = 2 * √3 := by ...` * `have h3 : √(63: R) = 3 * √7 := by ...` * `have h4: √(↑x: R) ^ (3 : N) = √(↑x : R) * (↑x: R) := by ...` * `have h41: √(↑x : R) ^ (3 : N) = √(↑x : R) ^ 2 * √(↑x: R) := by ...` * `have h42: √(↑x : R) ^ 2 = (↑x : R) := by ...` * `rw [Real.sq_sqrt]`: Applies rewrite rule. * `exact NNReal.zero_le_coe`: Proves a goal using the fact that non-negative real numbers are greater than or equal to zero. * `rw [h41, h42]`: Rewrites using hypotheses `h41` and `h42`. * `linarith`: Attempts to prove a linear inequality. * `rw [h1, h2, h3, h4]`: Rewrites using hypotheses `h1`, `h2`, `h3`, and `h4`. * `ring_nf`: Simplifies using ring normal form. * `<;> simp [mul_assoc]`: Simplifies using the `mul_assoc` simplification rule. * `try norm_cast; try norm_num; try simp_all; try ring_nf at * ; try native_decide ; try linarith; try nlinarith`: Applies a series of simplification tactics. * The code continues with more complex manipulations involving square roots and real numbers. **Right Code Block (LLM + APOLLO) - Continued:** * `have h2: √(15: R) * √(3: R) * √(7: R) * √(↑x: R) * (↑x : R) * (12 : R) = √(↑x : R) * (↑x: R) * √(35: R) * (36: R) := by` * `have h21: √(15: R) * √(3: R) * √(7: R) = √(315: R) := by` * `calc`: Starts a calculation block. * `√(15: R) * √(3: R) * √(7: R) = √(15 * 3 * 7) := by` * `rw [Real.sqrt_mul (by norm_num), ← Real.sqrt_mul (by norm_num)]` * `= √(315: R) := by norm_num` * `have h22: √(315 : R) * √(↑x : R) * (↑x: R) * (12 : R) = √(↑x : R) * (↑x: R) * √(35: R) * (36: R) := by` * `have h221: √(315: R) = √(35: R) * (3: R) := by` * `calc`: Starts a calculation block. * `√(315: R) = √(35*3^2) := by` * `= √(35) * √(3^2) := by` * `rw [Real.sqrt_mul (by norm_num)]` * `= √(35) * (3: R) := by` * `rw [Real.sqrt_sq (by norm_num)]` * `rw [h221]` * `ring_nf` * `<;> simp [Real.sqrt_mul]` * `<;> ring` * `rw [h21]` * `have h23: √(315: R) * √(↑x : R) * (↑x : R) * (12: R) = √(15: R) * √(3: R) * √(7: R) * √(↑x : R) * (↑x : R) * (12 : R) := by` * `rw [h21]` * `rw [h23]` * `linarith [h22]` * `rw [h2]` * `ring` ### Key Observations * Both code blocks start with the same imports and theorem definition. * The LLM code block is shorter and relies on more general tactics. * The LLM + APOLLO code block is longer and more explicit, breaking down the proof into smaller steps. * The LLM code block has the last three lines highlighted in red, suggesting a failure or incomplete proof. * The LLM + APOLLO code block appears to be more successful, with a more detailed and structured proof. ### Interpretation The image demonstrates the potential benefits of using the APOLLO system to augment Large Language Models in formal verification. The LLM alone struggles to complete the proof, as indicated by the highlighted lines. APOLLO provides additional guidance and tactics, allowing the system to break down the problem into smaller, more manageable steps. This results in a more detailed and potentially more successful proof. The comparison suggests that APOLLO enhances the reasoning capabilities of LLMs in the context of formal mathematical proofs.