## Code Comparison: LLM vs. LLM + APOLLO ### Overview The image presents a side-by-side comparison of code snippets, likely from a formal verification system like Lean, used to prove a mathematical theorem. The left section, labeled "LLM," shows a proof attempt using a Large Language Model. The right section, labeled "LLM + APOLLO," shows a proof attempt using a Large Language Model augmented with the APOLLO system. The code includes imports, theorem definitions, and proof steps. The goal is to demonstrate how APOLLO enhances the proof process. ### Components/Axes * **Headers:** * Left: "LLM" (Red background) * Right: "LLM + APOLLO" (Green background) * **Code Blocks:** Two blocks of code, one under each header, containing Lean code. * **Annotations:** "#1" is annotated in the right margin of each code block. * The code uses mathematical notation and Lean-specific syntax. ### Detailed Analysis or ### Content Details **Left Code Block (LLM):** * Imports: `import Mathlib`, `import Aesop` * Sets option `maxHeartbeats` to 0. * Opens `BigOperators Real Nat Topology Rat`. * Defines a theorem `mathd_algebra_293_llm` for `x : NNReal` (non-negative real number). * The theorem states: `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x)` * Proof starts with `:= by`. * `have h1`: Introduces a hypothesis stating `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63*x))`. * `rw`: Applies rewrite rules using `Real.sqrt_mul (by positivity)` twice. * `rw [h1]`: Rewrites using hypothesis `h1`. * `have h2`: Introduces another hypothesis stating `(60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x)`. * `ring_nf`: Simplifies using ring normal form. * `<;> simp [x_mul_x]`: Simplifies using the `x_mul_x` simplification rule. * `<;> linarith (show (0: R) ≤ x from by positivity]`: Attempts to prove a linear inequality. * `rw [h2]`: Rewrites using hypothesis `h2`. * `rw [Real.sqrt_mul (by positivity)]`: Applies rewrite rule. * `rw [Real.sqrt_sq (by simp [x])]`: Applies rewrite rule. * `all_goals positivity`: Attempts to prove all remaining goals are positivity conditions. * The last three lines are highlighted in red, indicating a potential failure or area of concern. **Right Code Block (LLM + APOLLO):** * Imports: `import Mathlib`, `import Aesop` * Sets option `maxHeartbeats` to 0. * Opens `BigOperators Real Nat Topology Rat`. * Defines a theorem `mathd_algebra_293_apollo` for `x : NNReal`. * The theorem states: `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x)` * Proof starts with `:= by`. * `have h1`: Introduces a hypothesis stating `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63 * x))`. * `rw`: Applies rewrite rules using `Real.sqrt_mul (by positivity)` twice. * `rw [h1]`: Rewrites using hypothesis `h1`. * `have h2`: Introduces another hypothesis stating `(60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x)`. * `ring_nf`: Simplifies using ring normal form. * `<;> simp [x_mul_x]`: Simplifies using the `x_mul_x` simplification rule. * `<;> linarith (show (0: R) ≤ x from by positivity]`: Attempts to prove a linear inequality. * `try norm_cast; try norm_num; try simp_all; try ring_nf at * ; try native_decide ; try linarith; try nlinarith`: Applies a series of simplification tactics. * `have h1: √(60: R) = 2 * √15 := by ...`: Introduces and proves a series of hypotheses about square roots of constants. * `have h2': √(12: R) = 2 * √3 := by ...` * `have h3 : √(63: R) = 3 * √7 := by ...` * `have h4: √(↑x: R) ^ (3 : N) = √(↑x : R) * (↑x: R) := by ...` * `have h41: √(↑x : R) ^ (3 : N) = √(↑x : R) ^ 2 * √(↑x: R) := by ...` * `have h42: √(↑x : R) ^ 2 = (↑x : R) := by ...` * `rw [Real.sq_sqrt]`: Applies rewrite rule. * `exact NNReal.zero_le_coe`: Proves a goal using the fact that non-negative real numbers are greater than or equal to zero. * `rw [h41, h42]`: Rewrites using hypotheses `h41` and `h42`. * `linarith`: Attempts to prove a linear inequality. * `rw [h1, h2, h3, h4]`: Rewrites using hypotheses `h1`, `h2`, `h3`, and `h4`. * `ring_nf`: Simplifies using ring normal form. * `<;> simp [mul_assoc]`: Simplifies using the `mul_assoc` simplification rule. * `try norm_cast; try norm_num; try simp_all; try ring_nf at * ; try native_decide ; try linarith; try nlinarith`: Applies a series of simplification tactics. * The code continues with more complex manipulations involving square roots and real numbers. **Right Code Block (LLM + APOLLO) - Continued:** * `have h2: √(15: R) * √(3: R) * √(7: R) * √(↑x: R) * (↑x : R) * (12 : R) = √(↑x : R) * (↑x: R) * √(35: R) * (36: R) := by` * `have h21: √(15: R) * √(3: R) * √(7: R) = √(315: R) := by` * `calc`: Starts a calculation block. * `√(15: R) * √(3: R) * √(7: R) = √(15 * 3 * 7) := by` * `rw [Real.sqrt_mul (by norm_num), ← Real.sqrt_mul (by norm_num)]` * `= √(315: R) := by norm_num` * `have h22: √(315 : R) * √(↑x : R) * (↑x: R) * (12 : R) = √(↑x : R) * (↑x: R) * √(35: R) * (36: R) := by` * `have h221: √(315: R) = √(35: R) * (3: R) := by` * `calc`: Starts a calculation block. * `√(315: R) = √(35*3^2) := by` * `= √(35) * √(3^2) := by` * `rw [Real.sqrt_mul (by norm_num)]` * `= √(35) * (3: R) := by` * `rw [Real.sqrt_sq (by norm_num)]` * `rw [h221]` * `ring_nf` * `<;> simp [Real.sqrt_mul]` * `<;> ring` * `rw [h21]` * `have h23: √(315: R) * √(↑x : R) * (↑x : R) * (12: R) = √(15: R) * √(3: R) * √(7: R) * √(↑x : R) * (↑x : R) * (12 : R) := by` * `rw [h21]` * `rw [h23]` * `linarith [h22]` * `rw [h2]` * `ring` ### Key Observations * Both code blocks start with the same imports and theorem definition. * The LLM code block is shorter and relies on more general tactics. * The LLM + APOLLO code block is longer and more explicit, breaking down the proof into smaller steps. * The LLM code block has the last three lines highlighted in red, suggesting a failure or incomplete proof. * The LLM + APOLLO code block appears to be more successful, with a more detailed and structured proof. ### Interpretation The image demonstrates the potential benefits of using the APOLLO system to augment Large Language Models in formal verification. The LLM alone struggles to complete the proof, as indicated by the highlighted lines. APOLLO provides additional guidance and tactics, allowing the system to break down the problem into smaller, more manageable steps. This results in a more detailed and potentially more successful proof. The comparison suggests that APOLLO enhances the reasoning capabilities of LLMs in the context of formal mathematical proofs.

## Diagram: Code Comparison - LLM vs. LLM + APOLLO ### Overview The image displays a side-by-side comparison of two code snippets written in the Lean 4 theorem proving language. The left panel, outlined and headered in red, represents an output generated by a standard "LLM" (Large Language Model). The right panel, outlined and headered in green, represents an output generated by "LLM + APOLLO". Both scripts attempt to prove the same mathematical theorem (`mathd_algebra_293`). The visual design (red vs. green) strongly implies that the left approach fails or is flawed, while the right approach succeeds. Specific lines of code in both panels are highlighted and tagged with "#1" to draw attention to differing methodologies. ### Components/Layout * **Left Panel (Red):** * **Header:** "LLM" (White text on a red background, top center). * **Border:** Solid red line. * **Content:** A single column of Lean 4 code. * **Highlight:** A block of code at the bottom is highlighted in light red with a dark red tag labeled `#1` positioned to the right of the first highlighted line. * **Right Panel (Green):** * **Header:** "LLM + APOLLO" (White text on a green background, top center). * **Border:** Solid green line. * **Content:** Lean 4 code that spans two columns. The code begins in the left half of the green box, reaches the bottom, and continues in a second column on the right half of the green box. * **Highlight:** A single line of code in the middle of the first column is highlighted in light green with a dark green tag labeled `#1` positioned to the right. ### Content Details #### Left Panel Transcription (LLM) ```lean import Mathlib import Aesop set_option maxHeartbeats 0 open BigOperators Real Nat Topology Rat theorem mathd_algebra_293_llm (x : NNReal) : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x) := by have h1 : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63 * x)) := by rw [← Real.sqrt_mul (by positivity), ← Real.sqrt_mul (by positivity)] rw [h1] have h2 : (60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x) := by ring_nf <;> simp [x_mul_x] <;> linarith [show (0 : ℝ) ≤ x from by positivity] ``` *Highlighted Red Section (Tag #1):* ```lean rw [h2] rw [Real.sqrt_mul (by positivity)] rw [Real.sqrt_sq (by simp [x])] all_goals positivity ``` #### Right Panel Transcription (LLM + APOLLO) *First Column:* ```lean import Mathlib import Aesop set_option maxHeartbeats 0 open BigOperators Real Nat Topology Rat theorem mathd_algebra_293_apollo (x : NNReal) : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x) := by have h1 : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63 * x)) := by rw [← Real.sqrt_mul (by positivity), ← Real.sqrt_mul (by positivity)] rw [h1] have h2 : (60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x) := by ring_nf <;> simp [x_mul_x] <;> linarith [show (0 : ℝ) ≤ x from by positivity] ``` *Highlighted Green Section (Tag #1):* ```lean try norm_cast ; try norm_num ; try simp_all ; try ring_nf at * ; try native_decide ; try linarith ; try nlinarith ``` *First Column (Continued below highlight):* ```lean have h1 : √(60 : ℝ) = 2 * √15 := by rw [Real.sqrt_eq_iff_sq_eq] <;> norm_num <;> ring_nf <;> norm_num have h2' : √(12 : ℝ) = 2 * √3 := by rw [Real.sqrt_eq_iff_sq_eq] <;> norm_num <;> ring_nf <;> norm_num have h3 : √(63 : ℝ) = 3 * √7 := by rw [Real.sqrt_eq_iff_sq_eq] <;> norm_num <;> ring_nf <;> norm_num have h4 : √(↑x : ℝ) ^ (3 : ℕ) = √(↑x : ℝ) * (↑x : ℝ) := by have h41 : √(↑x : ℝ) ^ (3 : ℕ) = √(↑x : ℝ) ^ 2 * √(↑x : ℝ) := by simp [pow_succ] have h42 : √(↑x : ℝ) ^ 2 = (↑x : ℝ) := by rw [Real.sq_sqrt] exact NNReal.zero_le_coe rw [h41, h42] linarith rw [h1, h2', h3, h4] ring_nf <;> simp [mul_assoc] try norm_cast ; try norm_num ; try simp_all ; try ring_nf at * ; try native_decide ; try linarith ; try nlinarith ... ``` *Second Column (Right side of the green panel):* ```lean ... have h2 : √(15 : ℝ) * √(3 : ℝ) * √(7 : ℝ) * √(↑x : ℝ) * (↑x : ℝ) * (12 : ℝ) = √(↑x : ℝ) * (↑x : ℝ) * √(35 : ℝ) * (36 : ℝ) := by have h21 : √(15 : ℝ) * √(3 : ℝ) * √(7 : ℝ) = √(315 : ℝ) := by calc √(15 : ℝ) * √(3 : ℝ) * √(7 : ℝ) = √(15 * 3 * 7) := by rw [← Real.sqrt_mul (by norm_num), ← Real.sqrt_mul (by norm_num)] _ = √(315 : ℝ) := by norm_num have h22 : √(315 : ℝ) * √(↑x : ℝ) * (↑x : ℝ) * (12 : ℝ) = √(↑x : ℝ) * (↑x : ℝ) * √(35 : ℝ) * (36 : ℝ) := by have h221 : √(315 : ℝ) = √(35 : ℝ) * (3 : ℝ) := by calc √(315 : ℝ) = √(35 * 3 ^ 2) := by norm_num _ = √(35) * √(3 ^ 2) := by rw [Real.sqrt_mul (by norm_num)] _ = √(35) * (3 : ℝ) := by rw [Real.sqrt_sq (by norm_num)] rw [h221] ring_nf <;> simp [Real.sqrt_mul] <;> ring rw [h21] have h23 : √(315 : ℝ) * √(↑x : ℝ) * (↑x : ℝ) * (12 : ℝ) = √(15 : ℝ) * √(3 : ℝ) * √(7 : ℝ) * √(↑x : ℝ) * (↑x : ℝ) * (12 : ℝ) := by rw [← h21] rw [h23] linarith [h22] rw [← h2] ring ``` ### Key Observations 1. **Initial Setup is Identical:** Both scripts start with the exact same imports, options, open namespaces, theorem declaration, and the first two `have` statements (`h1` and `h2`). 2. **Divergence Point:** The divergence occurs immediately after the proof of `h2`. 3. **The LLM Failure (Red #1):** The standard LLM attempts to finish the proof quickly using a few rewrite (`rw`) tactics. It tries to rewrite using `h2`, then split the square roots, and resolve the square of a square root. This approach is highlighted in red, indicating it is a hallucination, contains a syntax error, or fails to close the proof state due to missing side conditions (like coercions from `NNReal` to `Real`). 4. **The APOLLO Intervention (Green #1):** Instead of guessing the next few steps, the APOLLO system injects a robust "hammer" tactic string: `try norm_cast ; try norm_num ; try simp_all ; try ring_nf at * ; try native_decide ; try linarith ; try nlinarith`. This is designed to automatically simplify the state as much as possible without failing if a specific tactic doesn't apply. 5. **Granular Proof Steps (Green Panel):** After the `#1` highlight, the APOLLO version abandons the short-cut approach. It breaks the problem down into highly granular, explicit sub-proofs (`h1`, `h2'`, `h3`, `h4`). It explicitly handles the coercion of `x` from `NNReal` to `Real` (denoted by the `↑x` symbol) and manually proves the prime factorizations inside the square roots (e.g., `√(60) = 2 * √15`). It utilizes the `calc` environment for step-by-step equational reasoning. ### Interpretation This image serves as a technical demonstration of the limitations of standard Large Language Models in formal theorem proving, and how an augmented system ("APOLLO") overcomes them. * **The Problem with Standard LLMs:** The red panel shows that a standard LLM tries to write Lean code the way a human might write a math proof on a whiteboard—skipping steps and assuming the system will "understand" the algebraic leaps. However, Lean 4 is a strict formal verifier. The LLM's attempt to jump straight to the conclusion using `rw [Real.sqrt_sq (by simp [x])]` fails because it hasn't properly managed the types (non-negative reals vs. reals) or the exact syntactic shape of the goal. * **The APOLLO Solution:** The green panel demonstrates that APOLLO likely uses a combination of automated tactic search (the `#1` highlight) and agentic step-by-step planning. By forcing the LLM to prove every single algebraic simplification as its own isolated lemma (e.g., proving `√(63) = 3 * √7` before using it), APOLLO ensures the strict type-checker is satisfied at every step. * **Conclusion:** The graphic effectively communicates that while LLMs understand the *concept* of the proof, they require a scaffolding framework like APOLLO to generate the verbose, mathematically rigorous, and type-safe code required to actually compile a proof in Lean 4.

## Diagram: Mathematical Proof/Code Snippets ### Overview The image presents a side-by-side comparison of mathematical code and proof snippets, likely generated by a Large Language Model (LLM) and a system called "APOLLO". The content appears to be related to real number topology and calculations involving square roots and norms. The diagram is structured as a series of code blocks with annotations indicating the flow of logic or derivation steps. The left side is labeled "LLM", the center "APOLLO", and the right side "LLM + APOLLO". ### Components/Axes The diagram is not a traditional chart with axes. Instead, it consists of several distinct code blocks and text annotations. Key components include: * **Headers:** "LLM", "APOLLO", "LLM + APOLLO" at the top of each column. * **Code Blocks:** Representing mathematical expressions and code snippets. These are formatted with indentation and annotations. * **Annotations:** Textual notes (e.g., "by positivity", "by norm num", "calc") indicating the reasoning or operation applied. * **Mathematical Expressions:** Involving `Real.sqrt`, `Real.mul`, `Real.Topology Rat`, `norm`, `norm num`, and variables like `x`, `y`, `z`. * **Footer:** Contains information about the model versions and a timestamp. ### Detailed Analysis or Content Details **LLM Column (Left):** * `import Mathlib` * `import Aesop` * `set_option maxHeartbeats 0` * `open BigOperators Real Nat Topology Rat` * `theorem mathd_algebra_293_llm (x : NNReal) : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt (36 * x) * Real.sqrt (35 * x) := by` * `have hi : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt (60 * x) * (12 * x) * Real.sqrt (63 * x) := by` * `rw [Real.sqrt_mul (by positivity), Real.sqrt_mul (by positivity)]` * `have h1 : (60 * x) * (12 * x) * (63 * x) = (36 * x) * (35 * x) := by` * `rw [Real.sqrt_mul (by positivity), Real.sqrt_mul (by positivity)]` * `have h2 : (60 * x) * (12 * x) * (63 * x) = (36 * x) * (35 * x) := by` * `simp` * `trace_state 'show [norm x, try rfl by positivity]'` * `apply h1; try ring_nf; abel` * `have h3 : (60 * x) * (12 * x) * (63 * x) = (36 * x) * (35 * x) := by` * `rw [Real.sqrt_mul (by positivity), Real.sqrt_mul (by positivity)]` * `have h4 : (60 * x) * (12 * x) * (63 * x) = (36 * x) * (35 * x) := by` * `simp` * `trace_state 'show [norm x, try rfl by positivity]'` * `apply h3; try ring_nf; abel` **APOLLO Column (Center):** * `import Mathlib` * `import Aesop` * `set_option maxHeartbeats 0` * `open BigOperators Real Nat Topology Rat` * `theorem mathd_algebra_293_apollo (x : NNReal) : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt (36 * x) * Real.sqrt (35 * x) := by` * `have hi : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt (60 * x) * (12 * x) * Real.sqrt (63 * x) := by` * `rw [Real.sqrt_mul (by positivity)]` * `have h1 : (60 * x) * (12 * x) * (63 * x) = (36 * x) * (35 * x) := by` * `rw [Real.sqrt_mul (by positivity)]` * `have h2 : (60 * x) * (12 * x) * (63 * x) = (36 * x) * (35 * x) := by` * `calc` * `√(15 * 3 * 7) := by` * `√(15 * 3 * 7) := by` * `rw [Real.sqrt_mul (by norm num)]` * `have h2a : √(315) = √(3 * 3 * 5 * 7) := by` * `(36 * 12) + √(21) := √(3 * √(3 * √(5 * 7))) := by` * `have h2b : √(315) = √(35 * 9) := by` * `calc` * `√(35 * 9) = 3 * √(35) := by` * `rw [Real.sqrt_mul (by norm num)]` * `have h2b1 : √(35 * 9) = 3 * √(35) := by` * `rw [Real.sqrt_mul (by norm num)]` **LLM + APOLLO Column (Right):** * `have h2 : (√ (15) * √ (3) * √ (7)) * √ (7) := by` * `have h2 : (√ (15) * √ (3) * √ (7)) * √ (7) := by` * `calc` * `√(15 * 3 * 7) := by` * `√(15 * 3 * 7) := by` * `rw [Real.sqrt_mul (by norm num)]` * `have h2a : √(315) = √(3 * 3 * 5 * 7) := by` * `(36 * 12) + √(21) := √(3 * √(3 * √(5 * 7))) := by` * `have h2b : √(315) = √(35 * 9) := by` * `calc` * `√(35 * 9) = 3 * √(35) := by` * `rw [Real.sqrt_mul (by norm num)]` * `have h2b1 : √(35 * 9) = 3 * √(35) := by` * `rw [Real.sqrt_mul (by norm num)]` **Footer:** * `model_version = 'v0.3.26'` * `date = '2024-01-26'` * `commit = '8f8f599'` ### Key Observations * The LLM and APOLLO columns attempt to prove the same theorem. * The APOLLO column includes more detailed calculation steps ("calc") and intermediate results. * Both columns utilize `Real.sqrt_mul` with the justification "by positivity" or "by norm num". * The "trace\_state" annotations in the LLM column suggest debugging or tracing of the proof process. * The LLM + APOLLO column appears to be a combination of the outputs from the other two columns. ### Interpretation The diagram demonstrates a comparison of mathematical reasoning and proof generation between a standard LLM and the APOLLO system. APOLLO seems to provide a more step-by-step, detailed breakdown of the calculation, potentially making the proof more transparent and easier to verify. The use of "calc" and intermediate results suggests a more explicit computational approach. The LLM, while attempting the same proof, relies more on higher-level transformations and annotations like "trace\_state". The combination in the "LLM + APOLLO" column suggests a potential synergy where APOLLO's detailed calculations can augment the LLM's reasoning capabilities. The footer information indicates the specific versions of the models and the date of the output, providing context for reproducibility and analysis. The overall goal appears to be to evaluate and improve the ability of LLMs to perform rigorous mathematical proofs. The repeated use of `Real.sqrt_mul` suggests this is a core operation in the proof process, and the "by positivity" justification indicates a reliance on the non-negativity of real numbers.

\n ## Code Comparison Diagram: LLM vs. LLM + APOLLO Proof Generation ### Overview The image displays a side-by-side comparison of two Lean 4 code blocks, each contained within a colored, rounded rectangle. The left block, outlined in red, is labeled "LLM". The right block, outlined in green, is labeled "LLM + APOLLO". Both blocks contain formal mathematical proofs for what appears to be the same underlying algebraic theorem, but the proof strategies and lengths differ significantly. The "LLM + APOLLO" proof is substantially longer and more detailed. ### Components/Axes * **Layout:** Two vertical panels. * **Left Panel (LLM):** Red border, light pink background. Header text: "LLM" in a red box. * **Right Panel (LLM + APOLLO):** Green border, light green background. Header text: "LLM + APOLLO" in a green box. * **Content Type:** Formal proof code written in the Lean 4 theorem prover language. * **Language:** The primary language is Lean 4 code. Mathematical notation is embedded within the code (e.g., `√`, `*`, `^`, `:=`). * **Annotations:** The left panel contains a highlighted red section with the comment `#1`. ### Detailed Analysis #### **Left Panel: LLM Proof** **Theorem Name:** `mathd_algebra_293_llm` **Goal:** Prove that for `x : NNReal`, `Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x)`. **Proof Structure (Transcribed):** ```lean import Mathlib import Aesop set_option maxHeartbeats 0 open BigOperators Real Nat Topology Rat theorem mathd_algebra_293_llm (x : NNReal) : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = 36 * x * Real.sqrt (35 * x) := by have h1 : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63 * x)) := by rw [← Real.sqrt_mul (by positivity), ← Real.sqrt_mul (by positivity)] rw [h1] have h2 : (60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x) := by ring_nf <;> simp [x_mul_x] <;> linarith [show (0 : ℝ) ≤ x from by positivity] rw [h2] -- #1 rw [Real.sqrt_mul (by positivity)] rw [Real.sqrt_sq (by simp [x])] all_goals positivity ``` * **Key Steps:** The proof uses a `have` tactic to assert an intermediate equality `h1` by combining square roots. It then rewrites using `h1` and asserts a second equality `h2` about the product inside the square root. The final steps rewrite using `h2` and simplify the square root of a square. * **Highlighted Section:** The lines from `rw [h2]` to `all_goals positivity` are highlighted in a darker pink/red, with a comment `#1` in the right margin. #### **Right Panel: LLM + APOLLO Proof** **Theorem Name:** `mathd_algebra_293_apollo` **Goal:** Identical to the LLM proof. **Proof Structure (Transcribed - Key Sections):** The proof is much longer. It begins with similar imports and setup, then proceeds with a more granular, step-by-step approach. **Initial Steps (Similar to LLM):** ```lean have h1 : Real.sqrt (60 * x) * Real.sqrt (12 * x) * Real.sqrt (63 * x) = Real.sqrt ((60 * x) * (12 * x) * (63 * x)) := by rw [← Real.sqrt_mul (by positivity), ← Real.sqrt_mul (by positivity)] rw [h1] have h2 : (60 * x) * (12 * x) * (63 * x) = (36 * x) ^ 2 * (35 * x) := by ring_nf <;> simp [x_mul_x] <;> linarith [show (0 : ℝ) ≤ x from by positivity] ``` **Divergence and Elaboration (After `rw [h2]`):** Instead of directly simplifying the square root, the proof enters a long sequence of `try` tactics and then explicitly breaks down the square roots of the numerical coefficients. ```lean try norm_cast ; try norm_num ; try simp_all ; try ring_nf at * ; try native_decide ; try linarith ; try nlinarith have h1 : √(60 : ℝ) = 2 * √15 := by rw [Real.sqrt_eq_iff_sq_eq] <;> norm_num <;> ring_nf <;> norm_num have h2' : √(12 : ℝ) = 2 * √3 := by ... (similar structure) have h3 : √(63 : ℝ) = 3 * √7 := by ... have h4 : √(↑x : ℝ) ^ (3 : ℕ) = √(↑x : ℝ) * (↑x : ℝ) := by ... ``` The proof then continues with a lengthy `calc` block that meticulously shows the equality step-by-step, factoring and recombining the square roots of the prime factors (15, 3, 7) and the variable `x`. It uses lemmas like `h21`, `h22`, `h221`, and `h23` to manage intermediate results. **Final Lines:** ```lean rw [h23] linarith [h22] rw [← h2] ring ``` ### Key Observations 1. **Proof Strategy:** The LLM proof is concise, relying on powerful automation tactics (`ring_nf`, `simp`, `linarith`) to close the goal quickly after asserting the key algebraic identity `h2`. The LLM+APOLLO proof is verbose and explicit, manually decomposing the problem into lemmas about individual square roots and then reconstructing the proof in a `calc` block. 2. **Length and Detail:** The LLM+APOLLO proof is approximately 3-4 times longer in terms of lines of code. 3. **Use of Automation:** The LLM proof uses automation aggressively at the end. The LLM+APOLLO proof uses a block of `try` tactics early on but then proceeds with mostly manual, detailed steps. 4. **Highlighted Region:** The red highlight in the LLM panel draws attention to the final simplification steps, which are the point where the concise automation is applied. ### Interpretation This image is a technical comparison demonstrating two different approaches to automated theorem proving in Lean 4 for the same mathematical statement. * **What it shows:** It contrasts a "standard" LLM-generated proof (left) with a proof generated or augmented by a system called "APOLLO" (right). The LLM proof is efficient and relies on the prover's built-in automation. The APOLLO-augmented proof is more pedagogical and explicit, breaking the problem down into fundamental algebraic steps. * **Relationship between elements:** The side-by-side layout invites direct comparison. The identical theorem statement and initial steps establish a common ground, highlighting the divergence in proof strategy after the key identity `h2` is established. * **Potential Meaning:** The comparison likely aims to showcase a trade-off. The LLM proof is shorter and may be preferred for efficiency. The APOLLO proof, while longer, is more transparent, easier for a human to follow, and may be more robust or generalizable because it avoids relying on the "black box" of powerful automation tactics. The highlighted section in the LLM proof may indicate the specific point where APOLLO's more detailed approach begins. This could be illustrating APOLLO's ability to generate more verifiable, step-by-step proofs compared to an LLM that might take shortcuts via automation.

## Lean 4 Theorem Proofs: LLM vs LLM + APOLLO ### Overview The image compares two Lean 4 theorem proofs for the same algebraic identity: `Real.sqrt(60 * x) * Real.sqrt(12 * x) * Real.sqrt(63 * x) = 36 * x * Real.sqrt(35 * x)`. The left side shows a proof generated by LLM alone, while the right side demonstrates an enhanced proof using LLM + APOLLO. Both proofs use Lean's `Mathlib` and `Aesop` libraries with `BigOperators` and `Real.Nat.Topology`. --- ### Components/Axes **Left Panel (LLM):** - **Imports:** `Mathlib`, `Aesop` - **Options:** `set_option maxHeartbeats 0` - **Theorem:** `mathd_algebra_293_llm` - **Proof Tactics:** - `have h1 : ...` - `ring_nf` - `simp [x_mul_x]` - `linarith [show (0 : ℝ) ≤ x from by positivity]` - **Highlighted Section:** - Red box around `rw [h2]`, `rw [Real.sqrt_mul (by positivity)]`, and `all_goals positivity` **Right Panel (LLM + APOLLO):** - **Imports:** Same as LLM - **Theorem:** `mathd_algebra_293_apollo` - **Proof Tactics:** - `have h1, h2, h3, h4` with complex intermediate steps - `try norm_cast`, `try simp_all`, `try ring_nf` - `try linarith` with nested `simp` and `native_decide` - **Highlighted Section:** - Green box around `try norm_cast`, `try simp_all`, and `try ring_nf` --- ### Detailed Analysis **LLM Proof (Left):** 1. **Structure:** - Directly applies `ring_nf` and `simp` to reduce terms. - Uses `linarith` with a custom `show` clause to enforce positivity. 2. **Key Steps:** - `h1` binds the left-hand side (LHS) of the equation. - `ring_nf` simplifies polynomial expressions. - `linarith` proves the inequality `0 ≤ x` via positivity. **LLM + APOLLO Proof (Right):** 1. **Structure:** - Introduces multiple lemmas (`h1`, `h2`, `h3`, `h4`) for intermediate steps. - Uses `norm_cast` to handle norm conversions. - Employs `simp_all` and `native_decide` for automated simplification. 2. **Key Steps:** - `h2` and `h3` break down the equation into smaller components. - `simp [pow_succ]` and `simp [mul_assoc]` handle exponentiation and associativity. - `linarith` is used without custom `show` clauses, relying on APOLLO's optimizations. --- ### Key Observations 1. **Tactic Complexity:** - LLM + APOLLO uses 4x more `have` clauses and 3x more `try` blocks, suggesting deeper decomposition of the problem. 2. **Normalization:** - APOLLO introduces `norm_cast` and `simp_all`, indicating better handling of norm-related terms. 3. **Efficiency:** - LLM requires manual positivity enforcement (`show (0 ≤ x)`), while APOLLO automates this via `native_decide`. --- ### Interpretation The LLM + APOLLO proof demonstrates a more systematic approach to algebraic simplification: - **Decomposition:** Breaking the equation into smaller lemmas (`h1`, `h2`, etc.) allows incremental verification. - **Automation:** Tactics like `simp_all` and `native_decide` reduce manual case analysis, making the proof more robust. - **Normalization:** Explicit handling of norms (`norm_cast`) ensures correctness in real-number arithmetic. The LLM proof, while concise, relies on Lean's default behavior for positivity, which may fail for edge cases. APOLLO's additions suggest a focus on generalizability and error resilience. **Final Note:** Both proofs validate the same identity, but LLM + APOLLO provides a more transparent and maintainable structure, critical for complex mathematical reasoning.