## Diagram: LLM-Based Math Word Problem Solving Pipeline ### Overview This image is a technical flowchart illustrating a pipeline for solving math word problems using a Large Language Model (LLM) enhanced with pattern recognition and clustering. The process flows from left to right, starting with a dataset of problems, extracting reasoning patterns, and applying them to solve a new downstream task. The diagram uses color-coded boxes and arrows to denote different stages and data flow. ### Components/Axes The diagram is segmented into several key components, arranged spatially as follows: 1. **Left Column (Input & Demonstration):** * **Top-Left (Blue Box):** `Dataset`. Contains example questions (Q₁, Q₂, ... Qₙ). * **Bottom-Left (Orange Box):** `Seed Demonstrations`. Contains a detailed worked example (Q₁ and A₁) showing step-by-step reasoning. * **Icon & Text:** An LLM robot icon with the text `Let's think step by step` points from the Dataset to the Seed Demonstrations. 2. **Center Column (Pattern Processing):** * **Top-Center (Orange Box):** `Pattern Wise Context`. Contains multiple question-answer pairs (Q₁/A₁, Q₂/A₂, ... Qₖ/Aₖ), each labeled with a `(Reasoning Pattern X)`. * **Middle-Center (Green Boxes):** * `K-Clustering` (with a cluster icon). * `Adaptive K` (with a square root of x icon). * `Embeddings` (with a row of colored squares labeled: `twice`, `x`, `-`, `divide`, `÷`, `=`, `+`). * **Bottom-Center (Purple & Teal Boxes):** * `Pattern Discovery` containing `Prior Knowledge` (globe icon) and `LLM Prompting` (robot icon). * `Task Patterns` containing example phrases: `twice the age of`, `divide both sides`, `7+2=9 years old`. 3. **Right Column (Application & Output):** * **Top-Right (Green Box):** `Downstream Task`. Contains a `Pattern Wise Context` section (Q₁, A₁) and a new `Question` about a sport utility vehicle's value. * **Bottom-Right (Green Box):** `Final Answer`. Contains the LLM's step-by-step solution to the downstream task question, culminating in the answer `$20,000` with a green checkmark. * **Icons:** An LLM robot icon and a lightbulb icon are positioned between the Downstream Task and Final Answer boxes. **Flow Arrows:** Orange arrows connect the major stages: Dataset -> Seed Demonstrations -> Pattern Wise Context -> Downstream Task -> Final Answer. Green arrows connect the pattern processing components (K-Clustering, Embeddings, Pattern Discovery) to the main flow. ### Detailed Analysis **Textual Content Extraction:** * **Dataset Box:** * `Q₁: Liam is 16 years old now. Two years ago, ...` * `Q₂: Melanie, Sally, and Jessica each have ...` * `...` * `Qₙ: There were a total of 6 soccer games this ...` * **Seed Demonstrations Box:** * `Q₁: Liam is 16 years old now. Two years ago, Liam's age was twice the age of Vince. How old is Vince now?` * `A₁: Let's think step by step. 2 years ago, Liam's age was twice the age of Vince. So, we can write an equation based on this information: Liam's age 2 years ago = 2 x Vince's age 2 years ago. We know that Liam's current age is 16, so his age 2 years ago would have been 16 - 2 = 14. Now, we can substitute this value into our equation: 14 = 2 x Vince's age 2 years ago To solve for Vince's age, we can divide both sides of the equation by 2: 7 = Vince's age 2 years ago So, Vince is currently 7 + 2 = 9 years old.` * `...` * `Qₙ: ...` * **Pattern Wise Context Box:** * `Q₁: Nancy has saved 4900 cents ... (Reasoning Pattern 1) ↓` * `A₁: ... saved 4900 / 100 = 49 dollars ...` * `Q₂: Tom was at the beach for 5 days ... (Reasoning Pattern 2) ↓` * `A₂: ... a total of 7 x 5 = 35 seashells ...` * `...` * `Qₖ: Tim's cat had kittens ... (Reasoning Pattern k) ↓` * `Aₖ: ... hem away is 9 - 3 = 6 ...` * **Embeddings Box:** Labels under colored squares: `twice`, `x`, `-`, `divide`, `÷`, `=`, `+`. * **Task Patterns Box:** Phrases in dashed boxes: `twice the age of`, `divide both sides`, `7+2=9 years old`. * **Downstream Task Box:** * `Pattern Wise Context` * `Q₁: ...` * `A₁: (Reasoning Pattern 1)` * `...` * `Question:` * `The value of a sport utility vehicle this year is 16,000 dollars , which is 0.8 of what its value was last year. How much is the value of the vehicle last year?` * **Final Answer Box:** * `The problem tells us that the value of the sport utility vehicle this year is $16,000, which is 0.8 times its value last year. This means that the value last year is $16,000 / 0.8 = $20,000. The answer is $20,000.` ### Key Observations 1. **Pattern-Centric Approach:** The core of the pipeline is the extraction and reuse of "Reasoning Patterns" (e.g., unit conversion, multiplication, subtraction) from solved examples. 2. **Adaptive Clustering:** The system uses `K-Clustering` and `Adaptive K` on problem `Embeddings` to group similar problems, suggesting it dynamically determines the number of relevant patterns to apply. 3. **Hierarchical Pattern Application:** Patterns are discovered from `Prior Knowledge` and `LLM Prompting`, stored as `Task Patterns`, and then injected as `Pattern Wise Context` into the prompt for the downstream task. 4. **Chain-of-Thought Emphasis:** The `Seed Demonstrations` and `Final Answer` explicitly show step-by-step reasoning, highlighted with red text for key operations and numbers. 5. **Visual Flow:** The process is linear for a single problem (left to right) but involves a feedback loop where pattern discovery (center-bottom) informs the context provided to the main solving pipeline. ### Interpretation This diagram represents a sophisticated method for improving an LLM's mathematical reasoning. Instead of relying solely on its parametric knowledge, the system: 1. **Decomposes** problems into reusable reasoning patterns. 2. **Organizes** these patterns via clustering based on semantic embeddings. 3. **Contextualizes** the LLM by providing relevant patterns (`Pattern Wise Context`) alongside the new problem. The **key innovation** is the structured, adaptive retrieval of reasoning strategies. The `Adaptive K` component is particularly notable, as it implies the system can decide how many and which patterns are relevant for a given new problem, moving beyond a static few-shot prompt. The red-highlighted text in the examples serves as a training signal, explicitly marking the mathematical operations and results the model should learn to replicate. The pipeline demonstrates a move towards more **modular and interpretable AI problem-solving**. By isolating "Task Patterns," the system's reasoning steps become more transparent and potentially editable. The final answer is not just a number but a justified solution that mirrors the structure of the seed demonstrations, indicating successful pattern transfer.