## Textual Comparison of Prompting Methods for Math Problem Solving ### Overview The image compares three prompting approaches for solving a math problem: 0-shot, Few-shot Chain-of-Thought (CoT), and Analytical Prompting (Ours). Each method demonstrates how a language model processes a question about calculating the area of a square with specific vertices, using different reasoning strategies. ### Components/Axes 1. **Methods**: - **0-shot**: Direct question without guidance - **Few-shot CoT**: Includes labeled reasoning examples - **Analytical Prompting (Ours)**: Uses automated exemplar generation 2. **Textual Elements**: - Model Input sections (questions and context) - Model Output sections (reasoning and answers) - Annotations (red dots, green checkmark) ### Detailed Analysis #### 0-shot Method - **Model Input**: ``` Q: What is the area of the square with the four vertices at (-2, 2), (-2, -2), (-2, -6), and (-6, -2)? ``` - **Model Output**: ``` Think step by step. ``` - **Annotations**: Red dot labeled "Generic guidance of reasoning" #### Few-shot CoT Method - **Model Input**: ``` Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have? ``` (Irrelevant context) - **Model Output**: ``` A: Roger started with 5 balls. 2 cans of 3 balls each = 6 balls. The answer is 5 + 6 = 11. ``` - **Annotations**: Red dot labeled "Need labeled exemplars of reasoning" #### Analytical Prompting (Ours) - **Model Input**: ``` Q: What is the area of the square with the four vertices at (-2, 2), (-2, -2), (-2, -6), and (-6, -2)? # Instruction: Recall relevant exemplars # Solve the initial problem: ``` - **Model Output**: ``` # Relevant exemplars: Q: What is the area of the square with a side length of 5? A: The area of a square is found by squaring the length of its side. So, the area of this square is 5² = 25. # Solve the initial problem: To find the area of the square, we need to find the side length. The length is ... sqrt((-2 - (-2))² + (-2 - 2)²) = sqrt(32). So, the area of the square is (sqrt(32))² = 32. ``` - **Annotations**: Green checkmark with "Exemplars are automatically generated and tailored to each problem" ### Key Observations 1. **0-shot** provides no reasoning guidance 2. **Few-shot CoT** uses irrelevant exemplars (tennis balls) for a geometry problem 3. **Analytical Prompting** generates contextually relevant exemplars (square area calculation) 4. The correct answer (32) only appears in the Analytical Prompting output 5. Spatial positioning shows progression from generic to specialized reasoning support ### Interpretation The image demonstrates that: 1. **Analytical Prompting** outperforms other methods by: - Generating problem-specific exemplars - Enabling correct mathematical reasoning - Automatically adapting to problem context 2. **Few-shot CoT** fails due to: - Irrelevant exemplar selection - Inability to transfer knowledge across domains 3. **0-shot** approach lacks: - Any reasoning scaffolding - Domain-specific knowledge activation 4. The red/green annotations visually reinforce the effectiveness hierarchy: - Red = limitations in current methods - Green = successful implementation of novel approach This comparison highlights the importance of context-aware exemplar generation in enabling mathematical reasoning capabilities in language models.