## Case Study on the Effects of Question Paraphrasing ### Overview The image presents a side-by-side comparison of two mathematical word problems and their step-by-step solutions. The purpose is to demonstrate how paraphrasing a question can affect the reasoning process and final answer. The left column shows an "Original question" and an incorrect response, while the right column shows a "Paraphrased question" and a correct response. The layout is a two-column table with a dark blue header. ### Components/Axes * **Header:** A dark blue bar at the top with white text: "Case study on the effects of question paraphrasing". * **Layout:** Two vertical columns of equal width. * **Left Column (Original):** * **Label:** "Original question:" (bold) * **Question Text:** A word problem about a card game. * **Ground Truth:** "(ground truth answer: 8)" * **Response Label:** "Response:" (bold) * **Solution Steps:** A two-step solution. Key text in **red** indicates an error. * **Final Answer:** "Therefore, Jane played 4 rounds." followed by a black circle with a white "X" (✗). * **Right Column (Paraphrased):** * **Label:** "Paraphrased question:" (bold) * **Question Text:** A reworded version of the same problem, framed as a bridge competition. * **Ground Truth:** "(ground truth answer: 8)" * **Response Label:** "Response:" (bold) * **Solution Steps:** A three-step solution. Key text in **purple** indicates the correct logical adjustment. * **Final Answer:** "The number of games she won is equal to 8." followed by a black circle with a white checkmark (✓). ### Detailed Analysis **1. Original Question & Incorrect Response:** * **Question:** "In a card game, you are awarded 10 points if you win one round. While playing, Jane ends up with 60 points. If she lost 20 points, then how many rounds did she play? (ground truth answer: 8)" * **Response Steps:** * **Step 1 (Error Highlighted in Red):** "Determine the total points Jane earned **by subtracting the points she lost from her final score. Total points earned = Final score - Points lost. Total points earned = 60 – 20. Total points earned = 40.**" * **Step 2:** "Since Jane earns 10 points for each round she wins, we can find the number of rounds she played by dividing the total points earned by the points earned per round. Number of rounds = Total points earned / Points earned per round. Number of rounds = 40 / 10. Number of rounds = 4." * **Conclusion:** "Therefore, Jane played 4 rounds. ✗" **2. Paraphrased Question & Correct Response:** * **Question:** "At a bridge competition, every game won grants the player 10 points. Unfortunately, Jane started with an advantage but ended up losing 20 points and scoring only 60 points. If she had won x rounds and lost none, what would be the value of x? (ground truth answer: 8)" * **Response Steps:** * **Step 1:** "Understand the given information. Jane started with an advantage but ended up losing 20 points. She scored a total of 60 points. Every game won grants the player 10 points." * **Step 2 (Correct Logic in Purple):** "Determine the total points Jane would have had if she hadn't lost any points. **Since she lost 20 points, we need to add 20 to her total score to find out how many points she would have had if she hadn't lost any points. Total points without loss = 60 + 20 = 80 points.**" * **Step 3:** "Calculate the number of games Jane won. Since every game won grants the player 10 points, we can divide the total points without loss by 10 to find the number of games she won. Number of games won = Total points without loss / 10 = 8. The number of games she won is equal to 8. ✓" ### Key Observations 1. **Identical Core Problem:** Both questions describe the same scenario: a final score of 60 points, a loss of 20 points, and a reward of 10 points per win. The ground truth answer for both is 8. 2. **Critical Reasoning Error:** The response to the original question makes a fundamental algebraic error. It incorrectly interprets "lost 20 points" as a deduction from the final score to find "points earned," rather than understanding it as a penalty applied to a base score from wins. 3. **Paraphrasing Corrects the Error:** The paraphrased question, by explicitly stating "If she had won x rounds and lost none," forces the solver to consider a hypothetical scenario without losses. This framing leads the solver to correctly add the 20 lost points back to the final score (60 + 20 = 80) to find the total points from wins alone. 4. **Visual Cues:** The use of **red text** highlights the erroneous step in the first solution. The use of **purple text** highlights the corrective, logical step in the second solution. The ✗ and ✓ symbols provide immediate visual feedback on the correctness of the final answer. ### Interpretation This case study demonstrates a key principle in problem-solving and AI reasoning: **the phrasing of a question can significantly influence the cognitive path taken to solve it.** * **The Original Question's Flaw:** Its wording ("If she lost 20 points, then how many rounds did she play?") is ambiguous. It can be misinterpreted as asking for the number of rounds *played* (which could include losses), not just rounds *won*. The solver fell into the trap of treating the 20-point loss as a simple subtraction from the final tally, a common but incorrect approach to such problems. * **The Paraphrased Question's Strength:** It removes ambiguity by specifying the goal: find `x`, the number of games won *assuming no losses*. This precise framing guides the solver toward the correct algebraic model: `10x - 20 = 60`, which solves to `x = 8`. The paraphrase acts as a "scaffold," making the underlying mathematical structure more transparent. * **Broader Implication:** For AI systems and human learners alike, this highlights the importance of precise language in prompts and problem statements. A well-constructed paraphrase can prevent logical errors by clarifying intent and constraining the solution space to the correct interpretation. The study suggests that evaluating an AI's reasoning capability should consider not just its final answer, but its robustness to different, yet semantically equivalent, formulations of the same problem.