## Text Comparison: Paraphrastic Probing Case Study ### Overview The image presents a case study on the identification of critical tokens via paraphrastic probing. It shows two attempts to answer a word problem, with slight variations in the wording of the response and steps. Both attempts arrive at the correct answer. ### Components/Axes The image is divided into two columns, each containing: - A "Question" section, which is identical in both columns. - A "Response" section, which contains a step-by-step solution to the question. ### Detailed Analysis or ### Content Details **Left Column:** * **Question:** James decides to run 3 sprints 3 times a week. He runs 60 meters each sprint. How many total meters does he run a week? (ground truth answer: 540) * **Response:** * To find the total meters James runs a week, we need to calculate the total meters he runs in one sprint session and then multiply it by the number of sessions he has in a week. * Step 1: Calculate the total meters James runs in one sprint session. James runs 3 sprints of 60 meters each. Total meters in one sprint session = 3 * 60 = 180 meters. * Step 2: Calculate the total meters... * Therefore, James runs a total of 540 meters a week. * The answer is 540. (followed by a checkmark symbol) **Right Column:** * **Question:** James decides to run 3 sprints 3 times a week. He runs 60 meters each sprint. How many total meters does he run a week? (ground truth answer: 540) * **Response:** * To find the total distances James runs in a week, we need to calculate the total distance he runs in one sprint session and then multiply it by the number of sessions he has in a week. * Step 1: Calculate the total distance James runs in one sprint session. Distance per sprint = 60 meters. Number of sprints per session = 3. Total distance per session = Distance per sprint * Number of sprints per session. Total distance per session = 60 * 3. Total distance per session = 180 meters. * Step 2: Calculate the total distance James runs in a week. * Therefore, James runs a total of 540 meters in a week. (followed by a checkmark symbol) ### Key Observations * The "Question" is identical in both columns. * The "Response" sections differ slightly in wording. The left column uses "total meters," while the right column uses "total distances." * Both solutions arrive at the correct answer of 540 meters. * The right column provides a more detailed breakdown of the calculation in Step 1. ### Interpretation The case study demonstrates how paraphrastic probing can be used to identify critical tokens in a text. By slightly altering the wording of the response (e.g., "total meters" vs. "total distances"), the study aims to assess the impact of these changes on the model's ability to arrive at the correct answer. In this instance, both paraphrased responses lead to the correct solution, suggesting that the model is robust to these minor variations. The more detailed breakdown in the right column's Step 1 might provide additional insight into the model's reasoning process.