## Diagram: Paraphrastic Probing and Consistency Verification ### Overview The image presents a two-phase process for verifying answers to questions, using paraphrasing and consistency checks. Phase I focuses on paraphrasing the original question and generating an initial response, while Phase II verifies the consistency of the answer by considering alternative tokens and trajectories. ### Components/Axes **Phase I: Paraphrastic Probing** * **Step 1:** paraphrase the original question. * Original Question: "A bakery produces 60 loaves of bread each day... How many loaves of bread are sold in the afternoon?" * Paraphrased Question: "In a bustling bakery, daily production meets the demand for 60 freshly baked loaves... What is the number of loaves sold in the afternoon?" * **Step 2:** generate the initial response. * Question: "A bakery produces 60 loaves of bread each day... How many loaves of bread are sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in... Therefore, the number of loaves of bread sold in the afternoon is 5." * **Step 3:** Concatenate the paraphrased question with the initial answer. And obtain the probabilities for top-1 tokens and expected tokens at each position. * Question: "In a bustling bakery, daily production meets the demand for 60 freshly baked loaves... What is the number of loaves sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in... Therefore, the number of loaves of bread sold in the afternoon is 5." * Top-1 tokens: p(To) = 0.89, p(solve) = 0.92, ..., p(the) = 0.71, p(total) = 0.60, ..., p(sold) = 0.85, p(the) = 0.68.... * Expected tokens: p(To) = 0.89, p(solve) = 0.92, ..., p(the) = 0.71, p(number) = 0.25, ..., p(sold) = 0.85, p(in) = 0.22. * **Step 4:** Identify the critical token with the verifier Δ. * Δ(number) = p(total) - p(number) = 0.60 - 0.25 = 0.35. * Δ(in) = p(the) - p(in) = 0.68 - 0.22 = 0.46. 0.46 > 0.35. * The token 'in' is the chosen critical token. **Phase II: Consistency Verification** * **Step 1:** Obtain the candidate tokens at the critical token position. * Question: "A bakery produces 60 loaves of bread each day... How many loaves of bread are sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in." * Candidate tokens: 'during', 'the', ... * **Step 2:** Truncate the initial answer and replace the critical token with the critical tokens. * Process 1: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold during." * Process 2: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold the." * **Step 3:** Generate new trajectories from partial answers to both the original and paraphrased questions, and the same procedure is also applied to the initial answer. * **(Original)** Question: "A bakery produces 60 loaves of bread each day ... How many loaves of bread are sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in." * Process: "To solve this problem... Calculate the number of loaves sold in the morning. The bakery produces 60 loaves of bread each day. Two-thirds of the loaves are sold in the morning... Therefore, the number of loaves of bread sold in the afternoon is 5." * **(Original)** Question: "A bakery produces 60 loaves of bread each day ... How many loaves of bread are sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold the." * Process: "To solve this problem,... Calculate the number of loaves sold the first half of the day (before noon). Two-thirds of 60 loaves are sold before noon.... Therefore, the number of loaves of bread sold in the afternoon is 10." * **(Paraphrased)** Question: "In a bustling bakery, daily production meets the demand for 60 freshly baked loaves... What is the number of loaves sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in." * Process: "To solve this problem... Calculate the number of loaves sold in the morning. The bakery produces 60 loaves, and two-thirds of these loaves are sold in the morning. ...Therefore, the number of loaves of bread sold in the afternoon is 8." * **(Paraphrased)** Question: "In a bustling bakery, daily production meets the demand for 60 freshly baked loaves... What is the number of loaves sold in the afternoon?" * Process: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold the." * Process: "To solve this problem,... Calculate the number of loaves sold the morning after the bakery opens. The bakery starts with 60 loaves. Therefore, the number of loaves of bread sold in the afternoon is 10." * **Step 4:** Determine the final answer with consistency mechanism. * "The answers derived from the second input are more consistent than that from the first input. Thus, the final answer is 10." ### Detailed Analysis or ### Content Details The diagram outlines a process for verifying answers to questions, particularly focusing on mathematical word problems. It uses paraphrasing to generate alternative versions of the question and then checks the consistency of the answers obtained using different approaches. The probabilities of different tokens are used to identify critical tokens and evaluate the consistency of the answers. ### Key Observations * Phase I focuses on generating paraphrases and an initial answer. * Phase II focuses on verifying the consistency of the answer by considering alternative tokens and trajectories. * The probabilities of different tokens are used to identify critical tokens and evaluate the consistency of the answers. * The final answer is determined based on the consistency of the answers derived from different inputs. ### Interpretation The diagram illustrates a method for improving the accuracy of answers to questions by using paraphrasing and consistency checks. This approach is particularly useful for mathematical word problems, where there may be multiple ways to arrive at the correct answer. By considering alternative tokens and trajectories, the system can identify and correct errors in the initial answer. The use of probabilities to identify critical tokens allows the system to focus on the most important parts of the question and answer, improving the efficiency of the verification process. The final answer is determined based on the consistency of the answers derived from different inputs, which helps to ensure that the answer is accurate and reliable.