\n ## Diagram: Paraphrastic Probing & Consistency Verification ### Overview The image presents a diagram illustrating a two-phase process: "Paraphrastic Probing" and "Consistency Verification". The diagram details steps within each phase, demonstrating how a question is paraphrased, an initial answer is generated, and then the consistency of that answer is verified through token manipulation and trajectory generation. The diagram uses a flowchart-like structure with text boxes and arrows to show the flow of information. ### Components/Axes The diagram is divided into two main phases, labeled "Phase I: Paraphrastic Probing" (left side) and "Phase II: Consistency Verification" (right side). Each phase is further broken down into three steps, numbered 1-3. The diagram also includes a "Re-ranking & Selection" section at the bottom. Text boxes contain descriptions of each step, and arrows indicate the flow of the process. Highlighted text within the boxes indicates critical tokens. ### Detailed Analysis or Content Details **Phase I: Paraphrastic Probing** * **Step 1: Paraphrase the original question.** * Original Question: "A bakery produces 60 loaves of bread each day… How many loaves of bread are sold in the afternoon?" * Paraphrased Question: "In a bustling bakery, daily production meets the demand for 60 freshly baked loaves… What is the number of loaves sold in the afternoon?" * Highlighted Token: "during" * **Step 2: Generate the initial response.** * Initial Response: "A bakery produces 60 loaves of bread each day… How many loaves of bread are sold in the afternoon?" * To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in… Therefore, the number of loaves of bread sold in the afternoon is 5." * **Step 3: Concatenate the paraphrased question with the initial answer.** * Concatenated Question/Answer: "In a bustling bakery, daily production meets the demand for 60 freshly baked loaves… What is the number of loaves sold in the afternoon? A bakery produces 60 loaves of bread each day… How many loaves of bread are sold in the afternoon? To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in… Therefore, the number of loaves of bread sold in the afternoon is 5." * Highlighted Tokens: "5", "afternoon" **Phase II: Consistency Verification** * **Step 1: Obtain the candidate tokens at the critical token position.** * Question: "A bakery produces 60 loaves of bread each day… How many loaves of bread are sold in the afternoon?" * Highlighted Tokens: "during", "the" * **Step 2: Truncate the initial answer and replace the critical token with the critical tokens.** * Truncated Answer: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in…" * **Step 3: Generate new trajectories from partial answers to both the original and paraphrased questions, and meet the same procedure as applied to the initial answer.** * Original Question Trajectory: "Original) A bakery produces 60 loaves of bread each day… How many loaves of bread are sold in the afternoon? To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in…" * Paraphrased Question Trajectory: "Paraphrased) In a bustling bakery, daily production meets the demand for 60 freshly baked loaves… What is the number of loaves sold in the afternoon? To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in…" **Re-ranking & Selection (Bottom Section)** * "Re-ranking & Selection: Re-rank the generated trajectories based on their likelihood and select the most consistent answer." * Trajectory 1: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in the morning. The bakery produces 60 loaves of bread each day, two-thirds of the loaves are sold in the morning. Therefore, the number of loaves sold in the afternoon is 5." * Trajectory 2: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in the morning. The bakery produces 60 loaves, and two-thirds of these loaves are sold in the morning… Therefore, the number of loaves of bread sold in the afternoon is 5." * Trajectory 3: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in the morning. The bakery produces 60 loaves. Two-thirds of the loaves are sold in the morning. Therefore, the number of loaves sold in the afternoon is 5." * Trajectory 4: "To solve this problem, we will break it down into steps. Step 1: Calculate the number of loaves sold in the morning. The bakery produces 60 loaves. Therefore, the number of loaves sold in the afternoon is 5." * Final Answer: "Therefore, the number of loaves of bread sold in the afternoon is 5." ### Key Observations The diagram highlights the importance of token manipulation in verifying the consistency of answers generated from both original and paraphrased questions. The repeated answer of "5" suggests this is the expected solution to the problem. The process involves breaking down the problem into steps and using partial answers to assess consistency. ### Interpretation This diagram illustrates a method for ensuring the robustness of question-answering systems. By paraphrasing the original question and then verifying the consistency of the answer through token replacement and trajectory generation, the system can mitigate the effects of subtle variations in wording. The process aims to identify answers that are not only correct but also stable across different phrasings of the same question. The focus on "critical tokens" suggests that certain words or phrases are more important for maintaining consistency than others. The final selection of the most consistent answer indicates a preference for solutions that are less sensitive to paraphrasing. The diagram demonstrates a sophisticated approach to natural language understanding and reasoning, aiming to build more reliable and trustworthy AI systems. The repeated calculation of 2/3 of 60 loaves being sold in the morning, and the subsequent calculation of the remaining loaves being sold in the afternoon, is a key element of the problem-solving process.