## Diagram: Comparison of Chain-of-Thought Prompting vs. Self-Consistency ### Overview This diagram illustrates the difference between two prompting strategies for language models: "Chain-of-thought prompting" and "Self-consistency." The diagram is split horizontally into two sections. The top section demonstrates a single-path "Greedy decode" approach, while the bottom section demonstrates a multi-path "Self-consistency" approach that samples multiple reasoning paths to aggregate a final answer. ### Components/Axes The diagram is organized into two distinct horizontal regions: **1. Top Region (Chain-of-thought prompting):** * **Label:** "Chain-of-thought prompting" (left). * **Flow:** "Prompt" box → "Language model" box → "Greedy decode" box (light red) → "The answer is $14." box (light red). **2. Bottom Region (Self-consistency):** * **Label:** "Self-consistency" (left). * **Flow:** A large "Prompt" box containing few-shot examples → "Language model" box → "Sample a diverse set of reasoning paths" (dashed blue box containing three reasoning paths) → "Marginalize out reasoning paths to aggregate final answers" (blue arrow) → "The answer is $18." (green box). ### Detailed Analysis #### Top Section: Chain-of-thought prompting * **Process:** The model takes a prompt and performs a "Greedy decode." * **Content of "Greedy decode" box:** "This means she uses 3 + 4 = 7 eggs every day. She sells the remainder for $2 per egg, so in total she sells 7 * $2 = $14 per day. The answer is $14." * **Result:** The final output box states: "The answer is $14." #### Bottom Section: Self-consistency * **Input Prompt:** A box containing two examples: * *Example 1:* "Q: If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking lot? A: There are 3 cars in the parking lot already. 2 more arrive. Now there are 3 + 2 = 5 cars. The answer is 5." * *Example 2 (The target problem):* "Q: Janet's ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder for $2 per egg. How much does she make every day? A:" * **Reasoning Paths (inside the dashed blue box):** * **Path 1 (Top, Green box):** "She has 16 - 3 - 4 = 9 eggs left. So she makes $2 * 9 = $18 per day." → Leads to "The answer is $18." * **Path 2 (Middle, Red box):** "This means she she sells the remainder for $2 * (16 - 4 - 3) = $26 per day." → Leads to "The answer is $26." * **Path 3 (Bottom, Green box):** "She eats 3 for breakfast, so she has 16 - 3 = 13 left. Then she bakes muffins, so she has 13 - 4 = 9 eggs left. So she has 9 eggs * $2 = $18." → Leads to "The answer is $18." * **Aggregation:** The three paths are marginalized to aggregate the final answer. * **Final Result:** A green box labeled "The answer is $18." ### Key Observations * **Color Coding:** The diagram uses color to indicate correctness. In the top section, the "Greedy decode" box is red, suggesting an incorrect path. In the bottom section, the two paths resulting in "$18" are green (correct), while the path resulting in "$26" is red (incorrect). * **Logic Error:** The "Chain-of-thought" example (top) incorrectly calculates the number of eggs used (3 + 4 = 7) but fails to subtract them from the total (16) correctly in the context of the problem, leading to an incorrect answer of $14. * **Majority Voting:** The "Self-consistency" method generates three different reasoning paths. Two of these paths arrive at the correct answer ($18), while one arrives at an incorrect answer ($26). By aggregating these paths, the system selects the most frequent answer ($18). ### Interpretation This diagram demonstrates the superiority of the **Self-consistency** prompting strategy over standard **Chain-of-thought** prompting for complex reasoning tasks. * **Chain-of-thought (Greedy Decode):** This method is fragile. It relies on a single, deterministic path. If the language model makes a single logical error during the generation process (as seen in the top section where it miscalculates the remaining eggs), the final answer will be incorrect. * **Self-consistency:** This method introduces redundancy and robustness. By sampling multiple diverse reasoning paths, the model is less likely to be derailed by a single logical slip. The "marginalization" process acts as a form of majority voting, allowing the system to identify the most consistent result among the generated paths, effectively filtering out the "noise" of incorrect reasoning. This is a classic example of how ensemble methods improve performance in machine learning.