2511.09381v1

# Self-Correcting Large Language Models: Generation vs. Multiple Choice > Work was done prior to joining Amazon. ## Abstract Large language models have recently demonstrated remarkable abilities to self-correct their responses through iterative refinement, often referred to as self-consistency or self-reflection. However, the dynamics of this self-correction mechanism may differ substantially depending on whether the model is tasked with open-ended text generation or with selecting the most appropriate response from multiple predefined options. In this paper, we conduct a systematic investigation of these two paradigms by comparing performance trends and error-correction behaviors across various natural language understanding and reasoning tasks, covering language models of different scales and families. Our experimental results reveal distinct patterns of improvement and failure modes: While open-ended generation often benefits from the flexibility of re-interpretation and compositional refinement, multiple-choice selection can leverage clearer solution boundaries but may be limited by the provided options. This contrast also reflects the dual demands faced by emerging agentic LLM applications: effective agents must not only generate and refine open-ended plans or explanations, but also make reliable discrete choices when operating within constrained action spaces. Our findings, therefore, highlight that the design of self-correction mechanisms should take into account the interaction between task structure and output space, with implications for both knowledge-intensive reasoning and decision-oriented applications of LLMs. Codes and experiments are available at https://github.com/rahmanidashti/llm-self-correction Self-Correcting Large Language Models: Generation vs. Multiple Choice Hossein A. Rahmani $\dagger$ , Satyapriya Krishna $\ddagger$ thanks: Work was done prior to joining Amazon., Xi Wang $\nabla$ , Mohammadmehdi Naghiaei $\diamondsuit$ , Emine Yilmaz $\dagger$ $\dagger$ University College London, $\ddagger$ Amazon AGI, $\nabla$ University of Sheffield, $\diamondsuit$ University of Southern California {hossein.rahmani.22, emine.yilmaz}@ucl.ac.uk, skrishna@g.harvard.edu xi.wang@sheffield.ac.uk, naghiaei@usc.edu ## 1 Introduction Recent advances in Large Language Models (LLMs) have illustrated that iterative self-correction, where a model re-examines and revises its output under a self-reflection framework, can lead to significant performance gains across a variety of tasks (Madaan et al., 2023; Cook et al., 2024; Shinn et al., 2023; Gou et al., 2024, inter alia). This emergent ability is often attributed to the models’ capacity to integrate chain-of-thought reasoning Kamoi et al. (2024); Chang et al. (2024); Wei et al. (2022), prompting them to refine their own outputs as addressed by a human proofreader or mentor. Regarding performance validation, existing studies on self-correction have generally focused on free-form text generation (Huang et al., 2023; Madaan et al., 2023; Zelikman et al., 2022; Ma et al., 2025; Kumar et al., 2025; Krishna et al., 2024, inter alia), such as dialogue response, code optimization, and acronym generation. These tasks align with the strategy of language model optimization in addressing next token prediction. However, as LLM applications expand, evaluation restricted to free-form generation offers an incomplete picture. For instance, NVIDIA advocates the deployment of smaller language models in agentic systems for tasks such as API calls and orchestration with external tools, motivated by sustainability and efficiency considerations Belcak et al. (2025). This highlights the need to examine self-correction beyond open-ended generation. In this study, we categorize natural language modeling tasks into two broad paradigms: free-form text generation and multi-choice prediction. The former treats modeling as unconstrained sequence generation over the full vocabulary, while the latter frames it as classification over a fixed set of candidate answers. These paradigms are complementary: multi-choice tasks test precise discrimination under constraints, whereas free-form tasks assess expressive generation, and together they capture the main modes of LLM use in applications such as question answering, reasoning, and open-ended dialogue. In this paper, we investigate how self-correction unfolds when comparing open-ended generation against multiple-choice question scenarios. We hypothesize that while open-ended generation may benefit from enhanced flexibility and creativity, it also faces a larger search space and the risk of compounding errors. By contrast, multiple-choice models operate in a constrained space, which can reduce semantic drift yet limit creative corrections. Our study explores how these respective factors interact with iterative refinement, shedding light on whether self-correction aligns more naturally with either unconstrained or constrained output space. To address these questions, we conduct comprehensive experiments on two distinct datasets that differ in nature, one focusing on knowledge-intensive question answering and the other on reasoning-oriented problems. We perform iterative inference, giving the model multiple opportunities to reevaluate and revise. By comparing error rates, consistency across iterations, and eventual convergence in each paradigm, we expose nuanced trade-offs in how LLMs adapt to different output constraints under a self-correction regime. Our results provide practical insights for the design and deployment of LLM-based systems, highlighting opportunities to harness better or tailor self-correction behaviors for diverse application settings. Furthermore, we discuss how our findings inform the broader research agenda of aligning emergent capabilities in large-scale models with varied real-world task requirements. ## 2 Related Works Iterative Reasoning and Self-correction in LLMs. Large language models first showed an emergent ability to reason step-by-step when prompted with chain-of-thought (CoT) examples (Wei et al., 2022). Shortly after, Wang et al. (2023) demonstrated that sampling several independent reasoning traces and selecting the majority answer—dubbed self-consistency (SC)—boosts accuracy on arithmetic and commonsense tasks. Follow-up studies made the correction loop explicit by asking the model to critique its own draft before rewriting it, leading to sizeable gains in factual QA and code generation (Madaan et al., 2023). Variants that call external tools such as Python or knowledge bases during the critique stage further reduce hallucinations in open-ended generation (Chen et al., 2023; Yao et al., 2023; Gou et al., 2024). These works collectively suggest that LLMs can act as both solver and reviewer, but they focus almost exclusively on free-form text outputs. Verification–based Refinement. Instead of trusting the model’s final token distribution, several papers add lightweight verifiers. Cobbe et al. (2021) attach unit tests to code synthesis; Dixit et al. (2023) use factuality checkers for summarization; Pryzant (2023) adopt entailment models for reading comprehension. The common pattern is a two-step pipeline where the LLM proposes an answer, then a cheaper or more precise module scores it. Our work keeps the entire loop inside the language model, isolating the effect of output format itself (generation vs. multiple-choice) from external verification. Answer Selection and Multiple-Choice Prompting. Tasks with a closed candidate set (e.g., MMLU (Hendrycks et al., 2021), ARC (Clark and et al., 2018)) are typically solved by mapping each option to an independent prompt and picking the highest-logit answer (Brown and et al., 2020). Several groups have tried to retrofit iterative reasoning onto this template. Zhu and et al. (2024) prepend a self-explanation, rescore the options with the explanation as additional context, and report modest but consistent gains. Li and et al. (2024) show that calibrating logits with contrastive rationales helps low-parameter models, while Pan and et al. (2023) explore ensembling diverse rationales. Yet a systematic comparison between correction dynamics in open versus closed output spaces is missing; our study provides that head-to-head analysis. Bridging the paradigms. Contemporary benchmarks increasingly mix free-form and categorical sub-tasks—e.g., TruthfulQA has both short-answer and multiple-choice splits (Lin et al., 2022). Deployment settings such as tutoring agents or search assistants likewise alternate between generating explanations and selecting the best passages. Understanding whether self-correction behaves differently under these two regimes is therefore more than a methodological curiosity as it affects prompt engineering, compute budgeting, and safety guard-rail design. By re-implementing the main correction strategies from the literature under a unified experimental budget, we show that the shape of the output space itself controls how much an LLM can benefit from extra reflection rounds. ## 3 Open-ended Generation vs. Multiple-Choice Answer Selection Large language models are increasingly expected to handle a wide spectrum of downstream tasks, ranging from unconstrained natural language generation, such as open-domain question answering, to highly structured classification problems, like sentiment analysis. Two of the most commonly encountered settings are (i) open-ended generation, where the model must produce a free-form text response, and (ii) multiple-choice answer selection, where it must select a single correct option from a predefined set of choices. While these two paradigms are often operationalized using the same model architecture and weights, they impose fundamentally different constraints on the output space and influence how self-correction unfolds over successive inference steps. This section formalizes these two paradigms, describes how self-correction mechanisms are instantiated within each, and presents qualitative differences that help explain the empirical patterns observed in Section 5. Open-Ended Generation. In the open-ended generation setting, the model is required to produce an output sequence $y^{(0)}=(y^{(0)}_{1},\ldots,y^{(0)}_{T})\in\mathcal{V}^{*}$ , where $\mathcal{V}$ denotes the vocabulary and $T$ is the (variable) sequence length. The generation is conditioned on an input $x$ , which may correspond to a question, prompt, or instruction, such that the model defines a conditional distribution: $p(y^{(0)}\mid x)=\prod_{t=1}^{T}p(y^{(0)}_{t}\mid y^{(0)}_{<t},x)$ This formulation captures the standard auto-regressive decoding process for open-ended text generation. The generated sequence may consist of a sentence, paragraph, or longer passage, and there are no explicit structural constraints beyond syntactic plausibility and task relevance. Self-correction in this paradigm typically proceeds by prompting the model to critique its initial output — either via explicit instructions (“identify any flaws”) or implicit prompting strategies (“think step by step”) — followed by a new generation $y^{(1)}$ . This iterative process can be repeated multiple times, resulting in a sequence $\{y^{(k)}\}_{k=0}^{K}$ , where each revised answer aims to improve upon the previous one. A final answer can be selected using majority voting, log-probability re-ranking, or verifier-based scoring. Because generation is unconstrained, each iteration can introduce new content, restructure previous arguments, or expand omitted details. While this offers flexibility and the potential for substantial improvements, it also opens the door to risks such as semantic drift Ji et al. (2023b, a), where the answer becomes misaligned with the original question over time, or hallucinations, where fictitious facts are introduced in an attempt to improve fluency or apparent coherence. These failure modes tend to accumulate if the model “over-corrects” by deviating from the initial context Spataru (2024). Multiple-Choice Answer Selection. By contrast, the multi-choice setting restricts the output space to a finite set of candidate answers $A=\{a_{1},a_{2},\ldots,a_{M}\}$ . For each question $x$ , the model computes a logit vector $\ell(x)\in\mathbb{R}^{M}$ , from which a softmax distribution is derived, and selects the most probable answer. Self-correction in this paradigm does not involve rewriting text but rather involves revisiting the initial logits after incorporating additional information. One common strategy is to generate a rationale $r^{(t)}$ for why a particular answer is correct, then concatenate this rationale to the original prompt and recompute the logits to obtain $\ell^{(t+1)}(x,r^{(t)})$ Huang et al. (2023); Liu et al. (2024). Over successive iterations, this allows the model to refine its beliefs based on its own reasoning. However, since the answer set is fixed, the model cannot explore novel hypotheses or restructure the space of answers; instead, it can only shift probability mass among existing options. This bounded nature of the output space makes multiple-choice settings more stable and less prone to semantic drift, but also potentially less effective at recovering from early errors — especially if the correct answer has low initial probability and the generated rationales fail to meaningfully influence the logits. Qualitative Differences. The two paradigms, i.e., open-ended generation and multiple-choice selection, exhibit distinct self-correction dynamics due to their differing output constraints. In open-ended generation, performance gains are typically front-loaded, with the most significant improvements occurring in the first few iterations as the model repairs inconsistencies or fills in missing details Cook et al. (2024); Huang et al. (2023); Gou et al. (2024). However, this flexibility also increases the risk of semantic drift in later rounds Spataru (2024): if the model’s revisions start to go off-topic or introduce inaccuracies, the session can degrade without external intervention. In contrast, multiple-choice tasks show steadier, more incremental improvements, benefiting from the stability of a fixed answer set. They may suffer, however, from logit inertia when the correct option is initially underweighted. The model can be difficult to move to a low-probability answer unless a very compelling rationale shifts the balance. Generation tends to be more compute-intensive due to longer outputs per iteration, while multiple-choice achieves better accuracy-to-token efficiency by focusing on short discriminative outputs. Additionally, model scale interacts differently across formats. Larger models can better mitigate drift in generation through coherent reasoning chains, while smaller models perform more reliably in multiple-choice settings due to the structured nature of the output space and the guidance provided by explicit options. Understanding these qualitative and quantitative differences between the two paradigms is crucial for designing robust systems that use LLMs in iterative inference settings. Depending on the task requirements, whether correctness, stability, creativity, or inference budget is the primary constraint, one or the other format may be more appropriate, and self-correction strategies should be tailored accordingly. ## 4 Experimental Setup Problem Statement. In this study, we aim to evaluate the dynamics of iterative self-correction under constrained generation and multiple-choice selection across representative tasks. Let $x\in\mathcal{X}$ denote an input instance (e.g., a question) with ground-truth answer $y^{\star}$ . An LLM parameterised by $\theta$ produces an initial response $y^{(0)}$ whose format depends on the task paradigm. For open-ended generation, the model outputs a sequence $y^{(0)}\in V^{\ast}$ with $p_{\theta}\!\big(y^{(0)}\mid x\big)\;=\;\prod_{t=1}^{T}p_{\theta}\!\big(y^{(0)}t\mid y^{(0)}{<t},x\big)$ . In contrast, for multiple-choice selection, the model selects $y^{(0)}\in A=\{a_{1},\dots,a_{M}\}$ from logits $\ell(x)\in\mathbb{R}^{M}$ , i.e., $y^{(0)}\;=\;\arg\max_{a_{i}\in A}\ell_{i}(x),\qquad$ $\sigma_{i}^{(0)}(x)\;=\;\frac{e^{\ell_{i}(x)}}{\sum_{j=1}^{M}e^{\ell_{j}(x)}}$ . By applying iterative self-correct, given history $\mathcal{H}^{(k-1)}=(x,y^{(0)},\dots,y^{(k-1)})$ , the model produces a revision $y^{(k)}\sim p_{\theta}\!\big(\cdot\mid\mathcal{H}^{(k-1)}\big),\qquad k=1,\dots,K$ . We study the sequence $\mathcal{Y}(x)=\{y^{(k)}\}_{k=0}^{K}$ and aim to maximize task accuracy of the terminal output $y^{(K)}$ over $x\sim\mathcal{D}$ . We seek to observe how performance evolves with successive self-correction iterations and how error correction or degradation manifests in each paradigm. To that end, we set up experiments on two distinct question-answering benchmarks and examine multiple LLMs under various prompting strategies. <details> <summary>x1.png Details</summary>  ### Visual Description ## Line Graph: Accuracy Comparison: Generation vs Multiple-choice Methods ### Overview The image is a line graph comparing the accuracy of two methods ("Generation" and "Multiple-choice") across five iterations. Accuracy is measured on the y-axis (0% to 100%), and iterations are on the x-axis (0 to 5). Shaded regions around the lines represent confidence intervals. ### Components/Axes - **X-axis (Horizontal)**: Labeled "Iteration" with markers at 0, 1, 2, 3, 4, 5. - **Y-axis (Vertical)**: Labeled "Accuracy (%)" with markers at 0.0, 0.2, 0.4, 0.6, 0.8, 1.0. - **Legend**: Located in the top-right corner. - **Blue line with circles**: "Generation" - **Orange line with circles**: "Multiple-choice" - **Shaded Regions**: Gray for "Generation," light orange for "Multiple-choice," indicating variability/confidence intervals. ### Detailed Analysis 1. **Generation (Blue Line)**: - Starts at ~0.2% accuracy at iteration 0. - Increases gradually to ~0.35% by iteration 5. - Shaded region widens early (iterations 0–2) and narrows slightly later, suggesting higher variability in early iterations. 2. **Multiple-choice (Orange Line)**: - Starts at ~0.4% accuracy at iteration 0. - Rises steadily to ~0.55% by iteration 5. - Shaded region remains consistently narrow, indicating stable performance. 3. **Trends**: - Both methods show upward trends, but "Multiple-choice" maintains higher accuracy throughout. - The gap between the two methods narrows slightly from iteration 0 (0.2% difference) to iteration 5 (~0.2% difference), though "Multiple-choice" remains superior. ### Key Observations - "Multiple-choice" consistently outperforms "Generation" across all iterations. - "Generation" shows a steeper improvement rate but starts from a lower baseline. - Confidence intervals for "Generation" are broader, especially in early iterations, indicating less reliability. ### Interpretation The data suggests that "Multiple-choice" methods are more accurate and reliable than "Generation" methods in this context. While "Generation" demonstrates potential for improvement with more iterations, its current performance is less consistent. The narrowing gap implies that "Generation" could approach "Multiple-choice" performance with further refinement, but the shaded regions highlight persistent uncertainty in its results. This could inform decisions about prioritizing "Multiple-choice" for critical applications or investing in optimizing "Generation" methods. </details> (a) Baseline <details> <summary>x2.png Details</summary>  ### Visual Description ## Line Chart: Accuracy Comparison Across Iterations ### Overview The chart compares the accuracy trends of two methods ("Generation" and "Multiple-choice") across 5 iterations. Accuracy is measured on a percentage scale (0-100%), with shaded regions indicating confidence intervals or error margins. Both methods show improvement over iterations, but "Multiple-choice" consistently outperforms "Generation". ### Components/Axes - **X-axis (Horizontal)**: Labeled "Iteration", with markers at 0, 1, 2, 3, 4, 5. - **Y-axis (Vertical)**: Labeled "Accuracy (%)", scaled from 0.0 to 1.0 (0% to 100%). - **Legend**: Located in the top-right corner, with: - **Blue line/dots**: "Generation" - **Orange line/dots**: "Multiple-choice" - **Shaded Regions**: Gray bands around each line represent uncertainty/error margins. ### Detailed Analysis 1. **Generation (Blue)**: - **Iteration 0**: ~0.2% accuracy. - **Iteration 1**: ~0.25%. - **Iteration 2**: ~0.3%. - **Iteration 3**: ~0.32%. - **Iteration 4**: ~0.33%. - **Iteration 5**: ~0.35%. - **Trend**: Gradual upward slope, plateauing after iteration 3. Shaded region narrows slightly over time, indicating reduced uncertainty. 2. **Multiple-choice (Orange)**: - **Iteration 0**: ~0.35% accuracy. - **Iteration 1**: ~0.4%. - **Iteration 2**: ~0.45%. - **Iteration 3**: ~0.47%. - **Iteration 4**: ~0.48%. - **Iteration 5**: ~0.5%. - **Trend**: Steady upward trajectory. Shaded region remains wider than "Generation", suggesting higher variability. ### Key Observations - **Performance Gap**: "Multiple-choice" maintains ~0.1–0.15% higher accuracy than "Generation" across all iterations. - **Uncertainty**: The orange shaded region (Multiple-choice) is consistently broader, implying less confidence in its measurements. - **Convergence**: Both methods show diminishing returns after iteration 3, with accuracy gains slowing. ### Interpretation The data suggests that "Multiple-choice" methods yield higher accuracy but with greater variability, while "Generation" offers more stable (narrower confidence intervals) but lower performance. The widening gap between the two methods over iterations implies that "Multiple-choice" may scale better for complex tasks, though its reliability is less certain. The plateauing trends indicate potential limits to improvement for both approaches beyond 5 iterations. </details> (b) CoT <details> <summary>x3.png Details</summary>  ### Visual Description ## Line Graph: Accuracy Comparison Across Iterations ### Overview The image depicts a line graph comparing the accuracy of two methods ("Generation" and "Multiple-choice") over five iterations. The y-axis represents accuracy as a percentage (0–100%), and the x-axis represents iterations (0–5). Both data series show upward trends, with "Multiple-choice" consistently outperforming "Generation" across all iterations. ### Components/Axes - **X-axis (Horizontal)**: Labeled "Iteration" with markers at 0, 1, 2, 3, 4, 5. - **Y-axis (Vertical)**: Labeled "Accuracy (%)" with markers at 0.0, 0.2, 0.4, 0.6, 0.8, 1.0. - **Legend**: Located in the top-right corner, with: - **Blue line**: "Generation" (solid line with shaded uncertainty band). - **Orange line**: "Multiple-choice" (solid line with shaded uncertainty band). ### Detailed Analysis 1. **Generation (Blue Line)**: - Starts at ~0.25% accuracy at iteration 0. - Increases gradually to ~0.35% by iteration 5. - Uncertainty band (shaded blue) widens slightly over iterations, indicating increasing variability. 2. **Multiple-choice (Orange Line)**: - Starts at ~0.35% accuracy at iteration 0. - Increases steadily to ~0.5% by iteration 5. - Uncertainty band (shaded orange) remains relatively narrow, suggesting stable performance. 3. **Trends**: - Both lines show positive slopes, but "Multiple-choice" maintains a higher accuracy throughout. - The gap between the two lines widens slightly from iteration 0 to 5 (~0.1% difference at start vs. ~0.15% at end). ### Key Observations - "Multiple-choice" outperforms "Generation" by ~10–15% across all iterations. - Uncertainty bands suggest "Generation" has higher variability in results compared to "Multiple-choice." - No data points fall outside the shaded uncertainty regions. ### Interpretation The graph demonstrates that the "Multiple-choice" method consistently achieves higher accuracy than "Generation" as iterations increase. The widening gap suggests that "Multiple-choice" may be more robust or better suited for the task being measured. The uncertainty bands highlight that "Generation" results are less reliable, potentially due to higher sensitivity to input variations or model instability. This could imply that "Multiple-choice" is a preferable approach for applications requiring consistent performance, while "Generation" might require further optimization or regularization to reduce variability. </details> (c) SC <details> <summary>x4.png Details</summary>  ### Visual Description ## Line Graph: Accuracy Comparison Across Iterations ### Overview The image depicts a line graph comparing the accuracy of two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents accuracy as a percentage (0–100%), while the x-axis represents iterations (0–5). Shaded regions around each line indicate variability or confidence intervals. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with markers at 0, 1, 2, 3, 4, and 5. - **Y-axis (Accuracy %)**: Labeled "Accuracy (%)" with increments from 0.0 to 1.0. - **Legend**: Located at the bottom-left corner, with: - **Blue line**: "Generation" - **Orange line**: "Multiple-choice" - **Shaded Regions**: Gray for "Generation" and orange for "Multiple-choice," representing variability. ### Detailed Analysis 1. **Generation (Blue Line)**: - **Trend**: Slopes upward steadily from iteration 0 to 5. - **Data Points**: - Iteration 0: ~0.75% - Iteration 1: ~0.80% - Iteration 2: ~0.82% - Iteration 3: ~0.83% - Iteration 4: ~0.85% - **Variability**: Shaded region widens slightly between iterations 0–2, then narrows. 2. **Multiple-choice (Orange Line)**: - **Trend**: Slopes upward gradually but less steeply than "Generation." - **Data Points**: - Iteration 0: ~0.60% - Iteration 1: ~0.65% - Iteration 2: ~0.68% - Iteration 3: ~0.69% - Iteration 4: ~0.70% - **Variability**: Shaded region remains relatively consistent in width. ### Key Observations - **Accuracy Trends**: Both methods improve over iterations, but "Generation" consistently outperforms "Multiple-choice." - **Variability**: "Generation" shows higher variability (wider shaded regions), especially early in the iterations. - **Convergence**: By iteration 5, "Generation" reaches ~0.85% accuracy, while "Multiple-choice" plateaus near ~0.70%. ### Interpretation The data suggests that the "Generation" method demonstrates superior accuracy growth over iterations compared to "Multiple-choice." However, the wider confidence intervals for "Generation" imply greater uncertainty in its performance, potentially due to dynamic adjustments or stochastic elements in the method. The "Multiple-choice" method appears more stable but less effective, possibly due to fixed parameters or limited adaptability. These trends could reflect trade-offs between flexibility and reliability in the evaluated systems. </details> (d) Baseline <details> <summary>x5.png Details</summary>  ### Visual Description ## Line Graph: Accuracy Comparison Across Iterations ### Overview The image depicts a line graph comparing the accuracy of two methods—**Generation** (blue line) and **Multiple-choice** (orange line)—across five iterations (0 to 5). Both lines show upward trends, with shaded regions indicating variability or confidence intervals. The y-axis represents accuracy as a percentage (0–100%), while the x-axis denotes iteration steps. --- ### Components/Axes - **X-axis (Horizontal)**: Labeled "Iteration," with markers at 0, 1, 2, 3, 4, and 5. - **Y-axis (Vertical)**: Labeled "Accuracy (%)," scaled from 0.0 to 1.0 (equivalent to 0%–100%). - **Legend**: Located in the **top-left** corner, with: - **Blue line**: Labeled "Generation." - **Orange line**: Labeled "Multiple-choice." - **Shaded Regions**: Light blue (Generation) and light orange (Multiple-choice) bands around each line, likely representing confidence intervals or error margins. --- ### Detailed Analysis 1. **Generation (Blue Line)**: - **Trend**: Steadily increases from ~0.75 (75%) at iteration 0 to ~0.85 (85%) at iteration 5. - **Data Points**: - Iteration 0: ~0.75 (±0.05, based on shaded region). - Iteration 1: ~0.80 (±0.04). - Iteration 2: ~0.83 (±0.03). - Iteration 3: ~0.84 (±0.03). - Iteration 4: ~0.85 (±0.02). - Iteration 5: ~0.85 (±0.02). - **Shaded Region**: Narrows slightly over iterations, suggesting reduced uncertainty. 2. **Multiple-choice (Orange Line)**: - **Trend**: Gradual increase from ~0.55 (55%) at iteration 0 to ~0.65 (65%) at iteration 5. - **Data Points**: - Iteration 0: ~0.55 (±0.06). - Iteration 1: ~0.60 (±0.05). - Iteration 2: ~0.62 (±0.04). - Iteration 3: ~0.63 (±0.04). - Iteration 4: ~0.64 (±0.03). - Iteration 5: ~0.65 (±0.03). - **Shaded Region**: Remains relatively wide, indicating higher variability. --- ### Key Observations 1. **Performance Gap**: The Generation method consistently outperforms Multiple-choice by ~10–15 percentage points across all iterations. 2. **Convergence**: Both methods plateau near iteration 4–5, with Generation stabilizing at ~85% and Multiple-choice at ~65%. 3. **Uncertainty**: The shaded regions for Generation narrow over time, while Multiple-choice’s variability remains relatively constant. --- ### Interpretation - **Method Efficacy**: The Generation method demonstrates superior accuracy, likely due to iterative refinement or adaptive learning mechanisms. Its steeper slope suggests faster improvement compared to Multiple-choice. - **Stability**: The plateau in both methods implies diminishing returns after iteration 4, indicating potential limits to their effectiveness. - **Confidence Intervals**: The narrowing shaded region for Generation implies increasing confidence in its results, whereas Multiple-choice’s wider bands suggest less reliability. - **Practical Implications**: Generation may be preferable for tasks requiring high accuracy, while Multiple-choice could serve as a baseline or fallback. Further investigation into why Multiple-choice plateaus could reveal constraints in its design. --- ### Spatial Grounding & Verification - **Legend**: Top-left, clearly associating colors with labels. - **Color Consistency**: Blue (Generation) and orange (Multiple-choice) match legend entries exactly. - **Axis Alignment**: X-axis (iterations) and Y-axis (accuracy) are orthogonal, with no overlapping labels. --- ### Content Details - **No Text Blocks**: The image contains no standalone text blocks, tables, or embedded diagrams. - **Visual Hierarchy**: Lines and shaded regions are the primary focus, with minimal gridlines or annotations. --- ### Final Notes The graph highlights the iterative advantage of the Generation method, emphasizing its robustness and adaptability. The Multiple-choice method’s performance suggests it may rely on static heuristics or limited contextual understanding. Further analysis of the underlying algorithms could clarify these trends. </details> (e) CoT <details> <summary>x6.png Details</summary>  ### Visual Description ## Line Chart: Accuracy Comparison Across Iterations ### Overview The image is a line chart comparing the accuracy of two methods ("Generation" and "Multiple-choice") over five iterations. Accuracy is measured on the y-axis (0% to 100%), and iterations are labeled on the x-axis (0 to 5). The chart includes shaded confidence intervals around each line, indicating variability in measurements. ### Components/Axes - **Y-axis**: "Accuracy (%)" with ticks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. - **X-axis**: "Iteration" with ticks at 0, 1, 2, 3, 4, and 5. - **Legend**: Located at the bottom-left corner. Blue dots represent "Generation," and orange dots represent "Multiple-choice." - **Shaded Areas**: Gray bands around each line indicate confidence intervals (e.g., ±1 standard deviation). ### Detailed Analysis 1. **Generation (Blue Line)**: - **Trend**: Starts at ~0.75% accuracy at iteration 0, increases to ~0.85% by iteration 1, then plateaus between ~0.85% and ~0.87% for iterations 2–5. - **Data Points**: - Iteration 0: ~0.75% - Iteration 1: ~0.85% - Iterations 2–5: ~0.85–0.87% - **Confidence Interval**: Narrower shaded band (~±0.05% range). 2. **Multiple-choice (Orange Line)**: - **Trend**: Begins at ~0.60% accuracy at iteration 0, rises to ~0.65% by iteration 1, then stabilizes between ~0.65% and ~0.67% for iterations 2–5. - **Data Points**: - Iteration 0: ~0.60% - Iteration 1: ~0.65% - Iterations 2–5: ~0.65–0.67% - **Confidence Interval**: Wider shaded band (~±0.10% range). ### Key Observations - **Performance Gap**: "Generation" consistently outperforms "Multiple-choice" by ~0.20–0.25% across all iterations. - **Plateau Effect**: Both methods show minimal improvement after iteration 1, suggesting diminishing returns. - **Confidence Variability**: "Multiple-choice" exhibits greater uncertainty (wider shaded area) compared to "Generation." ### Interpretation The data suggests that the "Generation" method is more accurate and reliable than "Multiple-choice" for the measured task. The plateau in accuracy after iteration 1 implies that further iterations yield negligible improvements, highlighting potential inefficiencies in extended training or testing. The narrower confidence intervals for "Generation" reinforce its stability, while the wider intervals for "Multiple-choice" indicate higher variability in performance. This could reflect differences in algorithmic robustness or data handling between the two approaches. </details> (f) SC Figure 1: Average cumulative accuracy on generation and multiple-choice. (Top) Accuracy on the DisambiguationQA dataset shows that models perform better on the multiple-choice task when we iteratively self-correct the model response to the questions, while (bottom) shows the accuracy on the tinyTruthfulQA dataset, indicating that models perform better in generation tasks. Research Questions. Our study is guided by the following three research questions: - RQ1: How do self-correction dynamics differ between open-ended and multiple-choice tasks? - RQ2: How do model scale and prompting strategy influence self-correction across the two paradigms? - RQ3: How does iterative self-correction affect correctness, stability, and semantic drift, and what mechanisms explain these effects? Datasets. We evaluate on two benchmarks, DisambiguationQA and tinyTruthfulQA, that each provide parallel formulations for both multiple-choice questions and open-ended generation. This allows us to study self-correction dynamics under consistent task content but different output constraints. - DisambiguationQA Kazemi et al. (2025) is typically phrased in multiple-choice form, where each question presents a pronoun or reference with referential ambiguity and provides four candidate referents. However, the same questions can also be cast into an open-ended format by asking models to generate the referent rather than choose among options. Thus, DisambiguationQA instantiates a scenario where the answer space is tightly constrained but also amenable to open-ended generation in a parallel setup. - tinyTruthfulQA Polo et al. (2024) is a challenging subset of the TruthfulQA benchmark Lin et al. (2022) focused on short-form factual queries that tend to provoke false or misleading answers from LLMs. While TruthfulQA is usually evaluated via free-form generation, where models must produce a truthful answer, a multiple-choice variant has also been developed, offering for each question a small set of candidate answers drawn from the same reference answer pool. Therefore, tinyTruthfulQA inherits this dual-format nature, where the same questions support both open-ended and multiple-choice instantiations. This dataset exemplifies scenarios requiring knowledge retrieval and precision in generation. By evaluating both tasks, we cover one case where the ground-truth answer is within a closed set of options and one case where the answer must be generated. We therefore can compare how iterative self-correction dynamics differ when the model’s output is tightly constrained versus freely generative. Models. We evaluate the dynamics of iterative self‐correction under unconstrained generation and multiple‐choice selection using six pre‐trained language models ranging from small to large parameters. We evaluate SmolLM2-1.7B Allal et al. (2025), Qwen2.5-3B Qwen et al. (2025), Llama-3.1-8B Grattafiori et al. (2024), Qwen2.5-14B Qwen et al. (2025), DeepSeek-R1-Distill-Llama-8B Guo et al. (2025), and Gemini-2.0-Flash Comanici et al. (2025). These models represent diverse families and scales (from distilled smaller models to state-of-the-art large models). For each model and dataset, we compare three aforementioned prompting strategies: a direct Baseline prompt, zero‐shot chain‐of‐thought (CoT) prompting Kojima et al. (2022), and our iterative SC procedure that reviews and refines the model’s own previous response for up to five rounds. We use HuggingFace to run the models except Gemini-2.0-Flash, which is accessed through the API. Prompts. In our experiments, we use simplified prompts to minimize the impact of prompt design on performance across tasks, keeping the focus on the self-correction mechanism Huang et al. (2023). Specifically, we apply a basic prompt for the Baseline method and adopt zero-shot Chain-of-Thought (CoT) prompting Kojima et al. (2022) for both the CoT and Self-Consistency (SC) approaches. The initial prompts are used for the first attempt (iteration 0) under each strategy. They differ only in whether the model is encouraged to produce an explicit chain of reasoning before the final answer. For iterations beyond the first, we prepend instructions to review the prior attempts. In both cases, the model is reminded of its earlier answers (which are included in the conversation context) and encouraged to refine them. The CoT variant additionally maintains the directive to use a step-by-step reasoning process during revision. Our full prompts can be found in Appendix A.2. Final Answer Extraction. For all of our problems, we added the ‘ The final answer is: ’ suffix to the text of the prompt to encourage the model to produce the final answer in a format that we can easily extract. More details in Appendix A.1. ## 5 Results <details> <summary>x7.png Details</summary>  ### Visual Description ## Line Graph: Average Correct Flips vs. Iteration ### Overview The image is a line graph comparing two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Correct Flips" (0.000 to 0.100), and the x-axis represents "Iteration" (1 to 5). Shaded regions around each line indicate variability or confidence intervals. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with ticks at 1, 2, 3, 4, 5. - **Y-axis (Average Correct Flips)**: Labeled "Average Correct Flips" with ticks at 0.000, 0.025, 0.050, 0.075, 0.100. - **Legend**: Located in the top-right corner. - **Blue line**: "Generation" - **Orange line**: "Multiple-choice" - **Shaded Regions**: Gray for "Generation" and beige for "Multiple-choice," centered on their respective lines. ### Detailed Analysis 1. **Generation (Blue Line)**: - Iteration 1: ~0.075 - Iteration 2: ~0.050 - Iteration 3: ~0.040 - Iteration 4: ~0.030 - Iteration 5: ~0.025 - Trend: Steady decline with minimal fluctuation. - Shaded region: Narrower at later iterations, indicating reduced variability. 2. **Multiple-choice (Orange Line)**: - Iteration 1: ~0.075 - Iteration 2: ~0.080 (peak) - Iteration 3: ~0.040 - Iteration 4: ~0.040 - Iteration 5: ~0.025 - Trend: Initial increase at iteration 2, followed by a sharp decline and stabilization. - Shaded region: Broader at iteration 2, suggesting higher variability. ### Key Observations - Both methods show a general decline in "Average Correct Flips" over iterations. - "Multiple-choice" exhibits higher variability (wider shaded regions) compared to "Generation." - "Generation" maintains a consistent downward trend, while "Multiple-choice" has a notable spike at iteration 2. - Both lines converge at iteration 5 (~0.025), but "Generation" retains a narrower confidence interval. ### Interpretation The data suggests that both "Generation" and "Multiple-choice" methods improve over time, but "Generation" demonstrates more stability and consistency. The spike in "Multiple-choice" at iteration 2 may indicate an outlier or a temporary performance boost, followed by a correction. The narrowing shaded regions for "Generation" imply that its results are more reliable as iterations progress. The convergence at iteration 5 could signal that both methods reach similar performance levels, but "Generation" achieves this with greater precision. </details> (a) Baseline <details> <summary>x8.png Details</summary>  ### Visual Description ## Line Graph: Average Correct Flips vs Iteration ### Overview The image is a line graph comparing the performance of two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Correct Flips" (ranging from 0.000 to 0.100), while the x-axis shows "Iteration" (1 to 5). Shaded regions around the lines indicate confidence intervals. ### Components/Axes - **Y-axis**: "Average Correct Flips" (0.000 to 0.100 in increments of 0.025). - **X-axis**: "Iteration" (1 to 5). - **Legend**: - Blue line with circles: "Generation" - Orange line with circles: "Multiple-choice" - **Shading**: - Darker blue shaded area for "Generation" confidence interval. - Lighter orange shaded area for "Multiple-choice" confidence interval. - **Legend Position**: Top-right corner of the graph. ### Detailed Analysis 1. **Generation (Blue Line)**: - Iteration 1: ~0.075 - Iteration 2: ~0.050 - Iteration 3: ~0.050 - Iteration 4: ~0.025 - Iteration 5: ~0.025 - **Trend**: Steady decline from Iteration 1 to 4, then plateaus at Iteration 5. 2. **Multiple-choice (Orange Line)**: - Iteration 1: ~0.080 - Iteration 2: ~0.060 - Iteration 3: ~0.040 - Iteration 4: ~0.030 - Iteration 5: ~0.025 - **Trend**: Gradual decline across all iterations, converging with "Generation" at Iteration 5. 3. **Confidence Intervals**: - "Generation" has a narrower confidence interval (darker shading), indicating lower variability. - "Multiple-choice" has a wider confidence interval (lighter shading), suggesting higher uncertainty. ### Key Observations - Both methods show decreasing performance over iterations, but "Multiple-choice" starts with higher values. - At Iteration 5, both methods converge to ~0.025. - The shaded regions overlap significantly in later iterations, indicating similar performance uncertainty. ### Interpretation The data suggests that "Multiple-choice" initially outperforms "Generation" but converges with it by Iteration 5. The narrowing confidence interval for "Generation" implies stabilizing performance, while the wider interval for "Multiple-choice" indicates persistent variability. This could reflect a trade-off between initial accuracy and long-term consistency, with "Generation" potentially being more reliable over time despite slower initial gains. </details> (b) CoT <details> <summary>x9.png Details</summary>  ### Visual Description ## Line Graph: Average Correct Flips Across Iterations ### Overview The image is a line graph comparing two data series ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Correct Flips" (0.000 to 0.100), and the x-axis represents "Iteration" (1 to 5). Shaded regions around each line indicate variability or confidence intervals. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with integer ticks (1–5). - **Y-axis (Average Correct Flips)**: Labeled "Average Correct Flips" with increments of 0.025 (0.000–0.100). - **Legend**: Located in the top-right corner, with: - **Blue circles**: "Generation" - **Orange circles**: "Multiple-choice" - **Shaded Regions**: Gray for "Generation," light orange for "Multiple-choice," positioned symmetrically around each line. ### Detailed Analysis #### Generation (Blue) - **Data Points**: - Iteration 1: ~0.060 - Iteration 2: ~0.030 - Iteration 3: ~0.040 - Iteration 4: ~0.020 - Iteration 5: ~0.020 - **Trend**: Starts at ~0.060, drops sharply to ~0.030 at iteration 2, fluctuates slightly upward at iteration 3, then declines to ~0.020 by iteration 4 and stabilizes. #### Multiple-choice (Orange) - **Data Points**: - Iteration 1: ~0.050 - Iteration 2: ~0.040 - Iteration 3: ~0.020 - Iteration 4: ~0.030 - Iteration 5: ~0.030 - **Trend**: Starts at ~0.050, decreases to ~0.020 at iteration 3, then rises slightly to ~0.030 by iteration 4 and remains stable. ### Key Observations 1. **Initial Disparity**: "Generation" begins with a higher average (~0.060 vs. ~0.050) than "Multiple-choice" at iteration 1. 2. **Divergence at Iteration 2**: "Generation" drops below "Multiple-choice" (~0.030 vs. ~0.040). 3. **Stability in Later Iterations**: Both series converge to ~0.020–0.030 by iteration 4–5, with "Multiple-choice" showing slightly higher stability. 4. **Variability**: "Generation" has a larger shaded region (wider confidence interval) than "Multiple-choice," indicating greater uncertainty in its measurements. ### Interpretation The data suggests that "Generation" initially outperforms "Multiple-choice" but experiences a significant decline in performance by iteration 2, with persistent variability. "Multiple-choice" maintains more consistent performance across iterations, though at a lower magnitude. The widening shaded region for "Generation" implies higher uncertainty in its results, potentially due to methodological instability or external factors affecting its outcomes. This could indicate that "Multiple-choice" is more reliable for sustained performance, while "Generation" may require refinement to address its early drop-off and variability. </details> (c) SC <details> <summary>x10.png Details</summary>  ### Visual Description ## Line Graph: Average Incorrect Flips vs Iteration ### Overview The image is a line graph comparing the performance of two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Incorrect Flips" (ranging from 0.000 to 0.100), and the x-axis represents "Iteration" (1 to 5). Shaded regions around the lines indicate confidence intervals or variability. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with ticks at 1, 2, 3, 4, 5. - **Y-axis (Average Incorrect Flips)**: Labeled "Average Incorrect Flips" with ticks at 0.000, 0.025, 0.050, 0.075, 0.100. - **Legend**: Located in the top-right corner, with: - **Blue dashed line**: "Generation" - **Orange dashed line**: "Multiple-choice" - **Shaded Regions**: Light blue (Generation) and light orange (Multiple-choice) bands around the lines, representing variability. ### Detailed Analysis #### Generation (Blue Dashed Line) - **Iteration 1**: ~0.06 - **Iteration 2**: ~0.04 - **Iteration 3**: ~0.04 - **Iteration 4**: ~0.03 - **Iteration 5**: ~0.04 - **Trend**: Decreases from 0.06 (Iteration 1) to 0.03 (Iteration 4), then slightly increases to 0.04 (Iteration 5). Confidence interval widens at Iterations 1 and 5. #### Multiple-choice (Orange Dashed Line) - **Iteration 1**: ~0.08 - **Iteration 2**: ~0.05 - **Iteration 3**: ~0.06 - **Iteration 4**: ~0.04 - **Iteration 5**: ~0.03 - **Trend**: Starts at 0.08 (Iteration 1), dips to 0.05 (Iteration 2), fluctuates to 0.06 (Iteration 3), then decreases to 0.03 (Iteration 5). Confidence interval widens at Iteration 3. ### Key Observations 1. Both methods show improvement in reducing incorrect flips over iterations. 2. "Multiple-choice" starts with higher incorrect flips (0.08) but ends lower (0.03) than "Generation" (0.04) by Iteration 5. 3. "Generation" exhibits a more stable trend, while "Multiple-choice" has greater variability (e.g., peak at Iteration 3). 4. Confidence intervals overlap significantly at Iterations 2 and 4, suggesting uncertainty in performance differences. ### Interpretation The data suggests that "Multiple-choice" may outperform "Generation" in later iterations, despite higher initial variability. The final drop in "Multiple-choice" (Iteration 5) could indicate adaptive learning or optimization. However, the overlapping confidence intervals imply that the difference between methods is not statistically definitive in all iterations. The shaded regions highlight that "Generation" has higher uncertainty in early and late iterations, while "Multiple-choice" shows greater variability in mid-iterations. This might reflect differing algorithmic behaviors or sensitivity to input changes. </details> (d) Baseline <details> <summary>x11.png Details</summary>  ### Visual Description ## Line Graph: Average Incorrect Flips Over Iterations ### Overview The image is a line graph comparing two methods ("Generation" and "Multiple-choice") across five iterations, measuring "Average Incorrect Flips" on a y-axis (0.000–0.100) and "Iteration" on the x-axis (1–5). Shaded regions around the lines represent confidence intervals. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration," with ticks at 1, 2, 3, 4, 5. - **Y-axis (Average Incorrect Flips)**: Labeled "Average Incorrect Flips," with ticks at 0.000, 0.025, 0.050, 0.075, 0.100. - **Legend**: Located in the top-right corner, with: - **Blue dashed line**: "Generation" - **Orange dashed line**: "Multiple-choice" - **Shaded Regions**: Light blue (Generation) and light orange (Multiple-choice) indicate uncertainty intervals. ### Detailed Analysis #### Generation (Blue Dashed Line) - **Iteration 1**: ~0.06 - **Iteration 2**: ~0.05 - **Iteration 3**: ~0.03 - **Iteration 4**: ~0.04 - **Iteration 5**: ~0.02 - **Trend**: Decreasing overall, with a slight uptick in Iteration 4 before resuming decline. #### Multiple-choice (Orange Dashed Line) - **Iteration 1**: ~0.08 - **Iteration 2**: ~0.06 - **Iteration 3**: ~0.06 - **Iteration 4**: ~0.03 - **Iteration 5**: ~0.03 - **Trend**: Steady decline until Iteration 4, then plateaus. #### Shaded Regions - **Generation**: Confidence intervals widen in Iterations 1–2, narrow in Iterations 3–5. - **Multiple-choice**: Confidence intervals remain relatively consistent across iterations. ### Key Observations 1. Both methods show improvement in reducing incorrect flips over iterations. 2. "Multiple-choice" starts with higher error rates but declines more consistently. 3. "Generation" exhibits variability, with a temporary increase in Iteration 4. 4. Shaded regions suggest greater uncertainty in early iterations for both methods. ### Interpretation The data suggests that both "Generation" and "Multiple-choice" methods improve performance (reduce incorrect flips) as iterations increase. However, "Multiple-choice" demonstrates more stability, while "Generation" shows fluctuating performance, particularly in Iteration 4. The shaded confidence intervals highlight that results are less reliable in early iterations, emphasizing the need for larger sample sizes or extended testing to validate trends. The plateau in "Multiple-choice" at Iteration 5 may indicate diminishing returns or convergence toward an optimal threshold. </details> (e) CoT <details> <summary>x12.png Details</summary>  ### Visual Description ## Line Graph: Average Incorrect Flips Across Iterations ### Overview The image is a line graph comparing two data series: "Generation" (blue dashed line) and "Multiple-choice" (orange dashed line) across five iterations. The y-axis represents "Average Incorrect Flips" (ranging from 0.000 to 0.100), while the x-axis represents "Iteration" (1 to 5). Shaded regions around each line indicate variability or confidence intervals. A legend in the top-right corner identifies the two series. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with ticks at 1, 2, 3, 4, 5. - **Y-axis (Average Incorrect Flips)**: Labeled "Average Incorrect Flips" with values from 0.000 to 0.100 in increments of 0.025. - **Legend**: Located in the top-right corner, with: - Blue dashed line: "Generation" - Orange dashed line: "Multiple-choice" - **Shaded Regions**: Gray areas around each line, representing variability or uncertainty. ### Detailed Analysis #### Generation (Blue Dashed Line) - **Iteration 1**: ~0.06 - **Iteration 2**: ~0.045 - **Iteration 3**: ~0.03 - **Iteration 4**: ~0.025 - **Iteration 5**: ~0.02 - **Trend**: Steady decline from ~0.06 to ~0.02, indicating consistent improvement over iterations. #### Multiple-choice (Orange Dashed Line) - **Iteration 1**: ~0.05 - **Iteration 2**: ~0.03 - **Iteration 3**: ~0.04 - **Iteration 4**: ~0.03 - **Iteration 5**: ~0.025 - **Trend**: Fluctuates between ~0.03 and ~0.04, with no clear directional trend. ### Key Observations 1. **Generation** shows a consistent downward trend, suggesting improved performance (fewer incorrect flips) as iterations increase. 2. **Multiple-choice** exhibits variability, with no significant improvement or decline across iterations. 3. Shaded regions for both series indicate overlapping variability, but the "Generation" series has a narrower confidence interval, implying more stable results. ### Interpretation The data suggests that the "Generation" method outperforms "Multiple-choice" in reducing average incorrect flips over iterations. The steady decline in "Generation" implies it adapts or improves with repeated trials, while "Multiple-choice" remains inconsistent. The shaded regions highlight that "Generation" has lower uncertainty, making it a more reliable approach in this context. The lack of a clear trend in "Multiple-choice" may indicate limitations in its adaptability or sensitivity to iteration changes. </details> (f) SC Figure 2: Average Correct and Incorrect Flips on DisambiguationQA We now analyze the results in relation to our three research questions. Improvement Patterns Across Iterations (RQ1). To address RQ1, we first examine the aggregate performance reported in Figure 1, which compares accuracy across correction iterations for generation and multiple-choice formats. The generation paradigm improves rapidly in the first one or two iterations, showing that early revisions are effective at fixing obvious errors or adding missing information. However, after these early gains, performance often plateaus or declines, as additional revisions increase the risk of semantic drift and lead to new mistakes. In contrast, the multiple-choice paradigm improves more gradually and steadily. Accuracy rises incrementally with each round of self-correction, reflecting cautious re-weighting among fixed options. Yet this format struggles to recover from poor initial predictions: if the model’s first choice is wrong, subsequent iterations rarely flip it to the correct option, showing the effects of logit inertia. Figures 2 and 3 present the “flip” dynamics of self-correction on the two datasets, broken down into correct (a previously wrong answer corrected to right) and incorrect (a previously correct answer changed to wrong) flips over successive iterations. On DisambiguationQA (Figure 2), multiple-choice self-correction yields very few flips overall. Correct answers are stably retained, but wrong initial guesses are seldom corrected. Generation, by contrast, produces more frequent flips: many beneficial in early iterations (correcting ambiguous references) but increasingly harmful in later ones, as correct answers are sometimes replaced with incorrect ones, once the model starts to over-correct or drift. On tinyTruthfulQA (Figure 3), the contrast is sharper: generation produces a high number of flips, with many early correct flips (replacing misconceptions with truths), but also a rising number of incorrect flips in later rounds, reflecting semantic drift. Multiple-choice again remains stable, with minimal incorrect flips but limited ability to recover from an early mistake. Taken together, we show that open-ended generation offers adaptability and rapid early gains but suffers from instability in later iterations, whereas multiple-choice offers stability and incremental improvement but is hampered by inertia when the first choice is wrong. This confirms that self-correction effectiveness is strongly dependent on task format: open-ended generation can exploit flexibility to correct errors but risks drift, while multiple-choice provides reliable retention of correct answers at the expense of recoverability. If the model doesn’t get the answer right on the first attempt, it has a hard time changing to the correct option later. This fundamental difference in dynamics directly answers RQ1: self-correction behaves very differently in open-ended versus fixed-option scenarios, with each paradigm exhibiting its own pattern of improvement and failure modes. <details> <summary>x13.png Details</summary>  ### Visual Description ## Line Graph: Average Correct Flips Over Iterations ### Overview The image is a line graph comparing two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Correct Flips" (0.000–0.100), and the x-axis represents "Iteration" (1–5). Shaded regions around each line indicate variability/confidence intervals. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with ticks at 1, 2, 3, 4, 5. - **Y-axis (Average Correct Flips)**: Labeled "Average Correct Flips" with ticks at 0.000, 0.025, 0.050, 0.075, 0.100. - **Legend**: Located in the top-right corner, with: - **Blue line**: "Generation" - **Orange line**: "Multiple-choice" - **Shaded Regions**: Gray areas around each line, representing variability (wider for "Multiple-choice"). ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.050 - **Iteration 2**: ~0.050 (flat) - **Iteration 3**: ~0.035 - **Iteration 4**: ~0.025 - **Iteration 5**: ~0.030 - **Trend**: Slight downward trend with a minor recovery in Iteration 5. Shaded region is narrow, indicating low variability. #### Multiple-choice (Orange Line) - **Iteration 1**: ~0.060 - **Iteration 2**: ~0.050 - **Iteration 3**: ~0.025 - **Iteration 4**: ~0.010 - **Iteration 5**: ~0.020 - **Trend**: Sharp decline from Iteration 1 to 4, followed by a partial recovery in Iteration 5. Shaded region is wide, indicating high variability. ### Key Observations 1. **Divergence in Performance**: "Generation" maintains higher average correct flips than "Multiple-choice" after Iteration 2. 2. **Volatility**: "Multiple-choice" shows significantly higher variability (wider shaded regions), especially in Iterations 3–4. 3. **Recovery in Iteration 5**: Both methods show slight increases in Iteration 5, but "Multiple-choice" remains below "Generation." ### Interpretation The data suggests that the "Generation" method is more consistent and reliable over iterations, while "Multiple-choice" exhibits declining performance and greater uncertainty. The sharp drop in "Multiple-choice" (Iterations 3–4) may indicate methodological limitations or external factors affecting its effectiveness. The partial recovery in Iteration 5 for "Multiple-choice" could signal an adjustment or anomaly, but the overall trend underscores its inferior stability compared to "Generation." </details> (a) Baseline <details> <summary>x14.png Details</summary>  ### Visual Description ## Line Graph: Average Correct Flips Over Iterations ### Overview The image is a line graph comparing two data series ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Correct Flips" (0.000–0.100), and the x-axis represents "Iteration" (1–5). Shaded regions indicate confidence intervals for each series. ### Components/Axes - **Y-axis**: "Average Correct Flips" (0.000 to 0.100 in increments of 0.025). - **X-axis**: "Iteration" (1 to 5). - **Legend**: - Blue line with circles: "Generation" - Orange line with circles: "Multiple-choice" - **Shading**: - Blue-shaded region: Confidence interval for "Generation" - Orange-shaded region: Confidence interval for "Multiple-choice" ### Detailed Analysis #### Generation (Blue) - **Data Points**: - Iteration 1: ~0.075 - Iteration 2: ~0.075 - Iteration 3: ~0.050 - Iteration 4: ~0.040 - Iteration 5: ~0.040 - **Trend**: Starts at 0.075, plateaus at iteration 2, then decreases steadily to 0.040 by iteration 5. Confidence intervals widen slightly after iteration 2. #### Multiple-choice (Orange) - **Data Points**: - Iteration 1: ~0.075 - Iteration 2: ~0.025 - Iteration 3: ~0.025 - Iteration 4: ~0.020 - Iteration 5: ~0.020 - **Trend**: Sharp drop from 0.075 to 0.025 at iteration 2, followed by a gradual decline to 0.020 by iteration 5. Confidence intervals remain narrow. ### Key Observations 1. **Generation** shows a gradual decline in performance over iterations, with higher variability (wider confidence intervals). 2. **Multiple-choice** exhibits a steep initial drop, followed by stabilization, with tighter confidence intervals. 3. Both series start at the same value (0.075) at iteration 1 but diverge significantly by iteration 2. ### Interpretation The data suggests that "Generation" and "Multiple-choice" represent distinct strategies or models. The sharp decline in "Multiple-choice" after iteration 1 may indicate a one-time adjustment or saturation effect, while "Generation" shows a more gradual adaptation. The widening confidence intervals for "Generation" imply increasing uncertainty in its measurements over time. This could reflect a learning curve or diminishing returns in iterative processes. The initial parity at iteration 1 highlights a critical starting point where both methods perform equally, but their divergence underscores fundamental differences in their behavior under repeated iterations. </details> (b) CoT <details> <summary>x15.png Details</summary>  ### Visual Description ## Line Graph: Average Correct Flips Across Iterations ### Overview The image depicts a line graph comparing two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents "Average Correct Flips" (ranging from 0.000 to 0.100), while the x-axis represents "Iteration" (1 to 5). Shaded regions around each line indicate variability or confidence intervals. The legend is positioned in the top-right corner. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration," with discrete values 1, 2, 3, 4, 5. - **Y-axis (Average Correct Flips)**: Labeled "Average Correct Flips," scaled from 0.000 to 0.100 in increments of 0.025. - **Legend**: Located in the top-right corner, with: - **Blue line**: "Generation" - **Orange line**: "Multiple-choice" - **Shaded Regions**: Gray areas surrounding each line, representing variability or uncertainty. ### Detailed Analysis #### Generation (Blue Line) - **Trend**: Starts at ~0.050 (Iteration 1), drops to ~0.025 (Iteration 2), remains stable at ~0.025 (Iteration 3), rises to ~0.035 (Iteration 4), then slightly declines to ~0.025 (Iteration 5). - **Variability**: Shaded region widest at Iteration 4 (~0.015–0.055 range), narrowing to ~0.010–0.040 at Iteration 5. #### Multiple-choice (Orange Line) - **Trend**: Starts at ~0.060 (Iteration 1), decreases to ~0.030 (Iteration 2), stabilizes at ~0.020 (Iterations 3–5). - **Variability**: Shaded region narrowest (~0.010–0.030 at Iteration 2), widening slightly to ~0.005–0.035 by Iteration 5. ### Key Observations 1. **Stability**: The "Multiple-choice" method shows a steeper initial decline and stabilizes earlier (Iteration 3 onward), while "Generation" fluctuates more. 2. **Variability**: The "Generation" method exhibits significantly wider confidence intervals, especially at Iteration 4. 3. **Performance**: Both methods improve slightly over iterations, but "Multiple-choice" maintains a higher baseline after Iteration 2. ### Interpretation The data suggests that the "Multiple-choice" method is more consistent and reliable, as evidenced by its narrower confidence intervals and stable performance after Iteration 2. In contrast, the "Generation" method shows higher variability, with a notable spike in uncertainty at Iteration 4. The shaded regions likely represent 95% confidence intervals, indicating that the "Generation" method’s estimates are less precise. The trends imply that while both methods improve with iterations, "Multiple-choice" may be preferable in scenarios prioritizing stability over incremental gains. </details> (c) SC <details> <summary>x16.png Details</summary>  ### Visual Description ## Line Chart: Average Incorrect Flips Across Iterations ### Overview The chart compares two data series ("Generation" and "Multiple-choice") across five iterations, showing the average number of incorrect flips. Both series exhibit downward trends, with "Multiple-choice" consistently outperforming "Generation" in reducing errors. ### Components/Axes - **X-axis (Horizontal)**: Labeled "Iteration" with markers at 1, 2, 3, 4, 5. - **Y-axis (Vertical)**: Labeled "Average Incorrect Flips" with values from 0.000 to 0.100 in increments of 0.025. - **Legend**: Located in the top-right corner, with: - **Blue dashed line**: "Generation" - **Orange dashed line**: "Multiple-choice" - **Shaded Regions**: Gray areas around each line represent confidence intervals (uncertainty ranges). ### Detailed Analysis 1. **Generation (Blue Line)**: - Iteration 1: ~0.075 - Iteration 2: ~0.045 - Iteration 3: ~0.040 - Iteration 4: ~0.035 - Iteration 5: ~0.035 - **Trend**: Steep decline from Iteration 1 to 2, followed by gradual stabilization. 2. **Multiple-choice (Orange Line)**: - Iteration 1: ~0.035 - Iteration 2: ~0.025 - Iteration 3: ~0.020 - Iteration 4: ~0.015 - Iteration 5: ~0.010 - **Trend**: Steady linear decline across all iterations. ### Key Observations - "Multiple-choice" starts with significantly lower error rates than "Generation" at all iterations. - Both series show decreasing trends, but "Multiple-choice" maintains a more consistent rate of improvement. - The shaded confidence intervals for "Generation" are wider, indicating greater uncertainty in its measurements. ### Interpretation The data suggests that the "Multiple-choice" approach is more effective at reducing incorrect flips over iterations compared to "Generation." The steeper initial drop in "Generation" may indicate rapid early improvements, but its plateau at later iterations implies diminishing returns. The narrower confidence intervals for "Multiple-choice" suggest more reliable performance metrics. This could imply that "Multiple-choice" offers a more stable and scalable solution for error reduction in iterative processes. </details> (d) Baseline <details> <summary>x17.png Details</summary>  ### Visual Description ## Line Chart: Average Incorrect Flips Over Iterations ### Overview The chart illustrates the average number of incorrect flips for two methods ("Generation" and "Multiple-choice") across five iterations. The y-axis represents the average incorrect flips (0.000 to 0.100), while the x-axis shows iterations (1 to 5). Shaded regions around each line likely represent confidence intervals or variability. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration" with values 1, 2, 3, 4, 5. - **Y-axis (Average Incorrect Flips)**: Labeled "Average Incorrect Flips" with values 0.000, 0.025, 0.050, 0.075, 0.100. - **Legend**: Located in the top-right corner, with two entries: - **Blue dashed line**: "Generation" - **Orange dashed line**: "Multiple-choice" - **Shaded Regions**: Light blue (Generation) and light orange (Multiple-choice) areas around each line, indicating variability. ### Detailed Analysis - **Generation (Blue Dashed Line)**: - **Trend**: Slopes downward from Iteration 1 to 5. - **Data Points**: - Iteration 1: ~0.100 - Iteration 2: ~0.060 - Iteration 3: ~0.055 - Iteration 4: ~0.040 - Iteration 5: ~0.035 - **Shaded Area**: Wider at Iteration 1, narrowing as iterations increase, suggesting decreasing variability. - **Multiple-choice (Orange Dashed Line)**: - **Trend**: Relatively flat with a slight dip at Iteration 2. - **Data Points**: - Iteration 1: ~0.020 - Iteration 2: ~0.025 - Iteration 3: ~0.020 - Iteration 4: ~0.020 - Iteration 5: ~0.020 - **Shaded Area**: Narrower and more consistent across iterations, indicating stable variability. ### Key Observations 1. **Generation** shows a clear downward trend, with average incorrect flips decreasing by ~65% from Iteration 1 to 5. 2. **Multiple-choice** remains nearly constant, with minimal fluctuation (~0.020–0.025). 3. The shaded regions for Generation are significantly larger at Iteration 1, indicating higher uncertainty early in the process. ### Interpretation The data suggests that the "Generation" method improves over time, reducing average incorrect flips, while "Multiple-choice" maintains a stable but lower performance. The widening shaded area for Generation at Iteration 1 implies higher initial variability, which diminishes as iterations progress. This could indicate that Generation is more effective for error reduction but requires more iterations to stabilize. The consistent performance of Multiple-choice suggests it may be a reliable baseline, though less adaptive to iterative improvements. </details> (e) CoT <details> <summary>x18.png Details</summary>  ### Visual Description ## Line Graph: Average Incorrect Flips Over Iterations ### Overview The graph compares two methods ("Generation" and "Multiple-choice") across five iterations, measuring their average incorrect flips. Both methods show a downward trend, but "Generation" consistently exhibits higher error rates than "Multiple-choice." Shaded regions around the lines represent confidence intervals, with "Generation" having wider variability. ### Components/Axes - **X-axis (Iteration)**: Labeled "Iteration," with markers at 1, 2, 3, 4, and 5. - **Y-axis (Average Incorrect Flips)**: Labeled "Average Incorrect Flips," scaled from 0.000 to 0.100 in increments of 0.025. - **Legend**: Located in the top-right corner, with: - **Blue dashed line**: "Generation" - **Orange dashed line**: "Multiple-choice" - **Shaded Regions**: Light blue (Generation) and light orange (Multiple-choice) indicate confidence intervals. ### Detailed Analysis - **Generation (Blue)**: - Iteration 1: ~0.060 - Iteration 2: ~0.050 - Iteration 3: ~0.040 - Iteration 4: ~0.030 - Iteration 5: ~0.020 - Trend: Steady decline with a slight plateau between iterations 4–5. - Confidence interval: ±~0.015 (wider than Multiple-choice). - **Multiple-choice (Orange)**: - Iteration 1: ~0.050 - Iteration 2: ~0.040 - Iteration 3: ~0.030 - Iteration 4: ~0.020 - Iteration 5: ~0.015 - Trend: Consistent linear decline. - Confidence interval: ±~0.010. ### Key Observations 1. **Error Reduction**: Both methods improve over iterations, but "Multiple-choice" achieves lower final error rates (~0.015 vs. ~0.020 for "Generation"). 2. **Variability**: "Generation" shows greater uncertainty in its measurements, as evidenced by its broader confidence intervals. 3. **Convergence**: The gap between the two methods narrows slightly in later iterations but remains significant. ### Interpretation The data suggests that while both methods reduce errors with more iterations, "Multiple-choice" is more reliable and efficient in minimizing incorrect flips. The persistent higher error rate in "Generation" could indicate inherent limitations in its approach, such as sensitivity to noise or suboptimal parameter tuning. The widening confidence intervals for "Generation" imply less predictability, which might make it riskier for applications requiring consistent performance. This analysis could guide resource allocation or algorithm selection in scenarios where error minimization is critical. </details> (f) SC Figure 3: Average Correct and Incorrect Flips on tinyTruthfulQA Effects of Model Scale and Prompting Strategy (RQ2). <details> <summary>x19.png Details</summary>  ### Visual Description ## Line Graphs: Model Performance Across Prompting Methods ### Overview The image contains six line graphs arranged in a 2x3 grid, comparing model accuracy across iterations (0-5) for two QA tasks: **DisambiguationQA** (top row) and **tinyTruthfulQA** (bottom row). Each graph evaluates three prompting methods: **Baseline**, **CoT (Chain-of-Thought)**, and **Self-Consistency**. Accuracy (%) is plotted on the y-axis, while iterations (0-5) are on the x-axis. Models are differentiated by color and marker type in the legend. --- ### Components/Axes - **X-axis**: Iteration (0 to 5, integer steps). - **Y-axis**: Accuracy (%) (0% to 80%, with increments of ~20%). - **Legend**: Located at the bottom of all graphs. Models include: - **Generation** (blue circles) - **Multiple-choice** (orange squares) - **Gemini 2.0-Flash** (gray squares) - **Llama 3.1-8B** (gray triangles) - **DeepSeek-R1-Distill-Llama-8B** (gray diamonds) - **Qwen2.5-14B** (orange triangles) - **SmolLM2-1.7B** (orange diamonds) - **Qwen2.5-3B** (blue triangles) --- ### Detailed Analysis #### DisambiguationQA - Baseline - **Generation** (blue circles): Starts at ~0.3%, peaks at iteration 2 (~0.4%), then declines to ~0.2% by iteration 5. - **Multiple-choice** (orange squares): Starts at ~0.1%, peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **Gemini 2.0-Flash** (gray squares): Stable at ~0.4% across all iterations. - **Llama 3.1-8B** (gray triangles): Peaks at iteration 1 (~0.5%), then declines to ~0.3%. - **DeepSeek-R1-Distill-Llama-8B** (gray diamonds): Stable at ~0.4%. - **Qwen2.5-14B** (orange triangles): Starts at ~0.2%, peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **SmolLM2-1.7B** (orange diamonds): Stable at ~0.3%. - **Qwen2.5-3B** (blue triangles): Stable at ~0.3%. #### DisambiguationQA - CoT - **Generation**: Peaks at iteration 2 (~0.4%), then declines to ~0.2%. - **Multiple-choice**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **Gemini 2.0-Flash**: Stable at ~0.4%. - **Llama 3.1-8B**: Peaks at iteration 1 (~0.5%), then declines to ~0.3%. - **DeepSeek-R1-Distill-Llama-8B**: Stable at ~0.4%. - **Qwen2.5-14B**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **SmolLM2-1.7B**: Stable at ~0.3%. - **Qwen2.5-3B**: Stable at ~0.3%. #### DisambiguationQA - Self-Consistency - **Generation**: Peaks at iteration 2 (~0.4%), then declines to ~0.2%. - **Multiple-choice**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **Gemini 2.0-Flash**: Stable at ~0.4%. - **Llama 3.1-8B**: Peaks at iteration 1 (~0.5%), then declines to ~0.3%. - **DeepSeek-R1-Distill-Llama-8B**: Stable at ~0.4%. - **Qwen2.5-14B**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **SmolLM2-1.7B**: Stable at ~0.3%. - **Qwen2.5-3B**: Stable at ~0.3%. #### tinyTruthfulQA - Baseline - **Generation**: Starts at ~0.5%, peaks at iteration 2 (~0.6%), then declines to ~0.4%. - **Multiple-choice**: Starts at ~0.2%, peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **Gemini 2.0-Flash**: Stable at ~0.6%. - **Llama 3.1-8B**: Peaks at iteration 1 (~0.7%), then declines to ~0.5%. - **DeepSeek-R1-Distill-Llama-8B**: Stable at ~0.6%. - **Qwen2.5-14B**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **SmolLM2-1.7B**: Stable at ~0.3%. - **Qwen2.5-3B**: Stable at ~0.3%. #### tinyTruthfulQA - CoT - **Generation**: Peaks at iteration 2 (~0.6%), then declines to ~0.4%. - **Multiple-choice**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **Gemini 2.0-Flash**: Stable at ~0.6%. - **Llama 3.1-8B**: Peaks at iteration 1 (~0.7%), then declines to ~0.5%. - **DeepSeek-R1-Distill-Llama-8B**: Stable at ~0.6%. - **Qwen2.5-14B**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **SmolLM2-1.7B**: Stable at ~0.3%. - **Qwen2.5-3B**: Stable at ~0.3%. #### tinyTruthfulQA - Self-Consistency - **Generation**: Peaks at iteration 2 (~0.6%), then declines to ~0.4%. - **Multiple-choice**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **Gemini 2.0-Flash**: Stable at ~0.6%. - **Llama 3.1-8B**: Peaks at iteration 1 (~0.7%), then declines to ~0.5%. - **DeepSeek-R1-Distill-Llama-8B**: Stable at ~0.6%. - **Qwen2.5-14B**: Peaks at iteration 3 (~0.3%), then drops to ~0.2%. - **SmolLM2-1.7B**: Stable at ~0.3%. - **Qwen2.5-3B**: Stable at ~0.3%. --- ### Key Observations 1. **CoT and Self-Consistency** generally outperform **Baseline** in both tasks, with higher accuracy and stability. 2. **Gemini 2.0-Flash** and **DeepSeek-R1-Distill-Llama-8B** consistently achieve the highest accuracy (~0.6% in tinyTruthfulQA). 3. **Llama 3.1-8B** shows strong initial performance but declines over iterations. 4. **Multiple-choice** and **Qwen2.5-14B** exhibit the lowest accuracy, with minimal improvement across iterations. 5. **tinyTruthfulQA** graphs show higher baseline accuracy (~0.5-0.7%) compared to DisambiguationQA (~0.2-0.5%). --- ### Interpretation The data suggests that **prompting methods** significantly impact model performance. **CoT** and **Self-Consistency** improve accuracy over **Baseline**, likely by encouraging structured reasoning. **Gemini 2.0-Flash** and **DeepSeek-R1-Distill-Llama-8B** outperform other models, indicating superior architecture or training for these tasks. The decline in accuracy for some models (e.g., Llama 3.1-8B) over iterations may reflect overfitting or sensitivity to input variations. **tinyTruthfulQA** tasks are inherently more challenging, as evidenced by lower overall accuracy compared to DisambiguationQA. The stability of certain models (e.g., Gemini 2.0-Flash) highlights their robustness to iterative changes. </details> Figure 4: Accuracy per iteration per model on generation and multiple-choice. Here, we investigate how a model’s size and the prompting strategy influence self-correction, and whether these effects differ between the two output paradigms. Figure 4 provides a detailed view of accuracy per iteration for various models under different prompting methods. A clear finding is that task difficulty moderates these effects. On the challenging DisambiguationQA benchmark, accuracy is low for all models: even the largest (e.g., Gemini-2.0-Flash, Qwen2.5-14B) plateau around 50% in multiple-choice and below 20% in generation, while smaller models perform far worse. In contrast, on the easier tinyTruthfulQA, generative accuracy ranges from 60–90% and multiple-choice from 50–80%, with even small models performing well. Thus, model scale yields clear benefits on harder tasks, but differences narrow considerably on simpler ones. The prompting strategy has a modest but noticeable effect, more so on the difficult task. On DisambiguationQA, using an explicit CoT prompt or a SC approach yields slight accuracy improvements over the Baseline direct prompting. For example, prompting the model to “think step by step” or to consider multiple reasoning paths sometimes helps it disambiguate the question better, nudging up the accuracy by a few percentage points. These gains, while not dramatic, suggest that reasoning-oriented prompts can aid the model on ambiguous, challenging questions. In contrast, on tinyTruthfulQA, all three prompting strategies lead to very similar performance. The accuracy curves for different prompts on this task are nearly overlapping (Figure 4), indicating that when a question is relatively straightforward or the model already knows the domain (e.g., common truths vs. misconceptions), an elaborate prompt does not provide much benefit. In summary, prompting variations have a task-dependent impact: they can be slightly beneficial for resolving difficult queries (DisambiguationQA) but mostly redundant for simpler factual questions (tinyTruthfulQA). This aligns with the findings in the literature Sprague et al. (2025). Model scale shows a similarly nuanced effect. Larger models generally outperform smaller ones, especially on DisambiguationQA, where 14B+ models clearly surpass 1–3B models. On tinyTruthfulQA, however, the performance gap narrows, with small models often approaching large-model accuracy. In some cases, scaling produces diminishing returns, indicating that size matters more for difficult tasks but offers limited advantage once a task is already within reach. Notably, repeated iterations of self-correction do not consistently boost accuracy for either paradigm, regardless of model size or prompt strategy. Across our experiments, most performance curves over iterations (spanning iteration 0 through 5) are relatively flat after the initial step. As highlighted by Figure 4, it is rare to see a clear upward trajectory beyond the first one or two iterations; instead, accuracy often oscillates with minor gains or losses. For example, a model might correct a mistake at iteration 1, only to introduce a different mistake at iteration 3, ending up with a similar accuracy as it started. This plateauing behavior implies that giving the model many chances to self-correct yields diminishing returns. Neither larger scale nor advanced prompting fundamentally changes this outcome – their benefits tend to manifest in the first attempt or two, but they do not drive continual improvement with more iterations. In some cases, we even observed slight performance degradation with too many iterations (echoing the drift issues from RQ1). In summary, the impact of model scale and prompting strategy on self-correction is real but nuanced: larger models and CoT-style prompts can improve initial accuracy, especially on hard tasks, but these factors are task-dependent and ultimately insufficient to guarantee ongoing improvements through iterative self-correction alone. Multiple-choice and generation formats alike see their gains saturate early, and improvements from scaling or better prompting taper off without addressing the core limitations of each paradigm. Notably, we also found that the multiple-choice paradigm often reaped slightly more benefit from increased model size and reasoning prompts than the generation paradigm did (especially on DisambiguationQA), reinforcing the idea that constrained decision tasks can more readily capitalize on those enhancements. Still, neither paradigm achieves a dramatically upward performance trend with iteration – a key insight for understanding the boundaries of current self-correction capabilities. Trade-offs Between Adaptability and Stability (RQ3). RQ3 examines how iterative self-correction influences correctness, stability, and semantic drift across unconstrained and constrained outputs. In the generation setting, flexibility allows models to revise and often improve answers in the first one or two iterations, but this same flexibility leads to semantic drift in later rounds. As Figures 2 and 3 as well as the detailed plots of per model evaluation in Appendix C.1, generation produces many flips: early ones are often correct (e.g., resolving an ambiguity or correcting a misconception), but over time, incorrect flips dominate as the model over-edits or drifts away from the question. This suggests that while generation supports adaptability, it lacks effective internal checks to prevent harmful revisions. By contrast, in the multiple-choice setting, the output space is restricted to fixed options, which prevents drift altogether. Correct answers remain locked in across iterations, reflecting high stability. However, this comes with logit inertia: wrong initial answers persist, with very few corrective flips observed in Figures 2 and 3. The mechanism here is that once a wrong option is selected, the model rarely shifts its ranking enough to choose the correct one later, even when revisiting its reasoning. These patterns reveal a fundamental adaptability–stability trade-off. Generation is exploratory and can recover from initial mistakes, but risks undermining correctness as iterations accumulate. Multiple-choice ensures consistency once correct, but limits opportunities to fix errors. For system design, this implies that neither paradigm is universally optimal. Applications requiring stable outputs, such as safety-critical domains, benefit from constrained correction, though additional mechanisms may be needed to overcome inertia (e.g., external verification or re-ranking). Conversely, tasks where capturing every possible correction is crucial may favor open-ended revision, provided that safeguards against drift are implemented. Promising directions include hybrid strategies that combine paradigms, using generation to explore candidate answers followed by constrained verification to anchor correctness, and dynamic stopping rules that halt iteration once improvements saturate or harmful drift is detected. Addressing these trade-offs directly, by mitigating semantic drift in generation and reducing inertia in multiple-choice, will be key to making iterative self-correction a reliable capability of LLM systems. ## 6 Conclusion This study compared iterative self-correction in large language models across open-ended generation and multiple-choice question answering. Results show that the structure of the output space fundamentally shapes correction dynamics. Generation achieves rapid early gains by correcting errors in the first few iterations, but suffers from semantic drift as revisions accumulate, resulting in increasing rates of incorrect flips. Multiple-choice responses remain highly stable and avoid drift, but exhibit logit inertia: wrong initial answers are rarely overturned, and improvements are incremental at best. Model scale and prompting strategy modulate performance but do not alter these core patterns. Larger models and reasoning-oriented prompts (CoT, SC) yield slight improvements, especially on the harder DisambiguationQA task, but their effects are modest and task-dependent. Across both paradigms, accuracy generally plateaus after the first one or two iterations, showing that repeated self-correction brings limited benefit. These findings highlight an inherent adaptability–stability trade-off. Open-ended generation enables recovery from errors but risks instability, while multiple-choice ensures reliability but limits correction. Future work should explore hybrid strategies, such as using generation for exploration and constrained formats for verification, as well as dynamic stopping criteria to prevent late drift. Addressing drift and inertia directly will be essential for building reliable self-correcting LLM systems. ## Limitations This study focuses on benchmarks that provide parallel formulations for both open-ended generation and multiple-choice questions. While this setup enables a controlled analysis of self-correction across task formats, it also limits the number of datasets available for evaluation, as few benchmarks support both types of tasks. Moreover, our experiments are conducted using currently available models of moderate scale. Recent larger models, which may exhibit different self-correction dynamics and reasoning behaviors, are not included in our analysis. Future work could extend our study to such models to provide a more comprehensive understanding of scaling effects. ## Ethical Considerations We have carefully verified that the software, model checkpoints and existing datasets utilised in this work are permitted for access, distribution and, where relevant, modification. Our use and purpose comply with those terms. ## Acknowledgments This research is supported by the Engineering and Physical Sciences Research Council [EP/S021566/1] and the EPSRC Fellowship titled “Task Based Information Retrieval” [EP/P024289/1]. ## References - A2i (2025) A2i. 2025. TruthfulQA Truth Judge. Accessed: 2025. - Allal et al. (2025) Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martin Blazquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Agustín Piqueres Lajarín, Hynek Kydlíček, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan Son NGUYEN, Ben Burtenshaw, Clémentine Fourrier, Haojun Zhao, Hugo Larcher, Mathieu Morlon, Cyril Zakka, and 3 others. 2025. SmolLM2: When smol goes big — data-centric training of a fully open small language model. In Second Conference on Language Modeling. - Belcak et al. (2025) Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. 2025. Small language models are the future of agentic ai. arXiv preprint arXiv:2506.02153. - Brown and et al. (2020) Tom B Brown and et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165. - Chang et al. (2024) Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1–45. - Chen et al. (2023) Mark Y Chen, Chia-Wei Liu, Xuezhi Wang, Quoc V Le, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Program-aided language models: Language models as programs. arXiv preprint arXiv:2303.11366. - Clark and et al. (2018) Peter Clark and et al. 2018. Think you have reasoning solved? evaluating the arc challenge. arXiv preprint arXiv:1803.05457. - Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Anish Madaan, and et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. - Comanici et al. (2025) Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. - Cook et al. (2024) Jonathan Cook, Tim Rocktäschel, Jakob Nicolaus Foerster, Dennis Aumiller, and Alex Wang. 2024. TICKing all the boxes: Generated checklists improve LLM evaluation and generation. In Language Gamification - NeurIPS 2024 Workshop. - Dixit et al. (2023) Tanay Dixit, Fei Wang, Muhao Chen, and et al. 2023. Improving factuality of abstractive summarization without sacrificing summary quality. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 902–913. - Gou et al. (2024) Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2024. CRITIC: Large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations. - Grattafiori et al. (2024) Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. - Guo et al. (2025) Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. - Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Santi Basart, and et al. 2021. Measuring massive multitask language understanding. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 10013–10023. - Huang et al. (2023) Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. - Ji et al. (2023a) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023a. Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1–38. - Ji et al. (2023b) Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023b. Towards mitigating llm hallucination via self-reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827–1843. - Kamoi et al. (2024) Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. 2024. When can llms actually correct their own mistakes? a critical survey of self-correction of llms. Transactions of the Association for Computational Linguistics, 12:1417–1440. - Kazemi et al. (2025) Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, and 1 others. 2025. Big-bench extra hard. arXiv preprint arXiv:2502.19187. - Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213. - Krishna et al. (2024) Satyapriya Krishna, Chirag Agarwal, and Himabindu Lakkaraju. 2024. Understanding the effects of iterative prompting on truthfulness. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. - Kumar et al. (2025) Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, and Aleksandra Faust. 2025. Training language models to self-correct via reinforcement learning. In The Thirteenth International Conference on Learning Representations. - Li and et al. (2024) Wei Li and et al. 2024. Logitlens: Calibrating reasoning in language models with internal consistency. NeurIPS 2024. - Lin et al. (2022) Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022), pages 2129–2144. - Liu et al. (2024) Dancheng Liu, Amir Nassereldine, Ziming Yang, Chenhui Xu, Yuting Hu, Jiajie Li, Utkarsh Kumar, Changjae Lee, Ruiyang Qin, Yiyu Shi, and 1 others. 2024. Large language models have intrinsic self-correction ability. arXiv preprint arXiv:2406.15673. - Ma et al. (2025) Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, and Jia Li. 2025. S 2 R: Teaching LLMs to self-verify and self-correct via reinforcement learning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22632–22654. - Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. - Pan and et al. (2023) Xinyu Pan and et al. 2023. Multiple rationales for multiple-choice question answering. arXiv preprint arXiv:2305.03495. - Polo et al. (2024) Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. 2024. tinybenchmarks: evaluating llms with fewer examples. arXiv preprint arXiv:2402.14992. - Pryzant (2023) Ryan Pryzant. 2023. Automatic prompt optimization with "gradient descent" for language models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 494–507. - Qwen et al. (2025) Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. - Shinn et al. (2023) Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652. - Spataru (2024) Ava Spataru. 2024. Know when to stop: A study of semantic drift in text generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3656–3671. - Sprague et al. (2025) Zayne Rea Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. 2025. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. In The Thirteenth International Conference on Learning Representations. - Suzgun and Kalai (2024) Mirac Suzgun and Adam Tauman Kalai. 2024. Meta-prompting: Enhancing language models with task-agnostic scaffolding. arXiv preprint arXiv:2401.12954. - Suzgun et al. (2025) Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. 2025. Dynamic cheatsheet: Test-time learning with adaptive memory. arXiv preprint arXiv:2504.07952. - Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations. - Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. - Yao et al. (2023) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems. - Zelikman et al. (2022) Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. STar: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems. - Zheng et al. (2023) Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623. - Zhu and et al. (2024) Xue Zhu and et al. 2024. Mcrepair: Enhancing multiple-choice reasoning with self-explanation and rescoring. arXiv preprint arXiv:2405.18711. ## Appendix A Details on Experimental Setup ### A.1 Details on Final Answer Extraction For all of our problems, we added a short phrase to the text of the question to guide the model to give the final answer in a clear format: “ provide your final answer after the ‘The final answer is: ’.” To extract the answer, we split the output of the model using this phrase and take what comes after it. Since models sometimes change the phrase slightly, we also check for different variations until one is found: “The answer is: ”, “The answer is ”. “The final answer is: ”, “The final answer is ”. Once we get the final answer, we clean it up with a few simple steps: 1. If the answer is inside symbols like boxed, text, texttt, or wrapped in **, we remove those and keep only the text inside. 1. For multiple-choice questions, if the model adds extra text after the final answer (for example, by putting a newline \n), we split on \n and keep only the first part. We then lowercase both the final answer and the label, and then check the correctness with the following rules: - If the final answer and label are identical, we consider the final answer correct. - If they only differ by quotes or brackets around the answer, we consider it to be correct. - For multiple-choice questions, the label is in the format (<LETTER>). If the model only gives the letter (like A instead of (A)), we still count it as correct. ### A.2 Prompts #### A.2.1 Start Prompts Baseline Question: {question}. Provide your final answer after the ‘The final answer is: ’. Chain-of-Thought (CoT) Question: {question}. Think step by step, and provide your final answer after the ‘The final answer is: ’. #### A.2.2 Iterative (Self-Correction) Prompts Baseline Question: {question}. Review your previous responses, and provide your final answer after the ‘The final answer is: ’. Chain-of-Thought (CoT) Question: {question}. Review your previous responses, think step by step and provide your final answer after the ‘The final answer is: ’. ## Appendix B Evaluation Protocol Given the differences between task formats, we adopt distinct evaluation strategies tailored to the characteristics of each setting—open-ended generation and multiple-choice questions. For multiple-choice questions, we use Soft Match (SM) Suzgun and Kalai (2024); Suzgun et al. (2025), a lenient metric that considers an answer correct if the ground-truth label appears in the model’s output, disregarding minor formatting variations such as punctuation or whitespace. For open-ended generation, we employ the LLM-as-a-Judge Zheng et al. (2023) approach to assess the correctness of the generated answers relative to the ground-truth responses for each dataset. Specifically, we use the fine-tuned model https://github.com/yizhongw/truthfulqa_reeval introduced by A2i for evaluating generations on tinyTruthfulQA. For DisambiguationQA, we prompt a large model, GPT-4o, by providing the question, the model-generated answer, and the reference answer, asking it to determine whether the generated answer is correct. The exact prompt used for DisambiguationQA evaluation is shown below: Evaluation Prompt You are an expert in answer correctness evaluation. Given a question, its reference answer, and a generated answer, please evaluate the correctness of the generated answer based on the question and the reference answer. Here are the question, reference answer, and generated answer: - Question: {question} - Reference Answer: {gold answer} - Generated Answer: {generated answer} Please assess the correctness of the generated answer by considering the question and comparing it against the reference answer. Return yes if the generated answer is completely correct, otherwise, return ‘no’. The final answer must only be ‘yes’ or ‘no’, corresponding to the correctness of the generated answer. ## Appendix C Additional Experiments and Results <details> <summary>x20.png Details</summary>  ### Visual Description ## Bar Chart: Model Accuracy Comparison (Generation vs Multiple-choice) ### Overview The chart compares the accuracy of two methods—Generation and Multiple-choice—across six AI models: DeepSeek-R1, Llama-3-1-8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, and Gemini-2.0-Flash. Accuracy is measured in percentage, with values ranging from 0.0% to 0.6%. ### Components/Axes - **X-axis**: Model names (DeepSeek-R1, Llama-3-1-8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, Gemini-2.0-Flash). - **Y-axis**: Accuracy (%) from 0.0 to 0.6 in increments of 0.1. - **Legend**: - Blue bars = Generation - Orange bars = Multiple-choice - **Title**: Not explicitly visible in the image. ### Detailed Analysis 1. **DeepSeek-R1**: - Generation: ~0.23% (blue) - Multiple-choice: ~0.40% (orange) 2. **Llama-3-1-8B**: - Generation: ~0.30% (blue) - Multiple-choice: ~0.54% (orange) 3. **Qwen2.5-14B**: - Generation: ~0.48% (blue) - Multiple-choice: ~0.53% (orange) 4. **Qwen2.5-3B**: - Generation: ~0.33% (blue) - Multiple-choice: ~0.45% (orange) 5. **SmolLM2-1.7B**: - Generation: ~0.07% (blue) - Multiple-choice: ~0.36% (orange) 6. **Gemini-2.0-Flash**: - Generation: ~0.42% (blue) - Multiple-choice: ~0.57% (orange) ### Key Observations - **Trend Verification**: - Multiple-choice consistently outperforms Generation across all models. - The largest gap occurs in SmolLM2-1.7B (Generation: ~0.07%, Multiple-choice: ~0.36%). - Gemini-2.0-Flash shows the highest accuracy for both methods (~0.42% Generation, ~0.57% Multiple-choice). - **Outliers**: - SmolLM2-1.7B has the lowest Generation accuracy (~0.07%), significantly lower than other models. - Qwen2.5-14B has the highest Generation accuracy (~0.48%) but a smaller gap compared to Multiple-choice (~0.53%). ### Interpretation The data suggests that **Multiple-choice methods generally achieve higher accuracy than Generation** across diverse AI models. This could indicate that Multiple-choice frameworks are more robust or better aligned with evaluation criteria. However, the stark underperformance of Generation in SmolLM2-1.7B raises questions about model-specific limitations or training data quality. Gemini-2.0-Flash emerges as the strongest performer overall, suggesting advanced architecture or optimization. The results highlight the need for method-specific optimizations, particularly for smaller models like SmolLM2-1.7B. </details> (a) Baseline <details> <summary>x21.png Details</summary>  ### Visual Description ## Bar Chart: Model Accuracy Comparison ### Overview The chart compares the accuracy of six AI models (DeepSeek-R1, Llama-3-1-8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, Gemini-2.0-Flash) across two tasks: **Generation** (blue bars) and **Multiple-choice** (orange bars). Accuracy is measured in percentage, with values ranging from 0% to 0.5% on the y-axis. ### Components/Axes - **X-axis**: Model names (DeepSeek-R1, Llama-3-1-8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, Gemini-2.0-Flash). - **Y-axis**: Accuracy (%) from 0.0 to 0.5, with increments of 0.1. - **Legend**: - Blue = Generation - Orange = Multiple-choice - **Bar Placement**: Paired bars (Generation and Multiple-choice) are centered under each model label. ### Detailed Analysis - **DeepSeek-R1**: - Generation: ~0.2% - Multiple-choice: ~0.35% - **Llama-3-1-8B**: - Generation: ~0.32% - Multiple-choice: ~0.55% - **Qwen2.5-14B**: - Generation: ~0.45% - Multiple-choice: ~0.53% - **Qwen2.5-3B**: - Generation: ~0.29% - Multiple-choice: ~0.40% - **SmolLM2-1.7B**: - Generation: ~0.10% - Multiple-choice: ~0.40% - **Gemini-2.0-Flash**: - Generation: ~0.49% - Multiple-choice: ~0.53% ### Key Observations 1. **Multiple-choice tasks consistently outperform Generation tasks** across all models (e.g., Llama-3-1-8B: 0.55% vs. 0.32%). 2. **Qwen2.5-14B** achieves the highest accuracy in both tasks (~0.45% Generation, ~0.53% Multiple-choice). 3. **SmolLM2-1.7B** has the lowest Generation accuracy (~0.10%), despite matching Qwen2.5-3B in Multiple-choice. 4. **Gemini-2.0-Flash** performs strongly in both tasks (~0.49% Generation, ~0.53% Multiple-choice), suggesting efficiency. ### Interpretation The data suggests that **Multiple-choice tasks are inherently easier for these models**, likely due to structured answer formats reducing ambiguity. Larger models (e.g., Qwen2.5-14B, Gemini-2.0-Flash) generally excel, but smaller models like SmolLM2-1.7B underperform in Generation, indicating that model size alone does not guarantee task proficiency. The narrow gap between Generation and Multiple-choice accuracy for Gemini-2.0-Flash highlights its robustness in handling open-ended tasks. </details> (b) CoT <details> <summary>x22.png Details</summary>  ### Visual Description ## Bar Chart: Model Performance Comparison: Generation vs. Multiple-choice Accuracy ### Overview The chart compares the accuracy of six AI models (DeepSeek-R1, Llama-3-1.8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, Gemini-2.0-Flash) across two tasks: **Generation** (blue bars) and **Multiple-choice** (orange bars). Accuracy is measured on a 0–0.5 scale, with higher values indicating better performance. ### Components/Axes - **X-axis**: Model names (DeepSeek-R1, Llama-3-1.8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, Gemini-2.0-Flash), ordered left to right. - **Y-axis**: Accuracy (%) from 0.0 to 0.5 in increments of 0.1. - **Legend**: - Blue = Generation - Orange = Multiple-choice - **Bar Placement**: For each model, two bars are grouped side-by-side (blue left, orange right). ### Detailed Analysis 1. **DeepSeek-R1**: - Generation: ~0.22 - Multiple-choice: ~0.45 2. **Llama-3-1.8B**: - Generation: ~0.38 - Multiple-choice: ~0.46 3. **Qwen2.5-14B**: - Generation: ~0.41 - Multiple-choice: ~0.52 4. **Qwen2.5-3B**: - Generation: ~0.33 - Multiple-choice: ~0.48 5. **SmolLM2-1.7B**: - Generation: ~0.05 - Multiple-choice: ~0.23 6. **Gemini-2.0-Flash**: - Generation: ~0.45 - Multiple-choice: ~0.52 ### Key Observations - **Multiple-choice consistently outperforms Generation** across all models (orange bars are taller than blue bars). - **Gemini-2.0-Flash** achieves the highest accuracy in both tasks (~0.45 for Generation, ~0.52 for Multiple-choice). - **SmolLM2-1.7B** has the lowest performance, particularly in Generation (~0.05). - The **performance gap** between tasks varies: Gemini-2.0-Flash has a 0.07 difference, while SmolLM2-1.7B has a 0.18 difference. ### Interpretation The data suggests that **Multiple-choice tasks are structurally easier** for these models than open-ended Generation tasks. This aligns with the hypothesis that models excel at pattern recognition in constrained formats (e.g., selecting from predefined options) but struggle with creative or context-dependent outputs. Gemini-2.0-Flash’s dominance in both categories indicates superior architectural design or training data quality. Conversely, SmolLM2-1.7B’s poor Generation performance highlights limitations in handling unstructured tasks, possibly due to smaller model size or less robust training. The trend underscores the need for specialized architectures to bridge the gap between task types. </details> (c) SC <details> <summary>x23.png Details</summary>  ### Visual Description ## Bar Chart: Accuracy Comparison of Generation vs. Multiple-choice Methods Across AI Models ### Overview The chart compares the accuracy of two methods—**Generation** (blue bars) and **Multiple-choice** (orange bars)—across six AI models. Accuracy is measured in percentage, with values ranging from 0% to 0.9%. The legend at the bottom distinguishes the two methods by color. ### Components/Axes - **X-axis**: AI models (categories): - DeepSeek-R1 - Llama-3-1-8B - Qwen2-5-14B - Qwen2.5-3B - SmolLM2-1.7B - Gemini-2.0-Flash - **Y-axis**: Accuracy (%) with a scale from 0.0 to 0.9. - **Legend**: - Blue = Generation - Orange = Multiple-choice - **Spatial Grounding**: - Legend is positioned at the bottom center. - Bars are grouped by model, with blue (Generation) on the left and orange (Multiple-choice) on the right for each category. ### Detailed Analysis 1. **DeepSeek-R1**: - Generation: ~0.85% (blue) - Multiple-choice: ~0.62% (orange) 2. **Llama-3-1-8B**: - Generation: ~0.87% (blue) - Multiple-choice: ~0.71% (orange) 3. **Qwen2-5-14B**: - Generation: ~0.90% (blue) - Multiple-choice: ~0.81% (orange) 4. **Qwen2.5-3B**: - Generation: ~0.84% (blue) - Multiple-choice: ~0.75% (orange) 5. **SmolLM2-1.7B**: - Generation: ~0.58% (blue) - Multiple-choice: ~0.15% (orange) 6. **Gemini-2.0-Flash**: - Generation: ~0.85% (blue) - Multiple-choice: ~0.90% (orange) ### Key Observations - **Trend Verification**: - Generation (blue) consistently outperforms Multiple-choice (orange) across all models except **Gemini-2.0-Flash**, where Multiple-choice slightly exceeds Generation. - The largest gap between methods occurs in **SmolLM2-1.7B**, where Generation is ~0.58% vs. Multiple-choice at ~0.15%. - The highest accuracy for Generation is **Qwen2-5-14B** (~0.90%), while the highest for Multiple-choice is **Gemini-2.0-Flash** (~0.90%). ### Interpretation - **Method Effectiveness**: - Generation methods generally achieve higher accuracy, suggesting they are better suited for tasks requiring nuanced or open-ended responses. - Multiple-choice methods lag significantly in smaller models (e.g., SmolLM2-1.7B), indicating potential limitations in handling complex reasoning without predefined options. - **Model-Specific Anomalies**: - **Gemini-2.0-Flash** is the only model where Multiple-choice surpasses Generation, possibly due to its architecture being optimized for structured tasks. - Larger models (e.g., Qwen2-5-14B) show diminishing returns in the Generation vs. Multiple-choice gap, implying scalability benefits for both methods. - **Practical Implications**: - For high-stakes applications (e.g., medical diagnosis), Generation methods may be preferred for their adaptability. - Multiple-choice could be viable for resource-constrained environments if accuracy thresholds are met (e.g., Gemini-2.0-Flash). ### Uncertainties - Values are approximate due to the lack of precise numerical labels on the bars. - The chart does not specify the dataset or task type, which could influence the observed trends. </details> (d) Baseline <details> <summary>x24.png Details</summary>  ### Visual Description ## Bar Chart: Model Performance Comparison (Generation vs Multiple-choice Accuracy) ### Overview The chart compares the accuracy of six AI models across two tasks: Generation and Multiple-choice. Models are listed on the x-axis, with accuracy percentages (0-100%) on the y-axis. Blue bars represent Generation accuracy, while orange bars represent Multiple-choice accuracy. ### Components/Axes - **X-axis**: Model names (DeepSeek-R1, Llama-3-1-8B, Qwen2.5-14B, Qwen2.5-3B, SmolLM2-1.7B, Gemini-2.0-Flash) - **Y-axis**: Accuracy (%) from 0.0 to 0.8 in 0.2 increments - **Legend**: - Blue = Generation - Orange = Multiple-choice - **Legend Position**: Bottom center ### Detailed Analysis 1. **DeepSeek-R1** - Generation: ~0.85 - Multiple-choice: ~0.6 2. **Llama-3-1-8B** - Generation: ~0.82 - Multiple-choice: ~0.7 3. **Qwen2.5-14B** - Generation: ~0.83 - Multiple-choice: ~0.78 4. **Qwen2.5-3B** - Generation: ~0.88 - Multiple-choice: ~0.65 5. **SmolLM2-1.7B** - Generation: ~0.67 - Multiple-choice: ~0.2 6. **Gemini-2.0-Flash** - Generation: ~0.92 - Multiple-choice: ~0.85 ### Key Observations - **Trend Verification**: - Generation accuracy consistently exceeds Multiple-choice for all models except Gemini-2.0-Flash (where both are high). - Qwen2.5-3B shows the largest gap between tasks (0.88 vs 0.65). - SmolLM2-1.7B has the lowest Multiple-choice accuracy (0.2), creating an outlier. ### Interpretation 1. **Task Performance**: Generation tasks generally show higher accuracy across models, suggesting they may be better suited to these architectures or training objectives. 2. **Model Specialization**: Gemini-2.0-Flash dominates both tasks, indicating superior design or training for complex reasoning. 3. **Outlier Analysis**: SmolLM2-1.7B's drastic drop in Multiple-choice accuracy (0.2 vs 0.67 Generation) suggests potential limitations in handling structured reasoning tasks. 4. **Model Size Correlation**: Larger models (e.g., Gemini-2.0-Flash, Qwen2.5-14B) tend to perform better in both tasks, though exceptions exist (Qwen2.5-3B underperforms in Multiple-choice despite high Generation accuracy). ### Technical Implications - The data highlights trade-offs between task types and model capabilities. - Gemini-2.0-Flash's performance suggests it may be optimized for both open-ended and constrained reasoning. - SmolLM2-1.7B's results warrant investigation into architectural constraints or training data biases affecting Multiple-choice performance. </details> (e) CoT <details> <summary>x25.png Details</summary>  ### Visual Description ## Bar Chart: Model Accuracy Comparison (Generation vs Multiple-choice) ### Overview The chart compares the accuracy performance of two methods ("Generation" and "Multiple-choice") across seven AI models. Accuracy is measured in percentage, with values ranging from 0% to 80% on the y-axis. The x-axis lists model names, and the legend distinguishes the two methods by color (blue for Generation, orange for Multiple-choice). ### Components/Axes - **X-axis (Models)**: - DeepSeek-R1 - Llama-3-1-8B - Qwen2-5-14B - Qwen2-5-3B - SmolLM2-1.7B - Gemini-2.0-Flash - DistilLlama-8B - **Y-axis (Accuracy %)**: - Scale: 0.0 to 0.8 in increments of 0.2 - Labels: "Accuracy (%)" - **Legend**: - Position: Bottom center - Colors: - Blue = Generation - Orange = Multiple-choice ### Detailed Analysis 1. **DeepSeek-R1**: - Generation: ~85% (blue bar) - Multiple-choice: ~68% (orange bar) 2. **Llama-3-1-8B**: - Generation: ~75% (blue bar) - Multiple-choice: ~74% (orange bar) 3. **Qwen2-5-14B**: - Generation: ~81% (blue bar) - Multiple-choice: ~76% (orange bar) 4. **Qwen2-5-3B**: - Generation: ~87% (blue bar) - Multiple-choice: ~71% (orange bar) 5. **SmolLM2-1.7B**: - Generation: ~47% (blue bar) - Multiple-choice: ~20% (orange bar) 6. **Gemini-2.0-Flash**: - Generation: ~90% (blue bar) - Multiple-choice: ~86% (orange bar) 7. **DistilLlama-8B**: - Generation: ~78% (blue bar) - Multiple-choice: ~72% (orange bar) ### Key Observations - **Consistent Outperformance**: Generation methods consistently outperform Multiple-choice across all models, with accuracy gaps ranging from 5% (Llama-3-1-8B) to 30% (SmolLM2-1.7B). - **SmolLM2-1.7B Anomaly**: This model shows the largest disparity between methods (27% gap), with Generation at 47% and Multiple-choice at 20%. - **Gemini-2.0-Flash Exception**: Despite being the highest-performing model overall, its Multiple-choice accuracy (86%) is nearly equal to its Generation accuracy (90%), suggesting near-parity in this case. - **Low Baseline**: SmolLM2-1.7B has the lowest accuracy for both methods, indicating potential limitations in model size or training data. ### Interpretation The data demonstrates that **Generation methods significantly outperform Multiple-choice approaches** in most models, particularly in larger architectures like Gemini-2.0-Flash and Qwen2-5-3B. The exception with Gemini-2.0-Flash suggests that for highly capable models, Multiple-choice may approach Generation performance. However, SmolLM2-1.7B's poor performance across both methods highlights challenges in smaller models. This trend implies that Generation methods may be more robust or adaptable to diverse tasks, while Multiple-choice approaches might struggle with complex reasoning or domain-specific knowledge. The near-parity in Gemini-2.0-Flash warrants further investigation into whether Multiple-choice could be optimized for specific use cases in high-capacity models. </details> (f) SC Figure 5: Cumulative accuracy (after final self-correction iteration) using different models on (top) DisambiguationQA and (bottom) tinyTruthfulQA. The results indicate that models perform completely differently on self-correction of generation and multiple-choice questions, depending on the dataset. ### C.1 Results on Correct and Incorrect Flips Figures 6-11 show the correct and incorrect flips on different datasets and models. <details> <summary>x26.png Details</summary>  ### Visual Description ## Line Chart: SmolLM2-1.7B Performance Over Iterations ### Overview The chart visualizes the proportion of correct and incorrect flips for two tasks ("Generation" and "Multiple-Choice") across five iterations of a model. Data is represented by solid and dashed lines with corresponding markers. ### Components/Axes - **X-axis**: Iterations (labeled 1 to 5) - **Y-axis**: Proportion of Flips (ranging from 0.00 to 0.10) - **Legend**: - Top-left corner - Blue line: "Generation" (solid = correct flips, dashed = incorrect flips) - Orange line: "Multiple-Choice" (solid = correct flips, dashed = incorrect flips) ### Detailed Analysis 1. **Iteration 1**: - Generation: Correct ≈ 0.005, Incorrect ≈ 0.002 - Multiple-Choice: Correct ≈ 0.085, Incorrect ≈ 0.003 2. **Iteration 2**: - Generation: Correct ≈ 0.010, Incorrect ≈ 0.008 - Multiple-Choice: Correct ≈ 0.030, Incorrect ≈ 0.005 3. **Iteration 3**: - Generation: Correct ≈ 0.015, Incorrect ≈ 0.007 - Multiple-Choice: Correct ≈ 0.035, Incorrect ≈ 0.002 4. **Iteration 4**: - Generation: Correct ≈ 0.012, Incorrect ≈ 0.006 - Multiple-Choice: Correct ≈ 0.025, Incorrect ≈ 0.004 5. **Iteration 5**: - Generation: Correct ≈ 0.018, Incorrect ≈ 0.009 - Multiple-Choice: Correct ≈ 0.035, Incorrect ≈ 0.005 ### Key Observations - **Multiple-Choice** starts with a high proportion of correct flips (0.085 at Iteration 1) but declines sharply by Iteration 3, then stabilizes. - **Generation** shows a gradual increase in correct flips (from 0.005 to 0.018) with minor fluctuations. - Incorrect flips for both tasks remain low (<0.01) but exhibit slight upward trends over iterations. - Lines intersect around Iteration 3, indicating parity in performance between tasks at this point. ### Interpretation The data suggests that the model's performance improves for **Generation** tasks over iterations, while **Multiple-Choice** performance deteriorates initially before stabilizing. This divergence may reflect task-specific challenges: Generation tasks might benefit from iterative refinement, whereas Multiple-Choice could involve overfitting or data distribution shifts. The low but rising incorrect flips hint at emerging errors as the model adapts, potentially due to increased complexity or noise in later iterations. </details> (a) SmolLM2-1.7B <details> <summary>x27.png Details</summary>  ### Visual Description ## Line Graph: Qwen2.5-3B Performance Analysis ### Overview The image is a line graph titled "Qwen2.5-3B," comparing the proportion of flips (correct and incorrect) across iterations for two methods: "Generation" and "Multiple-Choice." The y-axis represents the "Proportion of Flips" (ranging from 0.02 to 0.14), and the x-axis represents "Iterations" (1 to 5). The graph includes a legend distinguishing "Correct Flip" (solid line) and "Incorrect Flip" (dashed line), though the lines are labeled as "Generation" (solid blue) and "Multiple-Choice" (dashed orange). ### Components/Axes - **Title**: "Qwen2.5-3B" - **Y-Axis**: "Proportion of Flips" (0.02 to 0.14, increments of 0.02) - **X-Axis**: "Iterations" (1 to 5, labeled numerically) - **Legend**: - "Correct Flip" (solid line, black) - "Incorrect Flip" (dashed line, black) - **Data Series**: - "Generation" (solid blue line) - "Multiple-Choice" (dashed orange line) ### Detailed Analysis #### Generation (Solid Blue Line) - **Iteration 1**: ~0.10 - **Iteration 2**: ~0.03 - **Iteration 3**: ~0.08 - **Iteration 4**: ~0.05 - **Iteration 5**: ~0.06 - **Trend**: The line starts at 0.10, drops sharply to 0.03 at iteration 2, rises to 0.08 at iteration 3, then declines to 0.05 at iteration 4, and slightly increases to 0.06 at iteration 5. The trend shows volatility with no clear upward or downward trajectory. #### Multiple-Choice (Dashed Orange Line) - **Iteration 1**: ~0.12 - **Iteration 2**: ~0.14 - **Iteration 3**: ~0.04 - **Iteration 4**: ~0.08 - **Iteration 5**: ~0.04 - **Trend**: The line begins at 0.12, peaks at 0.14 at iteration 2, then drops to 0.04 at iteration 3, rises to 0.08 at iteration 4, and falls to 0.04 at iteration 5. The trend shows a sharp initial increase followed by a decline and partial recovery. ### Key Observations 1. **Generation Method**: - The proportion of flips (likely correct, as per the solid line) starts at 0.10, dips significantly at iteration 2, and fluctuates thereafter. The lowest point is at iteration 2 (~0.03), suggesting a potential instability or adjustment phase. 2. **Multiple-Choice Method**: - The proportion of flips (likely incorrect, as per the dashed line) starts at 0.12, peaks at iteration 2 (~0.14), then declines sharply to 0.04 at iteration 3, with a partial recovery to 0.08 at iteration 4 before dropping again. This indicates a high initial error rate that improves over time. ### Interpretation - **Data Relationships**: - The "Generation" method (solid blue) shows a more variable performance in terms of flip proportions, with a notable dip at iteration 2. This could reflect challenges in stabilizing the model's output during early iterations. - The "Multiple-Choice" method (dashed orange) demonstrates a high initial error rate (incorrect flips) that decreases significantly by iteration 3, suggesting improved accuracy or reduced errors over time. - **Notable Patterns**: - The "Generation" method's fluctuations might indicate sensitivity to iterative adjustments, while the "Multiple-Choice" method's sharp decline in incorrect flips suggests a more stable or optimized performance after initial iterations. - **Anomalies**: - The "Generation" method's sharp drop at iteration 2 (from 0.10 to 0.03) is a significant outlier, potentially indicating a critical adjustment or error in the model's behavior during that phase. - The "Multiple-Choice" method's peak at iteration 2 (0.14) followed by a steep decline highlights a possible initial misconfiguration or overfitting that was later corrected. This analysis suggests that the "Multiple-Choice" method may outperform the "Generation" method in terms of stability and error reduction over iterations, though further context (e.g., task specifics, evaluation metrics) would be needed to confirm these trends. </details> (b) Qwen2.5-3B <details> <summary>x28.png Details</summary>  ### Visual Description ## Line Chart: Llama-3.1-8B Performance Over Iterations ### Overview The chart illustrates the proportion of "Flips" (likely model predictions changing from one answer to another) for two methods—**Generation** and **Multiple-Choice**—across five iterations. The y-axis represents the proportion of flips (0.04 to 0.14), while the x-axis tracks iterations (1 to 5). Two lines are plotted: a solid blue line for **Generation** and a dashed orange line for **Multiple-Choice**, with markers indicating "Correct Flip" (solid circle) and "Incorrect Flip" (dashed square). --- ### Components/Axes - **Title**: "Llama-3.1-8B" (top center). - **X-Axis**: Labeled "Iterations" with discrete values 1, 2, 3, 4, 5. - **Y-Axis**: Labeled "Proportion of Flips" with a scale from 0.04 to 0.14. - **Legend**: - **Generation**: Solid blue line with solid circle markers (top-right). - **Multiple-Choice**: Dashed orange line with dashed square markers (top-right). - **Correct Flip**: Solid circle (black). - **Incorrect Flip**: Dashed square (black). - **Data Points**: - **Generation** (blue): Solid circles at each iteration. - **Multiple-Choice** (orange): Dashed squares at each iteration. --- ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.11 proportion of flips. - **Iteration 2**: ~0.07 (lowest point). - **Iteration 3**: ~0.10 (peak). - **Iteration 4**: ~0.05 (lowest). - **Iteration 5**: ~0.07 (moderate recovery). #### Multiple-Choice (Orange Line) - **Iteration 1**: ~0.10. - **Iteration 2**: ~0.14 (highest peak). - **Iteration 3**: ~0.07 (sharp drop). - **Iteration 4**: ~0.11 (moderate recovery). - **Iteration 5**: ~0.03 (steep decline). --- ### Key Observations 1. **Generation** shows moderate stability, with fluctuations between ~0.05 and ~0.11. 2. **Multiple-Choice** exhibits high volatility, peaking at iteration 2 (~0.14) and crashing to ~0.03 by iteration 5. 3. **Legend Confusion**: The legend labels "Correct Flip" and "Incorrect Flip" with symbols (solid circle and dashed square), but these symbols are not explicitly plotted on the chart. The lines themselves (solid blue/orange) are labeled as "Generation" and "Multiple-Choice," suggesting a potential mismatch in the legend's design. --- ### Interpretation - **Trend Analysis**: - **Generation** maintains a relatively stable performance, with minor dips and recoveries. This suggests consistent behavior across iterations. - **Multiple-Choice** shows erratic behavior, with a sharp decline in later iterations. This could indicate overfitting, sensitivity to input changes, or instability in the method's logic. - **Legend Clarification**: The legend's "Correct Flip" and "Incorrect Flip" labels may refer to the markers (solid/dashed) rather than the lines. However, the chart does not visually distinguish between correct/incorrect flips beyond the line styles. This ambiguity could lead to misinterpretation. - **Outliers**: The **Multiple-Choice** line's sharp drop at iteration 5 (~0.03) is an outlier, suggesting a critical failure or methodological flaw in that iteration. --- ### Conclusion The chart highlights the performance disparity between **Generation** and **Multiple-Choice** methods in the Llama-3.1-8B model. While **Generation** demonstrates resilience, **Multiple-Choice** exhibits instability, particularly in later iterations. The legend's design may require revision to avoid confusion between line styles and flip types. </details> (c) Llama-3.1-8B <details> <summary>x29.png Details</summary>  ### Visual Description ## Line Graph: Qwen2.5-14B Performance Over Iterations ### Overview The graph illustrates the proportion of "flips" (changes in model outputs) across five iterations for four distinct strategies: Generation, Multiple-Choice, Correct Flip, and Incorrect Flip. The y-axis represents the proportion of flips (0.00–0.10), and the x-axis represents iterations (1–5). The legend is positioned at the top-right corner. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with markers at 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with increments of 0.02. - **Legend**: - Solid blue line: Generation - Dashed orange line: Multiple-Choice - Solid black line: Correct Flip - Dashed black line: Incorrect Flip ### Detailed Analysis 1. **Generation (Solid Blue Line)**: - Starts at ~0.08 (iteration 1), drops sharply to ~0.02 (iteration 2), then stabilizes around ~0.03–0.04 (iterations 3–5). - **Trend**: Steep initial decline followed by stabilization. 2. **Multiple-Choice (Dashed Orange Line)**: - Begins at ~0.04 (iteration 1), decreases to ~0.02 (iteration 2), then plummets to ~0.00 (iteration 3), remaining near 0.00 for iterations 4–5. - **Trend**: Rapid decline after iteration 2, becoming negligible by iteration 3. 3. **Correct Flip (Solid Black Line)**: - Starts at ~0.02 (iteration 1), peaks at ~0.06 (iteration 2), then declines to ~0.01 (iteration 5). - **Trend**: Early peak followed by a gradual decline. 4. **Incorrect Flip (Dashed Black Line)**: - Begins at ~0.06 (iteration 1), drops to ~0.01 (iteration 2), then stabilizes near 0.00–0.01 (iterations 3–5). - **Trend**: Sharp initial drop, followed by minimal fluctuation. ### Key Observations - **Generation** and **Incorrect Flip** exhibit the most significant early declines, suggesting reduced reliance on these strategies as iterations progress. - **Multiple-Choice** becomes nearly irrelevant after iteration 3, dropping to 0.00. - **Correct Flip** peaks at iteration 2 (~0.06), indicating a temporary increase in accurate adjustments before stabilizing. - All lines converge toward lower values by iteration 5, implying improved model consistency over time. ### Interpretation The data suggests that the Qwen2.5-14B model refines its decision-making process across iterations. The steep decline in **Generation** and **Incorrect Flip** indicates reduced dependency on error-prone or non-deterministic outputs. The near-elimination of **Multiple-Choice** flips implies the model moves away from relying on probabilistic or heuristic-based reasoning. The early peak in **Correct Flip** may reflect initial adjustments to align outputs with expected patterns, followed by stabilization as the model optimizes further. Overall, the trends highlight iterative improvements in output reliability, with later iterations showing fewer deviations (flips) across all strategies. </details> (d) Qwen2.5-14B <details> <summary>x30.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Across Iterations for DeepSeek-R1-Distill-Llama-8B ### Overview The chart illustrates the proportion of "flips" (likely model output changes) across five iterations for two methods: "Generation" (blue line) and "Multiple-Choice" (orange line). A legend indicates "Correct Flip" (solid) and "Incorrect Flip" (dashed), though these are not directly plotted in the chart. The y-axis represents the proportion of flips (0.00 to 0.08), and the x-axis represents iterations (1 to 5). ### Components/Axes - **Title**: "DeepSeek-R1-Distill-Llama-8B" - **Y-Axis**: "Proportion of Flips" (0.00 to 0.08, linear scale) - **X-Axis**: "Iterations" (1 to 5, integer labels) - **Legend**: - "Generation" (blue solid line) - "Multiple-Choice" (orange dashed line) - "Correct Flip" (solid black) - "Incorrect Flip" (dashed black) - **Data Points**: - Blue squares (Generation) - Orange squares (Multiple-Choice) ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.03 - **Iteration 2**: ~0.02 - **Iteration 3**: ~0.03 - **Iteration 4**: ~0.00 (notable drop) - **Iteration 5**: ~0.02 - **Trend**: Initial decline, followed by a sharp drop at iteration 4, then a slight recovery. #### Multiple-Choice (Orange Line) - **Iteration 1**: ~0.06 - **Iteration 2**: ~0.07 - **Iteration 3**: ~0.05 - **Iteration 4**: ~0.04 - **Iteration 5**: ~0.05 - **Trend**: Steady decline with a minor rebound at iteration 5. #### Legend and Data Point Alignment - The legend labels "Correct Flip" and "Incorrect Flip" are not directly represented in the chart. This may indicate a misalignment or omission in the visualization. The blue and orange lines correspond to "Generation" and "Multiple-Choice," respectively, as per the legend. ### Key Observations 1. **Generation Method**: Shows significant variability, with a sharp drop to 0.00 at iteration 4, suggesting a potential anomaly or model adjustment. 2. **Multiple-Choice Method**: Demonstrates a more consistent decline, with a slight increase at iteration 5, possibly indicating stabilization. 3. **Legend Discrepancy**: The "Correct Flip" and "Incorrect Flip" labels in the legend do not match the plotted data, raising questions about the chart's completeness or accuracy. ### Interpretation The data suggests that the "Generation" method exhibits higher volatility in flip proportions, particularly at iteration 4, where the proportion drops to zero. This could indicate a model failure or a deliberate reset. The "Multiple-Choice" method shows a more predictable trend, with a gradual reduction in flips, possibly reflecting a more stable or constrained decision-making process. The mismatch between the legend and the plotted data highlights a potential error in the visualization, which may require clarification or correction. The absence of "Correct Flip" and "Incorrect Flip" data points in the chart suggests that these categories might belong to a different dataset or a separate analysis not included here. </details> (e) DeepSeek-R1-Distill-Llama-8B <details> <summary>x31.png Details</summary>  ### Visual Description ## Line Chart: Gemini-2.0-Flash ### Overview The chart visualizes the proportion of "flips" (changes in model outputs) across five iterations for two methods: "Generation" (blue solid line) and "Multiple-Choice" (orange dashed line). A legend distinguishes "Correct Flip" (solid lines) and "Incorrect Flip" (dashed lines), though the chart does not explicitly separate these subcategories within the data series. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete values 1–5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.07. - **Legend**: Located in the top-right corner, with: - **Correct Flip**: Solid line (black). - **Incorrect Flip**: Dashed line (black). - **Data Series**: - **Generation**: Blue solid line. - **Multiple-Choice**: Orange dashed line. ### Detailed Analysis 1. **Generation (Blue Solid Line)**: - Iteration 1: ~0.035. - Iteration 2: Peaks at ~0.042. - Iteration 3: Drops to ~0.018. - Iteration 4: Rises to ~0.042. - Iteration 5: Stabilizes at ~0.042. - **Trend**: Fluctuates but remains relatively stable, with a slight upward trend toward the end. 2. **Multiple-Choice (Orange Dashed Line)**: - Iteration 1: Starts at ~0.042. - Iteration 2: Spikes sharply to ~0.065. - Iteration 3: Plummets to ~0.025. - Iteration 4: Remains flat at ~0.025. - Iteration 5: Drops to ~0.000. - **Trend**: High volatility, with a dramatic decline after iteration 2. ### Key Observations - The **Multiple-Choice** method exhibits extreme volatility, with a peak in iteration 2 (~0.065) and near-zero flips by iteration 5. - The **Generation** method shows moderate fluctuations but maintains a higher baseline proportion of flips (~0.035–0.042) across iterations. - The legend’s "Correct Flip" and "Incorrect Flip" labels are not visually distinguishable in the chart, as both data series use solid/black lines without explicit subcategory differentiation. ### Interpretation The data suggests that the **Multiple-Choice** method initially experiences a high rate of flips (possibly due to exploratory adjustments) but stabilizes or converges to near-zero flips by iteration 5. In contrast, the **Generation** method maintains a consistent proportion of flips, indicating a more stable but less adaptive behavior. The absence of explicit "Correct Flip" vs. "Incorrect Flip" subcategories in the chart limits direct interpretation of error rates, though the legend implies these distinctions exist in the underlying data. The sharp decline in Multiple-Choice flips may reflect model convergence or reduced uncertainty in later iterations. </details> (f) Gemini-2.0-Flash Figure 6: Models Correct and Incorrect Flips on Baseline on DisambiguationQA <details> <summary>x32.png Details</summary>  ### Visual Description ## Line Graph: SmolLM2-1.7B ### Overview The image is a line graph titled "SmolLM2-1.7B" depicting the proportion of flips (correct and incorrect) across five iterations. Two data series are plotted: "Generation" (blue solid line) and "Multiple-Choice" (orange dashed line). The y-axis represents the proportion of flips (0.00 to 0.07), and the x-axis represents iterations (1 to 5). A legend in the top-right corner distinguishes between "Correct Flip" (filled circle) and "Incorrect Flip" (open circle). ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with values 1 to 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with values from 0.00 to 0.07. - **Legend**: Located in the top-right corner. Entries include: - "Generation" (solid blue line) - "Multiple-Choice" (dashed orange line) - "Correct Flip" (filled circle) - "Incorrect Flip" (open circle) ### Detailed Analysis - **Generation (Blue Solid Line)**: - Iteration 1: ~0.02 - Iteration 2: ~0.01 - Iteration 3: ~0.005 - Iteration 4: ~0.002 - Iteration 5: ~0.005 - **Trend**: U-shaped curve, dipping to a minimum at iteration 4 before rising slightly at iteration 5. - **Multiple-Choice (Orange Dashed Line)**: - Iteration 1: ~0.06 - Iteration 2: ~0.03 - Iteration 3: ~0.04 - Iteration 4: ~0.02 - Iteration 5: ~0.01 - **Trend**: General downward trend with a peak at iteration 3. - **Markers**: - "Correct Flip" (filled circles) are consistently higher than "Incorrect Flip" (open circles) for both lines. For example: - At iteration 1, "Correct Flip" (blue) is ~0.02, while "Incorrect Flip" (blue) is ~0.01. - At iteration 3, "Correct Flip" (orange) is ~0.04, while "Incorrect Flip" (orange) is ~0.02. ### Key Observations 1. **Generation** shows a U-shaped trend, with the lowest proportion of flips at iteration 4 (~0.002) and a slight recovery at iteration 5 (~0.005). 2. **Multiple-Choice** starts with the highest proportion of flips (~0.06 at iteration 1) but decreases steadily, with a minor peak at iteration 3 (~0.04). 3. "Correct Flip" values are consistently higher than "Incorrect Flip" values for both lines, suggesting a higher frequency of correct flips over time. ### Interpretation The data suggests that both "Generation" and "Multiple-Choice" methods experience a reduction in the proportion of flips as iterations increase. However, "Multiple-Choice" begins with a significantly higher proportion of flips, which declines more sharply than "Generation." The U-shaped trend in "Generation" may indicate an initial adjustment period followed by stabilization. The consistent dominance of "Correct Flip" over "Incorrect Flip" implies that the model's performance improves in accuracy over iterations, with fewer incorrect flips. The peak in "Multiple-Choice" at iteration 3 could reflect a temporary increase in variability or a specific event in the data not explained by the graph. The overall trend highlights the importance of iteration count in refining model outputs, with "Generation" showing more stability compared to "Multiple-Choice." </details> (a) SmolLM2-1.7B <details> <summary>x33.png Details</summary>  ### Visual Description ## Line Graph: Qwen2.5-3B Performance Over Iterations ### Overview The graph compares the proportion of correct and incorrect flips for two methods ("Generation" and "Multiple-Choice") across five iterations. The y-axis represents the proportion of flips (0.02–0.10), and the x-axis represents iterations (1–5). Two lines are plotted: a blue line for "Generation" and an orange line for "Multiple-Choice." The legend distinguishes "Correct Flip" (solid line with circle markers) and "Incorrect Flip" (dashed line with square markers), though the graph uses solid lines for both methods. ### Components/Axes - **Title**: "Qwen2.5-3B" - **X-axis**: "Iterations" (labeled 1–5) - **Y-axis**: "Proportion of Flips" (scaled from 0.02 to 0.10) - **Legend**: - "Correct Flip": Solid line with circle markers (blue for "Generation," orange for "Multiple-Choice") - "Incorrect Flip": Dashed line with square markers (blue for "Generation," orange for "Multiple-Choice") - **Line Styles**: Both methods use solid lines, but markers differentiate correct/incorrect flips. ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: - Correct Flip (circle): ~0.05 - Incorrect Flip (square): ~0.07 - **Iteration 2**: - Correct Flip: ~0.08 - Incorrect Flip: ~0.04 - **Iteration 3**: - Correct Flip: ~0.03 - Incorrect Flip: ~0.05 - **Iteration 4**: - Correct Flip: ~0.01 - Incorrect Flip: ~0.05 - **Iteration 5**: - Correct Flip: ~0.005 - Incorrect Flip: ~0.04 #### Multiple-Choice (Orange Line) - **Iteration 1**: - Correct Flip: ~0.07 - Incorrect Flip: ~0.06 - **Iteration 2**: - Correct Flip: ~0.04 - Incorrect Flip: ~0.04 - **Iteration 3**: - Correct Flip: ~0.05 - Incorrect Flip: ~0.04 - **Iteration 4**: - Correct Flip: ~0.05 - Incorrect Flip: ~0.06 - **Iteration 5**: - Correct Flip: ~0.04 - Incorrect Flip: ~0.03 ### Key Observations 1. **Generation Method**: - Correct flips decline sharply from ~0.05 (Iteration 1) to ~0.005 (Iteration 5). - Incorrect flips decrease modestly from ~0.07 to ~0.04. - The blue line shows a U-shaped trend, peaking at Iteration 2 (~0.08 correct flips) before dropping. 2. **Multiple-Choice Method**: - Correct flips fluctuate between ~0.04 and ~0.07, with no clear trend. - Incorrect flips remain relatively stable (~0.03–0.06). - The orange line shows minor oscillations but no significant upward or downward trajectory. 3. **Legend Discrepancy**: - The legend indicates "Correct Flip" as a solid line with circles and "Incorrect Flip" as a dashed line with squares. However, both methods use solid lines, suggesting a potential inconsistency in the legend's line style representation. ### Interpretation - **Performance Trends**: The Generation method exhibits a significant decline in correct flips over iterations, while Multiple-Choice maintains stability. This suggests Generation may be more sensitive to iterative changes, whereas Multiple-Choice is robust. - **Error Dynamics**: For Generation, incorrect flips decrease but not as sharply as correct flips, indicating partial error correction. Multiple-Choice's incorrect flips remain consistent, implying a balanced error profile. - **Legend Clarity**: The mismatch between the legend's line styles (solid/dashed) and the graph's solid lines for both methods may cause confusion. The markers (circles/squares) are the primary indicators of correct/incorrect flips. This analysis highlights the need for clearer legend alignment with visual elements to avoid misinterpretation. The data underscores the trade-offs between method stability and performance degradation over iterations. </details> (b) Qwen2.5-3B <details> <summary>x34.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in Llama-3.1-8B Across Iterations ### Overview The chart illustrates the proportion of flips (correct and incorrect) for two methods—**Generation** and **Multiple-Choice**—across five iterations. The y-axis represents the proportion of flips (0.04–0.18), while the x-axis denotes iterations (1–5). Two lines and two marker series are plotted, with distinct colors and symbols for clarity. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations," with values 1 to 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips," scaled from 0.04 to 0.18. - **Legend**: Located in the top-right corner, with: - **Correct Flip**: Black dot (dashed line). - **Incorrect Flip**: Black square (dashed line). - **Lines**: - **Generation**: Solid blue line. - **Multiple-Choice**: Solid orange line. ### Detailed Analysis #### Generation (Blue Line) - **Trend**: Starts at ~0.10 (iteration 1), decreases steadily to ~0.04 by iteration 5. - **Data Points**: - Iteration 1: ~0.10 - Iteration 2: ~0.09 - Iteration 3: ~0.08 - Iteration 4: ~0.08 - Iteration 5: ~0.04 #### Multiple-Choice (Orange Line) - **Trend**: Peaks at ~0.18 (iteration 2), then declines to ~0.08 by iteration 5, with fluctuations. - **Data Points**: - Iteration 1: ~0.16 - Iteration 2: ~0.18 - Iteration 3: ~0.12 - Iteration 4: ~0.14 - Iteration 5: ~0.08 #### Correct Flip (Black Dots) - **Trend**: Starts at ~0.09 (iteration 1), decreases to ~0.05 by iteration 5. - **Data Points**: - Iteration 1: ~0.09 - Iteration 2: ~0.08 - Iteration 3: ~0.07 - Iteration 4: ~0.08 - Iteration 5: ~0.05 #### Incorrect Flip (Black Squares) - **Trend**: Starts at ~0.01 (iteration 1), peaks at ~0.03 (iteration 3), then drops to ~0.01 by iteration 5. - **Data Points**: - Iteration 1: ~0.01 - Iteration 2: ~0.02 - Iteration 3: ~0.03 - Iteration 4: ~0.02 - Iteration 5: ~0.01 ### Key Observations 1. **Generation Method**: Shows a consistent decline in flip proportion, suggesting improved stability or reduced variability over iterations. 2. **Multiple-Choice Method**: Exhibits volatility, with a sharp peak in iteration 2 followed by a gradual decline. The highest proportion of flips occurs in iteration 2. 3. **Correct Flip**: Declines steadily, indicating a reduction in correct flips over time, which may reflect diminishing accuracy or increased errors. 4. **Incorrect Flip**: Peaks in iteration 3, suggesting a temporary spike in errors, followed by a sharp decline. ### Interpretation - The **Generation** method demonstrates a more stable and controlled performance, with a clear downward trend in flip proportion. This could imply that the model becomes more reliable or less prone to errors as iterations progress. - The **Multiple-Choice** method shows erratic behavior, with a significant peak in iteration 2. This might indicate a temporary increase in flip activity, possibly due to model adjustments or data variability. - The **Correct Flip** and **Incorrect Flip** markers reveal a trade-off: as total flips decrease (Generation), correct flips also decline, while incorrect flips remain low but spike temporarily. This suggests that the model’s accuracy may be improving, but the reduction in total flips could be due to fewer opportunities for errors rather than enhanced correctness. - The **Incorrect Flip** spike in iteration 3 highlights a potential anomaly or instability in the model’s behavior during that phase, warranting further investigation. This chart underscores the importance of balancing flip proportion with accuracy, as reducing total flips does not necessarily equate to improved performance if correct flips are also declining. </details> (c) Llama-3.1-8B <details> <summary>x35.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in Qwen2.5-14B Model Performance ### Overview The chart illustrates the proportion of correct and incorrect flips in a Qwen2.5-14B language model across five iterations, comparing two methods: "Generation" (blue line) and "Multiple-Choice" (orange line). Flips are categorized as "Correct Flip" (solid markers) and "Incorrect Flip" (dashed markers). ### Components/Axes - **X-axis**: Iterations (1 to 5, labeled at integer intervals). - **Y-axis**: Proportion of Flips (0.00 to 0.08, in increments of 0.02). - **Legend**: Located in the top-right corner, with: - Blue line: "Generation" (solid = Correct Flip, dashed = Incorrect Flip). - Orange line: "Multiple-Choice" (solid = Correct Flip, dashed = Incorrect Flip). ### Detailed Analysis 1. **Generation (Blue Line)**: - **Iteration 1**: - Correct Flip: ~0.08 (highest point). - Incorrect Flip: ~0.00 (baseline). - **Iteration 2**: - Correct Flip: ~0.04 (halved from Iteration 1). - Incorrect Flip: ~0.02 (rising trend begins). - **Iteration 3**: - Correct Flip: ~0.00 (sharp drop to baseline). - Incorrect Flip: ~0.04 (peaks at mid-range). - **Iteration 4**: - Correct Flip: ~0.02 (partial recovery). - Incorrect Flip: ~0.06 (dominant trend). - **Iteration 5**: - Correct Flip: ~0.01 (minimal improvement). - Incorrect Flip: ~0.07 (near-maximum). 2. **Multiple-Choice (Orange Line)**: - **Iteration 1**: - Correct Flip: ~0.04 (moderate start). - Incorrect Flip: ~0.00 (baseline). - **Iteration 2**: - Correct Flip: ~0.02 (declining trend). - Incorrect Flip: ~0.02 (rising trend begins). - **Iteration 3**: - Correct Flip: ~0.01 (steady decline). - Incorrect Flip: ~0.03 (moderate increase). - **Iteration 4**: - Correct Flip: ~0.00 (baseline). - Incorrect Flip: ~0.05 (sharp rise). - **Iteration 5**: - Correct Flip: ~0.01 (slight rebound). - Incorrect Flip: ~0.06 (highest point). ### Key Observations - **Generation Method**: - Dominates early iterations (Iteration 1–2) with high correct flips. - Experiences a catastrophic drop in correct flips at Iteration 3, followed by partial recovery. - Incorrect flips escalate sharply after Iteration 3, suggesting instability. - **Multiple-Choice Method**: - Shows gradual decline in correct flips across all iterations. - Incorrect flips increase consistently, peaking at Iteration 5. - **Cross-Method Comparison**: - Generation starts stronger but becomes erratic; Multiple-Choice degrades more predictably. - Both methods exhibit a correlation between rising incorrect flips and falling correct flips. ### Interpretation The data suggests that the Qwen2.5-14B model's performance deteriorates with increasing iterations for both methods, but the **Generation** method exhibits higher volatility. The sharp drop in correct flips at Iteration 3 for Generation may indicate overfitting or noise amplification in later stages. The persistent rise in incorrect flips across iterations implies a systemic issue in model stability, particularly in the Generation approach. The Multiple-Choice method, while more stable, shows a steady decline in accuracy, possibly due to limited adaptability in iterative refinement. These trends highlight trade-offs between exploration (Generation) and exploitation (Multiple-Choice) in model training dynamics. </details> (d) Qwen2.5-14B <details> <summary>x36.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Across Iterations for DeepSeek-R1-Distill-Llama-8B ### Overview The chart visualizes the proportion of "flips" (likely model output changes) across five iterations for two methods: "Generation" (blue line) and "Multiple-Choice" (orange line). The y-axis represents the proportion of flips (0.01–0.06), and the x-axis represents iterations (1–5). A legend in the top-right corner labels the lines but includes ambiguous terms ("Correct Flip" and "Incorrect Flip") that do not align with the line styles. ### Components/Axes - **Title**: "DeepSeek-R1-Distill-Llama-8B" (top-center). - **X-axis**: "Iterations" (1–5, labeled at integer ticks). - **Y-axis**: "Proportion of Flips" (0.01–0.06, increments of 0.01). - **Legend**: Top-right corner, with: - **Generation**: Solid blue line (no markers). - **Multiple-Choice**: Solid orange line (no markers). - **Correct Flip**: Solid black line (no corresponding line in the chart). - **Incorrect Flip**: Dashed black line (no corresponding line in the chart). ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.04 (solid blue line starts here). - **Iteration 2**: ~0.02 (dips sharply). - **Iteration 3**: ~0.04 (rises back to initial value). - **Iteration 4**: ~0.02 (dips again). - **Iteration 5**: ~0.03 (moderate increase). #### Multiple-Choice (Orange Line) - **Iteration 1**: ~0.04 (starts near Generation). - **Iteration 2**: ~0.06 (peaks sharply). - **Iteration 3**: ~0.01 (plummets to lowest value). - **Iteration 4**: ~0.03 (moderate recovery). - **Iteration 5**: ~0.05 (sharp rise to second-highest value). ### Key Observations 1. **Volatility**: The Multiple-Choice line exhibits extreme fluctuations (0.01–0.06), while Generation remains relatively stable (0.02–0.04). 2. **Crossing Points**: The lines intersect at Iteration 3 (~0.04 for both) and Iteration 4 (~0.02–0.03 overlap). 3. **Legend Mismatch**: The legend includes "Correct Flip" and "Incorrect Flip" labels, but no lines match these styles (solid/dashed black). This suggests a potential error in the chart's legend or data representation. ### Interpretation - The data suggests that the "Multiple-Choice" method experiences significantly more variability in flip proportions across iterations compared to "Generation." The sharp peaks and troughs in the orange line could indicate instability or sensitivity to iteration-specific factors. - The legend's inclusion of "Correct Flip" and "Incorrect Flip" is puzzling, as no lines correspond to these labels. This discrepancy may imply a mislabeling error or a conceptual mismatch between the data and the legend. - The Generation method's stability might imply robustness in model output consistency, whereas the Multiple-Choice method's volatility could reflect higher uncertainty or dynamic behavior in its outputs. ### Spatial Grounding - **Legend**: Top-right corner, aligned with the chart's upper boundary. - **Lines**: Solid colors (blue/orange) without markers, occupying the central vertical space of the chart. - **Axes**: Centered labels with gridlines for reference. ### Content Details - **Numerical Approximations**: - Generation: [0.04, 0.02, 0.04, 0.02, 0.03]. - Multiple-Choice: [0.04, 0.06, 0.01, 0.03, 0.05]. - **Trend Verification**: - Generation: Slightly oscillatory but bounded between 0.02–0.04. - Multiple-Choice: Highly erratic, with a peak-to-trough range of 0.05 (0.06–0.01). ### Final Notes The chart highlights divergent behaviors between the two methods, with Multiple-Choice showing extreme sensitivity to iteration changes. The legend's ambiguity underscores the need for clarification on the definitions of "Correct Flip" and "Incorrect Flip" in this context. </details> (e) DeepSeek-R1-Distill-Llama-8B <details> <summary>x37.png Details</summary>  ### Visual Description ## Line Chart: Gemini-2.0-Flash ### Overview The chart visualizes the proportion of "Flips" (likely model outputs) across five iterations for two methods: "Generation" (blue solid line) and "Multiple-Choice" (orange dashed line). The y-axis represents the proportion of flips (0.02–0.10), while the x-axis tracks iterations (1–5). A legend in the top-right corner labels "Correct Flip" (solid black) and "Incorrect Flip" (dashed black), though these labels do not align with the plotted lines. ### Components/Axes - **Title**: "Gemini-2.0-Flash" (top-center). - **X-axis**: "Iterations" (1–5, evenly spaced). - **Y-axis**: "Proportion of Flips" (0.02–0.10, increments of 0.02). - **Legend**: Top-right corner. Labels: - "Correct Flip": Solid black line (no corresponding line in the chart). - "Incorrect Flip": Dashed black line (no corresponding line in the chart). - **Data Lines**: - **Blue Solid Line**: Labeled "Generation" in the legend (mismatched with legend's "Correct Flip"). - **Orange Dashed Line**: Labeled "Multiple-Choice" in the legend (mismatched with legend's "Incorrect Flip"). ### Detailed Analysis 1. **Generation (Blue Solid Line)**: - **Trend**: Starts at ~0.08 (Iteration 1), dips to ~0.02 (Iteration 2), peaks at ~0.085 (Iteration 3), then declines to ~0.05 (Iteration 5). - **Values**: - Iteration 1: 0.08 ± 0.005 - Iteration 2: 0.02 ± 0.003 - Iteration 3: 0.085 ± 0.004 - Iteration 4: 0.06 ± 0.003 - Iteration 5: 0.05 ± 0.002 2. **Multiple-Choice (Orange Dashed Line)**: - **Trend**: Begins at ~0.06 (Iteration 1), drops to ~0.02 (Iteration 2), further declines to ~0.015 (Iteration 3), then rises slightly to ~0.025 (Iteration 5). - **Values**: - Iteration 1: 0.06 ± 0.003 - Iteration 2: 0.02 ± 0.002 - Iteration 3: 0.015 ± 0.001 - Iteration 4: 0.03 ± 0.002 - Iteration 5: 0.025 ± 0.001 ### Key Observations - **Legend Mismatch**: The legend incorrectly associates "Correct Flip" with a solid black line and "Incorrect Flip" with a dashed black line, but the chart uses blue (solid) for "Generation" and orange (dashed) for "Multiple-Choice." - **Generation Line Volatility**: Sharp fluctuations (e.g., drop from 0.08 to 0.02 between Iterations 1–2, then a peak at Iteration 3). - **Multiple-Choice Stability**: Gradual decline followed by a minor rebound, suggesting lower variability compared to "Generation." ### Interpretation The chart likely compares two model strategies ("Generation" vs. "Multiple-Choice") across iterations. The "Generation" method shows higher initial performance but erratic behavior, while "Multiple-Choice" stabilizes at lower proportions. The legend's labels ("Correct Flip"/"Incorrect Flip") may be misaligned with the data, suggesting a potential error in the chart's design or a misinterpretation of the axes. The "Generation" line's volatility could indicate instability in model outputs, whereas "Multiple-Choice" might represent a more consistent but less effective approach. Further validation of the legend labels is critical to avoid misinterpretation. </details> (f) Gemini-2.0-Flash Figure 7: Models Correct and Incorrect Flips on CoT on DisambiguationQA <details> <summary>x38.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in SmolLM2-1.7B Across Iterations ### Overview The chart visualizes the proportion of "flips" (changes in model predictions) for two methods—**Generation** and **Multiple-Choice**—across five iterations of the SmolLM2-1.7B model. It distinguishes between **Correct Flips** (solid circles) and **Incorrect Flips** (dashed squares) using color-coded lines and markers. ### Components/Axes - **X-axis**: Labeled "Iterations" with discrete values 1–5. - **Y-axis**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.04. - **Legend**: Located in the top-right corner, with: - **Blue line**: Represents **Generation** method. - **Orange line**: Represents **Multiple-Choice** method. - **Solid circles**: Denote **Correct Flips**. - **Dashed squares**: Denote **Incorrect Flips**. ### Detailed Analysis #### Generation Method (Blue Line) - **Iteration 1**: Proportion of flips ≈ 0.008 (Correct Flip: solid circle). - **Iteration 2**: Proportion ≈ 0.000 (no flips). - **Iterations 3–5**: Remains at 0.000 (no flips). - **Trend**: Sharp decline from iteration 1 to 2, then stable. #### Multiple-Choice Method (Orange Line) - **Iteration 1**: Proportion ≈ 0.035 (Correct Flip: solid circle). - **Iteration 2**: Proportion ≈ 0.015 (Correct Flip: solid circle). - **Iteration 3**: Proportion ≈ 0.000 (no flips). - **Iteration 4**: Proportion ≈ 0.008 (Correct Flip: solid circle). - **Iteration 5**: Proportion ≈ 0.008 (Correct Flip: solid circle). - **Trend**: Initial drop from 0.035 to 0.015, then stabilization with a minor uptick at iteration 4. #### Incorrect Flips (Dashed Squares) - **Generation**: No visible dashed squares (proportion ≈ 0.000 across all iterations). - **Multiple-Choice**: - **Iteration 1**: Proportion ≈ 0.027 (dashed square). - **Iteration 2**: Proportion ≈ 0.000 (no dashed square). - **Iterations 3–5**: Proportion ≈ 0.008 (dashed square). - **Trend**: Persistent incorrect flips in later iterations for Multiple-Choice. ### Key Observations 1. **Generation Method**: Rapid improvement in accuracy, with flips dropping to zero by iteration 2. 2. **Multiple-Choice Method**: Higher initial flips but inconsistent performance, with incorrect flips resurfacing in later iterations. 3. **Incorrect Flips**: Dominant in Multiple-Choice, particularly in iterations 1 and 4–5, suggesting potential errors in this method. ### Interpretation The data suggests that the **Generation** method achieves faster convergence and stability, while the **Multiple-Choice** method exhibits higher variability and persistent errors (incorrect flips). The sharp decline in flips for Generation indicates improved model confidence over iterations, whereas Multiple-Choice’s fluctuating performance may reflect challenges in handling ambiguous or complex inputs. The resurgence of incorrect flips in later iterations for Multiple-Choice raises questions about its reliability in dynamic scenarios. This aligns with the hypothesis that iterative refinement benefits simpler methods like Generation more effectively than heuristic approaches like Multiple-Choice. </details> (a) SmolLM2-1.7B <details> <summary>x39.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Across Iterations for Qwen2.5-3B ### Overview The chart visualizes the proportion of flips (correct and incorrect) for two strategies—**Generation** (blue line) and **Multiple-Choice** (orange line)—across five iterations. The y-axis represents the proportion of flips (0.00 to 0.14), while the x-axis denotes iterations (1 to 5). Two markers indicate "Correct Flip" (solid black) and "Incorrect Flip" (dashed black), placed on specific data points. --- ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations," with discrete values 1 to 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips," scaled from 0.00 to 0.14 in increments of 0.02. - **Legend**: Located in the top-right corner, with four entries: - **Generation**: Blue solid line. - **Multiple-Choice**: Orange dashed line. - **Correct Flip**: Solid black marker. - **Incorrect Flip**: Dashed black marker. --- ### Detailed Analysis #### Generation (Blue Line) - **Trend**: Starts at ~0.09 (iteration 1), drops sharply to ~0.04 (iteration 2), plummets to ~0.00 (iteration 3), rises slightly to ~0.03 (iteration 4), and ends at ~0.01 (iteration 5). - **Markers**: - **Correct Flip** (solid black): Placed at iteration 1 (~0.09). - No other markers observed. #### Multiple-Choice (Orange Line) - **Trend**: Begins at ~0.08 (iteration 1), peaks at ~0.14 (iteration 2), declines to ~0.06 (iteration 3), then ~0.04 (iteration 4), and ends at ~0.03 (iteration 5). - **Markers**: - **Incorrect Flip** (dashed black): Placed at iteration 2 (~0.14). --- ### Key Observations 1. **Generation Strategy**: - Shows a steep decline in flip proportion from iteration 1 to 3, suggesting reduced variability or improved stability. - A minor rebound in iterations 4–5, but remains near-zero. 2. **Multiple-Choice Strategy**: - Exhibits a sharp peak at iteration 2 (~0.14), followed by a consistent decline. - The **Incorrect Flip** marker at iteration 2 aligns with the peak, indicating a high proportion of incorrect flips at this point. 3. **Marker Placement**: - The **Correct Flip** (iteration 1, Generation) and **Incorrect Flip** (iteration 2, Multiple-Choice) are spatially distinct, highlighting divergent performance at early iterations. --- ### Interpretation - **Strategy Performance**: - The **Generation** strategy demonstrates a rapid reduction in flip proportion, potentially indicating improved accuracy or confidence over iterations. - The **Multiple-Choice** strategy starts with high flip rates but declines sharply, with the **Incorrect Flip** marker suggesting a critical error or outlier at iteration 2. - **Trend Implications**: - The divergence between the two strategies (Generation’s decline vs. Multiple-Choice’s peak) may reflect differing approaches to answer selection or error correction. - The near-zero flip proportion for Generation after iteration 3 could imply stabilization or convergence to a correct answer. - **Anomalies**: - The **Incorrect Flip** marker at iteration 2 for Multiple-Choice coincides with its peak, raising questions about whether this represents a systemic issue or a one-time error. --- ### Spatial Grounding - **Legend**: Top-right corner, clearly associating colors/markers with strategies and flip types. - **Markers**: - Solid black (Correct Flip) at iteration 1 (Generation line). - Dashed black (Incorrect Flip) at iteration 2 (Multiple-Choice line). - **Axes**: Y-axis on the left, X-axis at the bottom, with gridlines for reference. --- ### Content Details - **Numerical Approximations** (with uncertainty): - **Generation**: - Iteration 1: ~0.09 - Iteration 2: ~0.04 - Iteration 3: ~0.00 - Iteration 4: ~0.03 - Iteration 5: ~0.01 - **Multiple-Choice**: - Iteration 1: ~0.08 - Iteration 2: ~0.14 - Iteration 3: ~0.06 - Iteration 4: ~0.04 - Iteration 5: ~0.03 --- ### Final Notes The chart highlights contrasting trajectories for the two strategies, with the **Generation** approach showing a more stable decline and the **Multiple-Choice** strategy exhibiting volatility. The markers provide critical context for specific flip events, suggesting areas for further investigation into error patterns. </details> (b) Qwen2.5-3B <details> <summary>x40.png Details</summary>  ### Visual Description ## Line Graph: Llama-3.1-8B ### Overview The image is a line graph comparing two data series ("Generation" and "Multiple-Choice") across five iterations. The y-axis represents the "Proportion of Flips" (0.02–0.14), and the x-axis represents "Iterations" (1–5). A legend in the top-right corner distinguishes the two series, with additional labels for "Correct Flip" and "Incorrect Flip" (not plotted as data points). --- ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with markers at 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with increments of 0.02 (0.02, 0.04, ..., 0.14). - **Legend**: Located in the top-right corner. - Solid blue line with circles = "Generation" - Dashed orange line with squares = "Multiple-Choice" - Black dot = "Correct Flip" (not plotted) - Black square = "Incorrect Flip" (not plotted) --- ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.11 - **Iteration 2**: Peaks at ~0.14 - **Iteration 3**: Drops to ~0.08 - **Iteration 4**: Rises to ~0.10 - **Iteration 5**: Declines to ~0.06 #### Multiple-Choice (Orange Line) - **Iteration 1**: ~0.09 - **Iteration 2**: Sharp drop to ~0.02 - **Iteration 3**: Remains flat at ~0.02 - **Iteration 4**: Rises to ~0.04 - **Iteration 5**: Slight increase to ~0.04 --- ### Key Observations 1. **Generation** exhibits high volatility, with a peak at iteration 2 and a trough at iteration 3, followed by a partial recovery. 2. **Multiple-Choice** shows a dramatic drop at iteration 2, stabilizing until iteration 3, then a gradual rise. 3. The "Correct Flip" and "Incorrect Flip" labels in the legend are not visually represented in the graph, suggesting they may relate to unplotted data or annotations. --- ### Interpretation - The **Generation** series demonstrates instability in flip proportions, potentially indicating inconsistent performance or external variability across iterations. - The **Multiple-Choice** series suggests a correction or adjustment after iteration 2, with a steady improvement from iteration 3 onward. - The absence of plotted data for "Correct Flip" and "Incorrect Flip" implies these categories might be part of a separate analysis or require additional context (e.g., a secondary metric not visualized here). - The divergence between the two series highlights differing behaviors: Generation’s fluctuation vs. Multiple-Choice’s stabilization and recovery. </details> (c) Llama-3.1-8B <details> <summary>x41.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in Qwen2.5-14B Across Iterations ### Overview The chart illustrates the proportion of "flips" (likely model output changes) across two methods ("Generation" and "Multiple-Choice") over five iterations. A secondary legend indicates "Correct Flip" (solid black) and "Incorrect Flip" (dashed black), though these lines are not visibly plotted. Key trends include sharp declines in the "Generation" method and fluctuating behavior in the "Multiple-Choice" method. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete markers at 1, 2, 3, 4, and 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips," scaled from 0.00 to 0.05 in increments of 0.01. - **Legend**: Located in the top-right corner, with: - **Generation**: Solid blue line with square markers. - **Multiple-Choice**: Dashed orange line with diamond markers. - **Correct Flip**: Solid black line (no visible data). - **Incorrect Flip**: Dashed black line (no visible data). ### Detailed Analysis 1. **Generation (Blue Line)**: - **Iteration 1**: Starts at ~0.045 (highest value). - **Iteration 2**: Drops to ~0.025. - **Iterations 3–5**: Remains flat at ~0.025 until iteration 4, then plummets to 0.00. - **Trend**: Sharp initial decline, followed by stabilization and a final collapse. 2. **Multiple-Choice (Orange Line)**: - **Iteration 1**: Begins at ~0.008. - **Iteration 2**: Rises to ~0.018. - **Iteration 3**: Drops to 0.00. - **Iteration 4**: Remains at 0.00. - **Iteration 5**: Spikes to ~0.025. - **Trend**: Volatile, with a late-stage surge. 3. **Correct/Incorrect Flips (Black Lines)**: - Both lines are flat at 0.00 across all iterations, suggesting no recorded flips in these categories. ### Key Observations - The "Generation" method shows a dramatic reduction in flips after iteration 2, stabilizing until iteration 4 before collapsing entirely. - The "Multiple-Choice" method exhibits erratic behavior, with a notable late-stage increase at iteration 5. - "Correct Flip" and "Incorrect Flip" categories show no activity, raising questions about their relevance to the plotted data. ### Interpretation The data suggests that the "Generation" method becomes more stable (or less prone to flips) over time, though its final collapse at iteration 5 is puzzling. The "Multiple-Choice" method’s late-stage spike may indicate a specific trigger or anomaly in that iteration. The absence of "Correct/Incorrect Flip" data implies these categories might be excluded from the analysis or represent a separate metric. The stark contrast between the two methods highlights divergent performance characteristics, potentially reflecting differences in model architecture or training objectives. </details> (d) Qwen2.5-14B <details> <summary>x42.png Details</summary>  ### Visual Description ## Line Chart: DeepSeek-R1-Distill-Llama-8B ### Overview The chart compares the proportion of "Flips" (likely model output changes) across two methods ("Generation" and "Multiple-Choice") over 5 iterations. It includes annotations for "Correct Flip" and "Incorrect Flip" markers, though their exact placement is ambiguous. ### Components/Axes - **X-axis**: "Iterations" (1 to 5, discrete steps). - **Y-axis**: "Proportion of Flips" (0.00 to 0.12, linear scale). - **Legend**: - **Generation**: Blue dashed line. - **Multiple-Choice**: Orange solid line. - **Correct Flip**: Black dot (unclear placement). - **Incorrect Flip**: Black square (unclear placement). - **Title**: Positioned at the top-center. ### Detailed Analysis 1. **Generation (Blue Dashed Line)**: - Iteration 1: ~0.03. - Iteration 2: ~0.02. - Iteration 3: ~0.02. - Iteration 4: ~0.02. - Iteration 5: ~0.04. - **Trend**: Starts at 0.03, drops to 0.02 (Iterations 2–4), then rises to 0.04. 2. **Multiple-Choice (Orange Solid Line)**: - Iteration 1: ~0.06. - Iteration 2: ~0.08. - Iteration 3: ~0.10. - Iteration 4: ~0.05. - Iteration 5: ~0.07. - **Trend**: Peaks at 0.10 (Iteration 3), then fluctuates downward and upward. 3. **Correct Flip/Incorrect Flip**: - No clear data points visible on the chart. Likely annotations or legend entries without direct graphical representation. ### Key Observations - **Multiple-Choice** consistently shows higher flip proportions than **Generation**, except in Iteration 5 where they converge (~0.07 vs. ~0.04). - **Generation** exhibits stability until Iteration 5, where it sharply increases. - **Correct Flip/Incorrect Flip** markers are not visually represented on the chart, suggesting potential ambiguity in their role. ### Interpretation - The **Multiple-Choice** method demonstrates greater variability in flip proportions, peaking at Iteration 3, which may indicate higher sensitivity to input perturbations or model uncertainty during that phase. - The **Generation** method shows resilience until Iteration 5, where a sudden increase suggests potential instability or adaptation to later-stage data. - The absence of visible **Correct Flip/Incorrect Flip** markers on the chart raises questions about their implementation or relevance to the plotted data. This could imply: - They are theoretical annotations not tied to specific iterations. - They represent aggregated metrics outside the iteration framework. - A design oversight in the chart's visualization. The data suggests that **Multiple-Choice** may be more prone to output flips (potentially errors or corrections) compared to **Generation**, though the final iteration's convergence warrants further investigation into model behavior under stress or complex inputs. </details> (e) DeepSeek-R1-Distill-Llama-8B <details> <summary>x43.png Details</summary>  ### Visual Description ## Line Chart: Gemini-2.0-Flash ### Overview The chart visualizes the proportion of flips (correct and incorrect) across iterations for two methods: "Generation" (blue line) and "Multiple-Choice" (orange line). The y-axis represents the proportion of flips (0.00–0.07), and the x-axis represents iterations (1–5). The legend in the top-right corner distinguishes "Correct Flip" (solid black) and "Incorrect Flip" (dashed black), though these labels do not directly align with the line colors (blue/orange), suggesting potential ambiguity in the chart's design. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete values 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.07 in increments of 0.01. - **Legend**: Located in the top-right corner. Labels: - "Correct Flip" (solid black circle) - "Incorrect Flip" (dashed black square) - **Lines**: - **Blue Line (Generation)**: Solid squares as markers. - **Orange Line (Multiple-Choice)**: Solid squares as markers. ### Detailed Analysis 1. **Generation (Blue Line)**: - **Iteration 1**: ~0.07 (highest value). - **Iteration 2**: ~0.03 (sharp decline). - **Iteration 3**: ~0.04 (slight increase). - **Iteration 4**: ~0.04 (stable). - **Iteration 5**: ~0.03 (moderate decrease). - **Trend**: Initial sharp decline followed by stabilization with minor fluctuations. 2. **Multiple-Choice (Orange Line)**: - **Iteration 1**: ~0.01 (lowest value). - **Iteration 2**: ~0.00 (near-zero). - **Iteration 3**: ~0.00 (remains near-zero). - **Iteration 4**: ~0.00 (remains near-zero). - **Iteration 5**: ~0.00 (remains near-zero). - **Trend**: Immediate and sustained near-zero proportion after iteration 1. ### Key Observations - The "Generation" method starts with a high proportion of flips (~0.07) but decreases significantly by iteration 2, stabilizing around ~0.03–0.04 in later iterations. - The "Multiple-Choice" method begins with a low proportion (~0.01) and drops to near-zero by iteration 2, remaining stable thereafter. - The legend labels ("Correct Flip" and "Incorrect Flip") do not directly correspond to the line colors (blue/orange), creating potential confusion about what the lines represent. ### Interpretation - **Data Implications**: - The "Generation" method exhibits a higher initial proportion of flips that decreases over iterations, suggesting possible optimization or adaptation during the process. - The "Multiple-Choice" method maintains a consistently low proportion of flips, indicating stability or effectiveness in minimizing flips from the start. - **Ambiguity**: The mismatch between the legend labels ("Correct/Incorrect Flip") and the line labels ("Generation/Multiple-Choice") raises questions about whether the lines represent methods or flip types. If the lines represent methods, the legend might be mislabeled or incomplete. - **Trend Significance**: The sharp decline in "Generation" flips after iteration 1 could reflect algorithmic adjustments, while the near-zero values for "Multiple-Choice" suggest a robust or static approach that avoids flips entirely after the first iteration. </details> (f) Gemini-2.0-Flash Figure 8: Models Correct and Incorrect Flips on SC on DisambiguationQA <details> <summary>x44.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in SmolLM2-1.7B Across Iterations ### Overview The chart visualizes the proportion of "flips" (likely model behavior changes) across five iterations for two methods: "Generation" (blue line) and "Multiple-Choice" (orange line). A legend distinguishes "Correct Flip" (solid line) and "Incorrect Flip" (dashed line), though only solid lines are present in the chart. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete markers at 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Ranges from 0.00 to 0.07 in increments of 0.01. - **Legend**: Located in the top-right corner, with: - Solid line: "Correct Flip" - Dashed line: "Incorrect Flip" - **Lines**: - Blue solid line: "Generation" - Orange solid line: "Multiple-Choice" ### Detailed Analysis 1. **Generation (Blue Line)**: - Iteration 1: ~0.04 - Iteration 2: ~0.03 - Iteration 3: Peaks at ~0.07 - Iteration 4: Drops to ~0.02 - Iteration 5: Remains at ~0.02 - **Trend**: Initial decline, sharp peak at iteration 3, then stabilization at lower values. 2. **Multiple-Choice (Orange Line)**: - Iteration 1: ~0.03 - Iterations 2–5: Drops to 0.00 and remains flat. - **Trend**: Immediate collapse to zero after iteration 1. 3. **Legend Consistency**: - Both lines are solid, conflicting with the legend's "Incorrect Flip" (dashed) label. This suggests either a chart error or mislabeling. ### Key Observations - **Generation** exhibits volatility, with a significant spike at iteration 3 followed by a sharp decline. - **Multiple-Choice** shows no activity beyond iteration 1, dropping to zero and remaining flat. - The legend's "Incorrect Flip" label is not visually represented in the chart, creating ambiguity. ### Interpretation - The **Generation** method demonstrates dynamic behavior, with flips peaking at iteration 3 before stabilizing. This could indicate adaptive learning or model instability during training. - The **Multiple-Choice** method's abrupt drop to zero suggests it either fails to generate flips after the first iteration or is designed to suppress flips entirely. - The legend's mismatch (solid lines vs. "Incorrect Flip" dashed label) raises questions about data accuracy or chart design. If "Incorrect Flip" were represented, it might imply a subset of flips being misclassified, but this is speculative without visual evidence. - The stark contrast between the two methods highlights differing strategies: Generation allows for variable flips, while Multiple-Choice enforces strict suppression after iteration 1. </details> (a) SmolLM2-1.7B <details> <summary>x45.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Across Iterations for Qwen2.5-3B ### Overview The chart illustrates the proportion of "flips" (likely correct or incorrect responses) for two methods—**Generation** and **Multiple-Choice**—across five iterations. The y-axis represents the proportion of flips (0.00 to 0.10), and the x-axis represents iterations (1 to 5). Two data series are plotted: a blue line for **Generation** and an orange line for **Multiple-Choice**. A legend distinguishes **Correct Flip** (solid line) and **Incorrect Flip** (dashed line), though the chart only shows solid lines for both methods. --- ### Components/Axes - **Title**: "Qwen2.5-3B" - **Y-Axis**: "Proportion of Flips" (scale: 0.00 to 0.10, increments of 0.02) - **X-Axis**: "Iterations" (labeled 1 to 5) - **Legend**: - **Generation**: Blue solid line - **Multiple-Choice**: Orange solid line - **Correct Flip**: Solid line (blue) - **Incorrect Flip**: Dashed line (orange) --- ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.01 - **Iteration 2**: ~0.03 - **Iteration 3**: ~0.02 - **Iteration 4**: ~0.04 - **Iteration 5**: ~0.02 - **Trend**: Starts low, peaks at iteration 2, then fluctuates with a slight increase at iteration 4 before dropping. #### Multiple-Choice (Orange Line) - **Iteration 1**: ~0.08 - **Iteration 2**: ~0.06 - **Iteration 3**: ~0.04 - **Iteration 4**: ~0.05 - **Iteration 5**: ~0.02 - **Trend**: Starts high, decreases steadily, with a minor uptick at iteration 4 before a sharp drop. --- ### Key Observations 1. **Initial Disparity**: Multiple-Choice begins with a significantly higher proportion of flips (~0.08) compared to Generation (~0.01) at iteration 1. 2. **Divergent Trends**: - Generation shows volatility but stabilizes around 0.02–0.04 after iteration 2. - Multiple-Choice declines consistently, with a brief rise at iteration 4. 3. **Legend Ambiguity**: The legend labels "Correct Flip" (solid) and "Incorrect Flip" (dashed) conflict with the chart’s solid lines for both methods. This suggests a possible mislabeling or misinterpretation of the data. --- ### Interpretation - **Data Meaning**: The chart likely tracks the proportion of **correct flips** (e.g., model adjustments or corrections) for two methods over iterations. The **Generation** method shows a more variable but stabilizing trend, while **Multiple-Choice** declines sharply, suggesting it may be less effective or less adaptable over time. - **Legend Confusion**: The legend’s "Correct Flip" and "Incorrect Flip" labels do not align with the solid lines for both methods. This could indicate: - A mislabeling error in the legend. - The lines represent **total flips** (correct + incorrect), with the legend incorrectly categorizing them. - **Implications**: If the lines represent **correct flips**, the data suggests that **Generation** may improve over iterations, while **Multiple-Choice** deteriorates. If they represent **incorrect flips**, the opposite would be true. Further clarification of the legend is critical for accurate interpretation. --- **Note**: The chart lacks explicit data points for "Incorrect Flip," and the legend’s labels may not correspond to the plotted lines. This ambiguity limits definitive conclusions without additional context. </details> (b) Qwen2.5-3B <details> <summary>x46.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in Llama-3.1-8B Across Iterations ### Overview The chart illustrates the proportion of correct and incorrect flips for two prompting strategies ("Generation" and "Multiple-Choice") across five iterations. The y-axis represents the proportion of flips (0.02–0.14), and the x-axis represents iterations (1–5). Two lines are plotted: a blue line for "Generation" and an orange dashed line for "Multiple-Choice," each annotated with markers for correct (filled circles) and incorrect (open squares) flips. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations," with discrete values 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips," scaled from 0.02 to 0.14 in increments of 0.02. - **Legend**: Located in the top-right corner. - **Correct Flip**: Black filled circles. - **Incorrect Flip**: Black open squares. - **Lines**: - **Blue Solid Line**: Represents "Generation" strategy. - **Orange Dashed Line**: Represents "Multiple-Choice" strategy. ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: Correct flip = ~0.14 (circle), Incorrect flip = ~0.14 (square). - **Iteration 2**: Correct flip = ~0.08 (circle), Incorrect flip = ~0.12 (square). - **Iteration 3**: Correct flip = ~0.10 (circle), Incorrect flip = ~0.08 (square). - **Iteration 4**: Correct flip = ~0.06 (circle), Incorrect flip = ~0.08 (square). - **Iteration 5**: Correct flip = ~0.06 (circle), Incorrect flip = ~0.06 (square). #### Multiple-Choice (Orange Dashed Line) - **Iteration 1**: Correct flip = ~0.09 (circle), Incorrect flip = ~0.11 (square). - **Iteration 2**: Correct flip = ~0.04 (circle), Incorrect flip = ~0.08 (square). - **Iteration 3**: Correct flip = ~0.06 (circle), Incorrect flip = ~0.06 (square). - **Iteration 4**: Correct flip = ~0.02 (circle), Incorrect flip = ~0.04 (square). - **Iteration 5**: Correct flip = ~0.04 (circle), Incorrect flip = ~0.04 (square). ### Key Observations 1. **Trend for Generation**: - Correct flips start high (~0.14) in Iteration 1, drop to ~0.08 in Iteration 2, then stabilize around ~0.06–0.10 in later iterations. - Incorrect flips peak at ~0.12 in Iteration 2, then decline to ~0.06 by Iteration 5. 2. **Trend for Multiple-Choice**: - Correct flips start at ~0.09 in Iteration 1, drop to ~0.02 in Iteration 4, then rebound to ~0.04 in Iteration 5. - Incorrect flips decrease from ~0.11 in Iteration 1 to ~0.04 in Iteration 4, then stabilize at ~0.04 in Iteration 5. ### Interpretation - **Performance Degradation**: Both strategies show a general decline in correct flips over iterations, suggesting potential overfitting or adaptation to specific prompts. However, "Multiple-Choice" exhibits sharper declines, indicating less robustness compared to "Generation." - **Incorrect Flip Patterns**: The "Generation" strategy’s incorrect flips decrease steadily after Iteration 2, while "Multiple-Choice" shows a more erratic decline. This could imply that "Generation" better manages error reduction over time. - **Outliers**: The sharp drop in "Multiple-Choice" correct flips at Iteration 4 (~0.02) is notable, possibly reflecting a critical failure or misalignment in prompting strategy during that iteration. - **Implications**: The data highlights trade-offs between prompting methods. While "Generation" maintains more stable performance, "Multiple-Choice" may struggle with consistency, raising questions about its suitability for iterative refinement tasks. </details> (c) Llama-3.1-8B <details> <summary>x47.png Details</summary>  ### Visual Description ## Line Chart: Qwen2.5-14B Performance Analysis ### Overview The chart illustrates the proportion of flips (correct and incorrect) for two methods—Generation and Multiple-Choice—across five iterations. It also includes markers for correct and incorrect flips, though their relationship to the lines is ambiguous. The y-axis represents the proportion of flips (0.00 to 0.05), and the x-axis represents iterations (1 to 5). ### Components/Axes - **Title**: "Qwen2.5-14B" - **Y-Axis**: "Proportion of Flips" (scale: 0.00 to 0.05) - **X-Axis**: "Iterations" (1 to 5) - **Legend**: - **Generation**: Blue line - **Multiple-Choice**: Orange line - **Correct Flip**: Solid black marker - **Incorrect Flip**: Dashed black marker ### Detailed Analysis - **Generation (Blue Line)**: - Iteration 1: ~0.03 - Iteration 2: ~0.03 - Iteration 3: ~0.05 (peak) - Iteration 4: ~0.02 - Iteration 5: ~0.01 - **Multiple-Choice (Orange Line)**: - Iteration 1: ~0.03 - Iteration 2: ~0.01 - Iteration 3: ~0.01 - Iteration 4: ~0.01 - Iteration 5: ~0.00 - **Correct Flip (Solid Black Markers)**: - Iteration 1: ~0.03 - Iteration 2: ~0.01 - Iteration 3: ~0.00 - Iteration 4: ~0.00 - Iteration 5: ~0.00 - **Incorrect Flip (Dashed Black Markers)**: - Iteration 1: ~0.02 - Iteration 2: ~0.02 - Iteration 3: ~0.01 - Iteration 4: ~0.01 - Iteration 5: ~0.00 ### Key Observations 1. **Generation Method**: - Peaks at iteration 3 (0.05) before declining sharply. - Shows a U-shaped trend with a sharp drop after iteration 3. 2. **Multiple-Choice Method**: - Starts at 0.03 (iteration 1) and declines steadily to 0.00 by iteration 5. 3. **Correct/Incorrect Flips**: - Correct flips (solid black) decrease monotonically after iteration 1. - Incorrect flips (dashed black) also decline but remain higher than correct flips in early iterations. 4. **Discrepancies**: - The sum of correct and incorrect flips (e.g., 0.03 + 0.02 = 0.05 at iteration 1) exceeds the Generation line value (0.03), suggesting potential misalignment in data representation. ### Interpretation - The chart highlights the performance of two methods (Generation and Multiple-Choice) in terms of flip proportions. The Generation method exhibits a sharp peak at iteration 3, possibly indicating a temporary anomaly or optimization point. The Multiple-Choice method shows a consistent decline, suggesting diminishing returns over iterations. - The Correct and Incorrect Flip markers do not align with the lines, raising questions about their relationship. For example, the total flips (correct + incorrect) often exceed the line values, implying either overlapping data series or a misinterpretation of the legend. This could indicate a need for clarification in the data labeling or visualization design. - The decline in both correct and incorrect flips after iteration 3 suggests that the model's performance stabilizes or deteriorates over time, depending on the context of "flips" (e.g., model corrections or errors). </details> (d) Qwen2.5-14B <details> <summary>x48.png Details</summary>  ### Visual Description ## Line Chart: DeepSeek-R1-Distill-Llama-8B ### Overview The chart illustrates the proportion of flips (correct and incorrect) for two methods—**Generation** and **Multiple-Choice**—across five iterations. The y-axis represents the proportion of flips (0.00 to 0.08), while the x-axis denotes iterations (1 to 5). Two lines are plotted: a blue line for **Generation** and an orange line for **Multiple-Choice**, with data points marked as filled (correct flips) and open (incorrect flips). ### Components/Axes - **Y-axis**: "Proportion of Flips" (0.00 to 0.08, increments of 0.02). - **X-axis**: "Iterations" (1 to 5). - **Legend**: - **Blue line**: "Generation" (filled circles for correct flips, open circles for incorrect flips). - **Orange line**: "Multiple-Choice" (filled circles for correct flips, open circles for incorrect flips). ### Detailed Analysis - **Generation (Blue Line)**: - Iteration 1: ~0.06 (filled circle, correct flip). - Iteration 2: ~0.08 (filled circle, correct flip). - Iteration 3: ~0.02 (filled circle, correct flip). - Iteration 4: ~0.05 (filled circle, correct flip). - Iteration 5: ~0.06 (filled circle, correct flip). - **Multiple-Choice (Orange Line)**: - Iteration 1: ~0.04 (open circle, incorrect flip). - Iteration 2: ~0.06 (open circle, incorrect flip). - Iteration 3: ~0.00 (open circle, incorrect flip). - Iteration 4: ~0.03 (open circle, incorrect flip). - Iteration 5: ~0.05 (open circle, incorrect flip). ### Key Observations 1. **Generation Line**: - Peaks at iteration 2 (~0.08) and reaches a trough at iteration 3 (~0.02). - Shows a general upward trend after iteration 3, stabilizing at ~0.06 by iteration 5. 2. **Multiple-Choice Line**: - Drops sharply to 0.00 at iteration 3, then increases to ~0.05 by iteration 5. - Exhibits a V-shaped pattern with a minimum at iteration 3. ### Interpretation The data suggests that the **Generation** method experiences significant fluctuations in correct flips, with a notable dip at iteration 3. The **Multiple-Choice** method shows a dramatic reduction in incorrect flips at iteration 3, followed by a recovery. This could indicate that the model's performance for Multiple-Choice improved after iteration 3, while Generation's performance stabilized. The sharp drop in Multiple-Choice at iteration 3 might reflect a model adjustment or a change in data distribution. The trends highlight the dynamic nature of the model's behavior across iterations, with potential implications for optimization strategies. </details> (e) DeepSeek-R1-Distill-Llama-8B <details> <summary>x49.png Details</summary>  ### Visual Description ## Line Graph: Gemini-2.0-Flash ### Overview The image is a line graph titled "Gemini-2.0-Flash" that compares two data series: "Generation" (blue line) and "Multiple-Choice" (orange line). The x-axis represents "Iterations" (1–5), and the y-axis represents "Proportion of Flips" (0.00 to 0.04). A legend in the top-right corner distinguishes "Correct Flip" (solid black circle) and "Incorrect Flip" (dashed black square). ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete markers at 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.04 in increments of 0.01. - **Legend**: Located in the top-right corner. - "Correct Flip": Solid black circle. - "Incorrect Flip": Dashed black square. - **Data Series**: - "Generation": Blue solid line. - "Multiple-Choice": Orange dashed line. ### Detailed Analysis - **Generation (Blue Line)**: - Iteration 1: ~0.02. - Iteration 2: ~0.00. - Iteration 3: ~0.02. - Iteration 4: ~0.00. - Iteration 5: ~0.04. - **Trend**: The line starts at 0.02, drops to 0.00 at iteration 2, rises to 0.02 at iteration 3, drops to 0.00 at iteration 4, and spikes sharply to 0.04 at iteration 5. - **Multiple-Choice (Orange Line)**: - Iteration 1: ~0.00. - Iteration 2: ~0.03. - Iteration 3: ~0.00. - Iteration 4: ~0.01. - Iteration 5: ~0.00. - **Trend**: The line starts at 0.00, rises to 0.03 at iteration 2, drops to 0.00 at iteration 3, rises to 0.01 at iteration 4, and drops to 0.00 at iteration 5. ### Key Observations 1. **Generation Line**: Exhibits a volatile pattern with a significant spike at iteration 5, suggesting a sudden increase in flip proportion. 2. **Multiple-Choice Line**: Peaks at iteration 2 (0.03) and shows a sharp decline afterward, indicating early variability followed by stabilization. 3. **Legend Discrepancy**: The legend labels "Correct Flip" and "Incorrect Flip" do not directly correspond to the data series labels ("Generation" and "Multiple-Choice"). This may imply that the legend refers to markers (e.g., data points) rather than the lines themselves, but no markers are explicitly shown in the image. ### Interpretation The graph illustrates how the proportion of flips varies across iterations for two methods: "Generation" and "Multiple-Choice." The "Generation" line’s sharp increase at iteration 5 could indicate a critical change in behavior or an anomaly, while the "Multiple-Choice" line’s early peak suggests higher initial variability. The legend’s "Correct Flip" and "Incorrect Flip" labels remain ambiguous without explicit markers on the lines, but they may relate to the nature of the flips (e.g., correct vs. incorrect predictions) rather than the data series themselves. The data highlights the dynamic relationship between iteration count and flip proportion, with the "Generation" method showing more pronounced fluctuations over time. </details> (f) Gemini-2.0-Flash Figure 9: Models Correct and Incorrect Flips on Baseline on tinyTruthfulQA <details> <summary>x50.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in SmolLM2-1.7B Across Iterations ### Overview The chart visualizes the proportion of "flips" (correct and incorrect) for two model configurations ("Generation" and "Multiple-Choice") across five iterations. The y-axis represents the proportion of flips (0.00–0.12), and the x-axis represents iterations (1–5). Two data series are plotted: solid lines for correct flips and dashed lines for incorrect flips. ### Components/Axes - **Title**: "SmolLM2-1.7B" - **X-axis**: "Iterations" (labeled 1–5) - **Y-axis**: "Proportion of Flips" (0.00–0.12) - **Legend**: - Top-right corner - Labels: - "Generation" (blue solid line for correct flips, blue dashed line for incorrect flips) - "Multiple-Choice" (orange solid line for correct flips, orange dashed line for incorrect flips) ### Detailed Analysis #### Generation (Blue) - **Iteration 1**: - Correct Flip: ~0.05 - Incorrect Flip: ~0.01 - **Iteration 2**: - Correct Flip: ~0.04 - Incorrect Flip: ~0.00 - **Iteration 3**: - Correct Flip: ~0.05 - Incorrect Flip: ~0.01 - **Iteration 4**: - Correct Flip: ~0.02 - Incorrect Flip: ~0.00 - **Iteration 5**: - Correct Flip: ~0.03 - Incorrect Flip: ~0.00 #### Multiple-Choice (Orange) - **Iteration 1**: - Correct Flip: ~0.02 - Incorrect Flip: ~0.01 - **Iteration 2**: - Correct Flip: ~0.03 - Incorrect Flip: ~0.00 - **Iteration 3**: - Correct Flip: ~0.01 - Incorrect Flip: ~0.00 - **Iteration 4**: - Correct Flip: ~0.03 - Incorrect Flip: ~0.01 - **Iteration 5**: - Correct Flip: ~0.01 - Incorrect Flip: ~0.00 ### Key Observations 1. **Generation** shows a sharp decline in correct flips from Iteration 1 (0.05) to Iteration 2 (0.04), followed by stabilization. Incorrect flips remain negligible after Iteration 2. 2. **Multiple-Choice** exhibits higher variability, with peaks at Iterations 2 (0.03) and 4 (0.03) and a trough at Iteration 3 (0.01). Incorrect flips are consistently low. 3. Both configurations show minimal incorrect flips after Iteration 2, suggesting improved stability. ### Interpretation The data suggests that the "Generation" configuration initially experiences a significant drop in correct flips but stabilizes by Iteration 3, with negligible incorrect flips thereafter. The "Multiple-Choice" configuration demonstrates erratic performance, with correct flips fluctuating but incorrect flips remaining low. The sharp decline in Generation’s correct flips between Iterations 1–2 may indicate an initial adjustment phase, while the subsequent stability implies convergence toward optimal performance. The minimal incorrect flips across both configurations highlight robustness in the model’s decision-making process after early iterations. </details> (a) SmolLM2-1.7B <details> <summary>x51.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Across Iterations for Qwen2.5-3B ### Overview The chart illustrates the proportion of flips (correct and incorrect) for two methods—**Generation** and **Multiple-Choice**—across five iterations. The y-axis represents the proportion of flips (ranging from 0.02 to 0.14), while the x-axis denotes iterations (1 to 5). The legend, positioned in the top-right corner, distinguishes between **Correct Flip** (solid line) and **Incorrect Flip** (dashed line). The blue line (Generation) and orange line (Multiple-Choice) show distinct trends over iterations. ### Components/Axes - **Title**: "Qwen2.5-3B" (top-center). - **X-axis**: Labeled "Iterations" with discrete values 1, 2, 3, 4, 5. - **Y-axis**: Labeled "Proportion of Flips" with increments of 0.02 (0.02, 0.04, 0.06, 0.08, 0.10, 0.12, 0.14). - **Legend**: Located in the top-right corner, with: - **Correct Flip**: Solid line (blue). - **Incorrect Flip**: Dashed line (orange). ### Detailed Analysis - **Generation (Blue, Solid Line)**: - **Iteration 1**: ~0.10 (highest value). - **Iteration 2**: ~0.08. - **Iteration 3**: ~0.06. - **Iteration 4**: ~0.04. - **Iteration 5**: ~0.04. - **Trend**: Steady decline from 0.10 to 0.04 over iterations. - **Multiple-Choice (Orange, Dashed Line)**: - **Iteration 1**: ~0.04 (lowest value). - **Iteration 2**: ~0.06. - **Iteration 3**: ~0.06. - **Iteration 4**: ~0.06. - **Iteration 5**: ~0.06. - **Trend**: Slight increase from 0.04 to 0.06, stabilizing after iteration 2. ### Key Observations 1. **Generation** shows a consistent downward trend in the proportion of flips, suggesting a reduction in correct flips over iterations. 2. **Multiple-Choice** exhibits a minor upward trend, indicating an increase in incorrect flips, though it stabilizes after iteration 2. 3. The legend’s color coding (solid for correct, dashed for incorrect) aligns with the lines’ visual representation. ### Interpretation The data suggests that the **Generation** method’s performance in terms of correct flips deteriorates as iterations increase, while the **Multiple-Choice** method’s incorrect flips rise slightly but plateau. This could imply that the Generation method becomes less reliable over time, whereas the Multiple-Choice method’s errors stabilize. The legend’s placement and color coding ensure clarity, but the lack of explicit error bars or confidence intervals limits the ability to assess statistical significance. The trends may reflect underlying algorithmic behavior or data distribution shifts across iterations. </details> (b) Qwen2.5-3B <details> <summary>x52.png Details</summary>  ### Visual Description ## Line Graph: Proportion of Flips in Llama-3.1-8B Across Iterations ### Overview The graph compares the proportion of "flips" (changes in model responses) across five iterations for two methods: "Generation" (blue solid line) and "Multiple-Choice" (orange dashed line). It also includes two marker-based series: "Correct Flip" (black solid circles) and "Incorrect Flip" (black dashed squares). The y-axis represents the proportion of flips (0.025–0.200), while the x-axis shows iterations (1–5). ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete values 1–5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.025 to 0.200. - **Legend**: Located in the top-right corner, with four entries: - **Generation**: Blue solid line. - **Multiple-Choice**: Orange dashed line. - **Correct Flip**: Black solid circles. - **Incorrect Flip**: Black dashed squares. ### Detailed Analysis 1. **Generation (Blue Solid Line)**: - Iteration 1: ~0.18 - Iteration 2: ~0.12 - Iteration 3: ~0.16 - Iteration 4: ~0.11 - Iteration 5: ~0.10 - **Trend**: Peaks at iteration 2, then declines with minor fluctuations. 2. **Multiple-Choice (Orange Dashed Line)**: - Iteration 1: ~0.17 - Iteration 2: ~0.04 - Iteration 3: ~0.05 - Iteration 4: ~0.06 - Iteration 5: ~0.04 - **Trend**: Sharp drop at iteration 2, followed by gradual increases and stabilization. 3. **Correct Flip (Black Solid Circles)**: - Iteration 1: ~0.19 - Iteration 2: ~0.13 - Iteration 3: ~0.15 - Iteration 4: ~0.12 - Iteration 5: ~0.10 - **Trend**: Steady decline across iterations. 4. **Incorrect Flip (Black Dashed Squares)**: - Iteration 1: ~0.01 - Iteration 2: ~0.07 - Iteration 3: ~0.05 - Iteration 4: ~0.08 - Iteration 5: ~0.06 - **Trend**: Initial increase at iteration 2, followed by fluctuations. ### Key Observations - The **Generation** method shows the highest initial flip proportion (~0.18) but decreases over time. - **Multiple-Choice** starts high (~0.17) but plummets at iteration 2, suggesting a significant shift in behavior. - **Correct Flip** consistently declines, indicating fewer accurate response changes over iterations. - **Incorrect Flip** rises sharply at iteration 2, then stabilizes, implying increased erroneous flips after early iterations. ### Interpretation The data suggests that the **Generation** method exhibits volatile flip behavior, with a notable drop after iteration 2. The **Multiple-Choice** method’s drastic decline at iteration 2 may indicate a shift in response strategy or model adaptation. The **Correct Flip** trend implies reduced accuracy in response changes over time, while the **Incorrect Flip** increase hints at growing errors in later iterations. This could reflect model instability or learning dynamics during iterative processes. The divergence between methods highlights trade-offs between response consistency and adaptability. </details> (c) Llama-3.1-8B <details> <summary>x53.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Across Iterations for Qwen2.5-14B ### Overview The chart illustrates the proportion of "flips" (likely model output changes) across 5 iterations for two categories: "Generation" (blue line) and "Multiple-Choice" (orange line). The y-axis represents the proportion of flips (0.00 to 0.06), and the x-axis represents iterations (1 to 5). A legend distinguishes "Correct Flip" (solid black) and "Incorrect Flip" (dashed black), though these are not visually represented in the chart. --- ### Components/Axes - **Title**: "Qwen2.5-14B" (top center). - **Y-Axis**: "Proportion of Flips" (0.00 to 0.06, linear scale). - **X-Axis**: "Iterations" (1 to 5, discrete steps). - **Legend**: - **Generation**: Blue solid line. - **Multiple-Choice**: Orange solid line. - **Correct Flip**: Solid black (not visible in the chart). - **Incorrect Flip**: Dashed black (not visible in the chart). --- ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: ~0.03 (blue dot). - **Iteration 2**: ~0.04 (blue dot). - **Iteration 3**: ~0.02 (blue dot). - **Iteration 4**: ~0.01 (blue dot). - **Iteration 5**: ~0.00 (blue dot). - **Trend**: Steady decline from 0.03 to 0.00 over 5 iterations. #### Multiple-Choice (Orange Line) - **Iteration 1**: ~0.02 (orange dot). - **Iteration 2**: ~0.01 (orange dot). - **Iteration 3**: ~0.00 (orange dot). - **Iteration 4**: ~0.01 (orange dot). - **Iteration 5**: ~0.01 (orange dot). - **Trend**: Initial drop to 0.00 at iteration 3, followed by a slight increase to 0.01 at iterations 4 and 5. --- ### Key Observations 1. **Generation Line**: Shows a consistent downward trend, suggesting a reduction in flip proportion as iterations increase. 2. **Multiple-Choice Line**: Exhibits a non-linear pattern, with a sharp drop at iteration 3 and a rebound in later iterations. 3. **Legend Discrepancy**: The "Correct Flip" and "Incorrect Flip" labels in the legend do not correspond to any visible data series in the chart. This may indicate a mislabeling or omission in the image. --- ### Interpretation The chart suggests that the "Generation" model experiences a significant reduction in flip proportion over iterations, potentially indicating improved stability or accuracy. The "Multiple-Choice" model shows more variability, with a temporary dip and subsequent recovery. The absence of "Correct Flip" and "Incorrect Flip" data in the chart raises questions about the completeness of the visualization or potential errors in the legend. The trends imply that iterative refinement may enhance model performance, but further clarification on the "Correct/Incorrect Flip" categories is needed for a comprehensive analysis. </details> (d) Qwen2.5-14B <details> <summary>x54.png Details</summary>  ### Visual Description ## Line Chart: DeepSeek-R1-Distill-Llama-8B ### Overview The chart visualizes the proportion of "flips" (changes in model outputs) across iterations for two methods: "Generation" and "Multiple-Choice". It compares correct and incorrect flips using distinct markers. The y-axis represents the proportion of flips (0.00–0.06), and the x-axis shows iterations (1–5). ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with markers at positions 1–5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.06 in increments of 0.01. - **Legend**: Located in the top-right corner, with: - **Solid black circles**: Correct Flips - **Dashed black squares**: Incorrect Flips - **Data Series**: - **Blue line**: "Generation" method - **Orange line**: "Multiple-Choice" method ### Detailed Analysis 1. **Generation (Blue Line)**: - **Iteration 1**: Correct Flips ≈ 0.055, Incorrect Flips ≈ 0.002. - **Iteration 2**: Correct Flips peak at ≈ 0.06, Incorrect Flips drop to ≈ 0.0005. - **Iteration 3**: Correct Flips ≈ 0.042, Incorrect Flips ≈ 0.001. - **Iteration 4**: Correct Flips ≈ 0.041, Incorrect Flips ≈ 0.0015. - **Iteration 5**: Correct Flips ≈ 0.032, Incorrect Flips ≈ 0.002. 2. **Multiple-Choice (Orange Line)**: - **Iteration 1**: Correct Flips ≈ 0.02, Incorrect Flips ≈ 0.0005. - **Iteration 2**: Correct Flips ≈ 0.055, Incorrect Flips ≈ 0.0005. - **Iteration 3**: Correct Flips ≈ 0.01, Incorrect Flips ≈ 0.001. - **Iteration 4**: Correct Flips ≈ 0.01, Incorrect Flips ≈ 0.001. - **Iteration 5**: Correct Flips ≈ 0.02, Incorrect Flips ≈ 0.0015. ### Key Observations - **Peaks and Troughs**: Both methods show volatility, with sharp fluctuations at iteration 2 (e.g., Generation's correct flips spike to 0.06, while Multiple-Choice drops to 0.01 at iteration 3). - **Anomalies**: The orange line (Multiple-Choice) exhibits a pronounced dip at iteration 3, suggesting a potential outlier or methodological shift. - **Trend Divergence**: Generation consistently shows higher correct flips than Multiple-Choice, except at iteration 2 where they briefly align. ### Interpretation The data suggests that the "Generation" method generally produces more correct flips than "Multiple-Choice," though both exhibit instability. The sharp drop in Multiple-Choice at iteration 3 may indicate a failure mode or external factor affecting performance. The correlation between correct and incorrect flips (e.g., high correct flips often coincide with low incorrect flips) implies a trade-off between accuracy and consistency. Further investigation is needed to address the anomaly at iteration 3 for Multiple-Choice. </details> (e) DeepSeek-R1-Distill-Llama-8B <details> <summary>x55.png Details</summary>  ### Visual Description ## Line Chart: Gemini-2.0-Flash Performance Across Iterations ### Overview The chart visualizes the proportion of "Correct Flips" and "Incorrect Flips" for two methods ("Generation" and "Multiple-Choice") across five iterations. The y-axis represents the proportion of flips (0.00–0.08), while the x-axis tracks iterations (1–5). Two data series are plotted: a solid blue line for "Generation" and a dashed orange line for "Multiple-Choice." The legend in the top-right corner distinguishes "Correct Flip" (filled markers) and "Incorrect Flip" (hollow markers). ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete values 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.08 in increments of 0.02. - **Legend**: Located in the top-right corner. - "Correct Flip": Solid black circles (filled). - "Incorrect Flip": Hollow black squares (unfilled). - **Data Series**: - **Generation**: Solid blue line with filled markers for correct flips and hollow markers for incorrect flips. - **Multiple-Choice**: Dashed orange line with filled markers for correct flips and hollow markers for incorrect flips. ### Detailed Analysis #### Generation (Blue Line) - **Iteration 1**: Proportion of correct flips ≈ 0.075; incorrect flips ≈ 0.005. - **Iteration 2**: Proportion of correct flips ≈ 0.03; incorrect flips ≈ 0.04. - **Iteration 3**: Proportion of correct flips ≈ 0.06; incorrect flips ≈ 0.02. - **Iteration 4**: Proportion of correct flips ≈ 0.04; incorrect flips ≈ 0.03. - **Iteration 5**: Proportion of correct flips ≈ 0.02; incorrect flips ≈ 0.05. #### Multiple-Choice (Orange Line) - **Iteration 1**: Proportion of correct flips ≈ 0.06; incorrect flips ≈ 0.03. - **Iteration 2**: Proportion of correct flips ≈ 0.02; incorrect flips ≈ 0.04. - **Iteration 3**: Proportion of correct flips ≈ 0.01; incorrect flips ≈ 0.02. - **Iteration 4**: Proportion of correct flips ≈ 0.01; incorrect flips ≈ 0.03. - **Iteration 5**: Proportion of correct flips ≈ 0.005; incorrect flips ≈ 0.02. ### Key Observations 1. **Generation Method**: - Shows significant volatility, with a peak in correct flips at iteration 3 (~0.06) and a trough at iteration 5 (~0.02). - Incorrect flips increase sharply after iteration 3, reaching ~0.05 by iteration 5. 2. **Multiple-Choice Method**: - Demonstrates a steady decline in correct flips from ~0.06 (iteration 1) to ~0.005 (iteration 5). - Incorrect flips remain relatively stable (~0.02–0.04) but show a slight upward trend after iteration 3. 3. **Cross-Series Comparison**: - At iteration 3, "Generation" outperforms "Multiple-Choice" in correct flips (0.06 vs. 0.01). - By iteration 5, "Generation" has a higher proportion of incorrect flips (~0.05) compared to "Multiple-Choice" (~0.02). ### Interpretation The data suggests that the "Generation" method initially performs better in terms of correct flips but becomes increasingly unstable over iterations, leading to higher incorrect flip rates. In contrast, the "Multiple-Choice" method shows consistent but declining performance, with fewer incorrect flips overall. The peak in correct flips for "Generation" at iteration 3 may indicate an optimal point before degradation, while the steady decline in "Multiple-Choice" could reflect inherent limitations in its approach. The divergence in trends highlights trade-offs between adaptability (Generation) and consistency (Multiple-Choice) in this context. </details> (f) Gemini-2.0-Flash Figure 10: Models Correct and Incorrect Flips on CoT on tinyTruthfulQA <details> <summary>x56.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips Over Iterations for SmolLM2-1.7B ### Overview The chart illustrates the proportion of flips (likely correct or incorrect) for two methods—**Generation** and **Multiple-Choice**—across 5 iterations. The y-axis represents the proportion of flips (0.00 to 0.04), and the x-axis represents iterations (1 to 5). Two lines are plotted: a solid blue line for **Generation** and a dashed orange line for **Multiple-Choice**. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with values 1 to 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with values from 0.00 to 0.04. - **Legend**: - **Generation**: Solid blue line. - **Multiple-Choice**: Dashed orange line. - **Correct Flip**: Solid (matches Generation line). - **Incorrect Flip**: Dashed (matches Multiple-Choice line). ### Detailed Analysis - **Generation (Solid Blue Line)**: - Iteration 1: ~0.04 - Iteration 2: ~0.01 - Iteration 3: ~0.005 - Iteration 4: ~0.002 - Iteration 5: ~0.001 - **Trend**: Steeply decreasing from 0.04 to 0.001, with a sharp drop between iterations 1 and 2. - **Multiple-Choice (Dashed Orange Line)**: - Iteration 1: ~0.03 - Iteration 2: ~0.02 - Iteration 3: ~0.01 - Iteration 4: ~0.005 - Iteration 5: ~0.005 - **Trend**: Gradual decrease from 0.03 to 0.005, with a plateau at iteration 5. ### Key Observations 1. **Generation** shows a more pronounced decline in the proportion of flips compared to **Multiple-Choice**. 2. Both methods exhibit a downward trend, suggesting a reduction in flip frequency over iterations. 3. **Multiple-Choice** stabilizes at iteration 5, while **Generation** continues to decrease. ### Interpretation The data suggests that both **Generation** and **Multiple-Choice** methods experience a reduction in flip proportions as iterations increase. The steeper decline in **Generation** may indicate faster convergence or stabilization, while the plateau in **Multiple-Choice** could imply a different optimization dynamic. The legend’s association of **Correct Flip** with the solid line (Generation) and **Incorrect Flip** with the dashed line (Multiple-Choice) implies that the lines represent the proportion of correct flips for each method. This trend might reflect improvements in model performance or error correction over iterations. However, without explicit labels for "Correct" vs. "Incorrect" in the data, this interpretation relies on the legend’s visual cues. </details> (a) SmolLM2-1.7B <details> <summary>x57.png Details</summary>  ### Visual Description ## Line Graph: Proportion of Flips in Qwen2.5-3B Model Across Iterations ### Overview The image is a line graph titled "Qwen2.5-3B" that visualizes the proportion of "flips" (likely model output changes) across five iterations. Two primary data series are plotted: "Generation" (blue solid line with circles) and "Multiple-Choice" (orange dashed line with squares). A legend distinguishes "Correct Flip" (solid black circles) and "Incorrect Flip" (dashed black squares), though these markers are not explicitly plotted on the graph. The y-axis represents the proportion of flips (0.00–0.10), and the x-axis represents iterations (1–5). --- ### Components/Axes - **X-Axis (Iterations)**: Labeled "Iterations" with discrete markers at 1, 2, 3, 4, and 5. - **Y-Axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.10 in increments of 0.02. - **Legend**: Located in the top-right corner, with four entries: - **Generation**: Solid blue line with filled circles (●). - **Multiple-Choice**: Dashed orange line with hollow squares (□). - **Correct Flip**: Solid black circles (●). - **Incorrect Flip**: Dashed black squares (□). - **Title**: "Qwen2.5-3B" is centered at the top of the graph. --- ### Detailed Analysis #### Generation (Blue Solid Line) - **Trend**: Starts at ~0.085 (iteration 1), drops sharply to ~0.035 (iteration 2), rises to ~0.06 (iteration 3), then fluctuates between ~0.04 and ~0.05 (iterations 4–5). - **Data Points**: - Iteration 1: ~0.085 - Iteration 2: ~0.035 - Iteration 3: ~0.06 - Iteration 4: ~0.04 - Iteration 5: ~0.04 #### Multiple-Choice (Orange Dashed Line) - **Trend**: Begins at ~0.10 (iteration 1), plunges to ~0.02 (iteration 2), dips further to ~0.01 (iteration 3), rises to ~0.04 (iteration 4), and drops to ~0.02 (iteration 5). - **Data Points**: - Iteration 1: ~0.10 - Iteration 2: ~0.02 - Iteration 3: ~0.01 - Iteration 4: ~0.04 - Iteration 5: ~0.02 #### Correct Flip (Solid Black Circles) and Incorrect Flip (Dashed Black Squares) - **Observation**: These markers are not explicitly plotted on the graph. Their presence in the legend suggests they may represent subcategories of the "Generation" and "Multiple-Choice" lines (e.g., correct/incorrect flips within each line). However, no distinct data points for these markers are visible. --- ### Key Observations 1. **Generation Line**: Shows a general downward trend after iteration 1, stabilizing around 0.04 by iteration 5. 2. **Multiple-Choice Line**: Exhibits high volatility, with a sharp decline from iteration 1 to 3, followed by erratic fluctuations. 3. **Legend Ambiguity**: The "Correct Flip" and "Incorrect Flip" markers in the legend do not correspond to visible data points on the graph, suggesting potential mislabeling or incomplete data representation. 4. **Y-Axis Scale**: The proportion of flips is relatively low (<0.10), indicating minimal output instability in later iterations. --- ### Interpretation - **Model Behavior**: The "Generation" line’s decline suggests improved stability in the model’s output over iterations, while the "Multiple-Choice" line’s volatility implies persistent uncertainty in decision-making tasks. - **Legend Discrepancy**: The absence of plotted "Correct Flip" and "Incorrect Flip" data points raises questions about their relevance. They may represent theoretical categories not visualized here or errors in the graph’s design. - **Practical Implications**: The graph highlights the importance of iteration in refining model outputs, with "Generation" tasks showing more predictable behavior than "Multiple-Choice" tasks. --- ### Spatial Grounding - **Legend Position**: Top-right corner, aligned with the graph’s upper boundary. - **Line Placement**: "Generation" (blue) is consistently above "Multiple-Choice" (orange) in the early iterations but intersects at iteration 3. - **Axis Labels**: Centered below (x-axis) and rotated vertically on the left (y-axis). --- ### Content Details - **Numerical Approximations**: All values are estimated based on grid alignment (e.g., ~0.085 for iteration 1 of the Generation line). - **Uncertainty**: Due to the lack of gridlines between major ticks, values are approximate (e.g., "~0.04" could range between 0.035–0.045). --- ### Final Notes The graph provides insights into model performance trends but lacks clarity on the role of "Correct Flip" and "Incorrect Flip" markers. Further data or annotations would be needed to resolve this ambiguity. </details> (b) Qwen2.5-3B <details> <summary>x58.png Details</summary>  ### Visual Description ## Line Graph: Llama-3.1-8B Performance Over Iterations ### Overview The graph illustrates the proportion of "Flips" (changes in model outputs) across five iterations for four distinct model behaviors: Generation, Multiple-Choice, Correct Flip, and Incorrect Flip. Data is visualized using four distinct lines with markers. ### Components/Axes - **X-axis (Iterations)**: Discrete values 1–5, labeled "Iterations". - **Y-axis (Proportion of Flips)**: Continuous scale from 0.000 to 0.175, labeled "Proportion of Flips". - **Legend**: Positioned at the top-right corner, with four entries: - **Generation**: Solid blue line with square markers. - **Multiple-Choice**: Dashed orange line with square markers. - **Correct Flip**: Solid black line with circle markers. - **Incorrect Flip**: Dashed black line with circle markers. ### Detailed Analysis 1. **Generation (Blue Solid Line)**: - Iteration 1: ~0.105 - Iteration 2: Peaks at ~0.175 - Iteration 3: Drops to ~0.075 - Iteration 4: Rises to ~0.15 - Iteration 5: Declines to ~0.055 - *Trend*: Volatile, with a peak in iteration 2 and a general decline afterward. 2. **Multiple-Choice (Orange Dashed Line)**: - Iteration 1: ~0.06 - Iteration 2: ~0.04 - Iteration 3: ~0.025 - Iteration 4: ~0.03 - Iteration 5: ~0.02 - *Trend*: Steady decline with minor fluctuations. 3. **Correct Flip (Black Solid Line)**: - Iteration 1: ~0.025 - Iteration 2: ~0.01 - Iteration 3: ~0.005 - Iteration 4: ~0.02 - Iteration 5: ~0.02 - *Trend*: Minimal values, slight recovery in later iterations. 4. **Incorrect Flip (Black Dashed Line)**: - Iteration 1: ~0.15 - Iteration 2: Peaks at ~0.175 - Iteration 3: Drops to ~0.125 - Iteration 4: Rises to ~0.15 - Iteration 5: Declines to ~0.05 - *Trend*: Inverse relationship with Generation; peaks in iterations 2 and 4. ### Key Observations - **Inverse Correlation**: Generation and Incorrect Flip trends are nearly opposite (e.g., Generation peaks at iteration 2, while Incorrect Flip peaks there too, but declines as Generation recovers in iteration 4). - **Stability**: Multiple-Choice flips remain consistently low (<0.06), suggesting minimal variability in this behavior. - **Outliers**: Correct Flip values are orders of magnitude lower than other categories, indicating rare or negligible occurrences. ### Interpretation The data suggests that the Llama-3.1-8B model exhibits significant variability in "Generation" and "Incorrect Flip" behaviors across iterations, with a notable inverse relationship between these two metrics. The stability of Multiple-Choice flips implies robustness in this specific task. The persistently low Correct Flip values may indicate limitations in the model's ability to consistently align with expected outputs, warranting further investigation into training data or architectural adjustments. The volatility in Generation flips could reflect dynamic adaptation to input variations, while the cyclical pattern in Incorrect Flips might highlight recurring error modes. </details> (c) Llama-3.1-8B <details> <summary>x59.png Details</summary>  ### Visual Description ## Line Chart: Qwen2.5-14B ### Overview The chart illustrates the proportion of flips (correct and incorrect) for two methods—**Generation** and **Multiple-Choice**—across five iterations. The y-axis represents the proportion of flips (0.00 to 0.08), while the x-axis denotes iterations (1 to 5). Two lines are plotted: a blue line for **Generation** and an orange line for **Multiple-Choice**, with markers indicating correct (solid circles) and incorrect (squares) flips. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete values 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.08. - **Legend**: Located in the top-right corner, with: - **Blue line**: "Generation" (solid circles for correct flips, squares for incorrect flips). - **Orange line**: "Multiple-Choice" (solid circles for correct flips, squares for incorrect flips). ### Detailed Analysis - **Generation (Blue Line)**: - **Iteration 1**: Proportion of flips = ~0.08 (highest value). - **Iteration 2**: Proportion of flips = ~0.03 (sharp decline). - **Iterations 3–5**: Proportion of flips stabilizes at ~0.01. - **Trend**: Steep decline from iteration 1 to 2, followed by stabilization. - **Multiple-Choice (Orange Line)**: - **Iteration 1**: Proportion of flips = ~0.01. - **Iteration 2**: Proportion of flips = ~0.02 (peak). - **Iterations 3–5**: Proportion of flips drops to ~0.00. - **Trend**: Initial increase to iteration 2, then sharp decline. ### Key Observations 1. **Generation** shows a significant drop in flips after iteration 2, suggesting improved stability or reduced errors. 2. **Multiple-Choice** exhibits a brief spike in flips at iteration 2, followed by near-zero values, indicating potential overcorrection or method-specific instability. 3. **Correct vs. Incorrect Flips**: The legend distinguishes markers, but the chart does not explicitly separate correct/incorrect proportions. The lines likely represent total flips, with markers indicating the type (e.g., solid circles for correct, squares for incorrect). However, the data points do not provide explicit splits between correct and incorrect flips. ### Interpretation - The **Generation** method demonstrates a clear trend of reducing flips over iterations, which may reflect improved performance or error correction. The initial high proportion of flips (0.08) suggests significant variability or errors in early iterations. - The **Multiple-Choice** method shows a transient increase in flips at iteration 2, which could indicate a temporary issue or a methodological shift. Its subsequent drop to near-zero values implies stabilization or a lack of further flips. - The **correct/incorrect flip markers** are visually distinct but lack explicit numerical data, making it challenging to quantify the exact proportion of correct vs. incorrect flips. This ambiguity limits deeper analysis of error types. - The chart highlights the importance of iteration in refining model behavior, with **Generation** showing more consistent improvement compared to **Multiple-Choice**. </details> (d) Qwen2.5-14B <details> <summary>x60.png Details</summary>  ### Visual Description ## Line Chart: Proportion of Flips in DeepSeek-R1-Distill-Llama-8B Across Iterations ### Overview The chart visualizes the proportion of "flips" (likely model output changes) for two methods—**Generation** and **Multiple-Choice**—across five iterations of a model (DeepSeek-R1-Distill-Llama-8B). The y-axis represents the proportion of flips (0.02–0.12), and the x-axis represents iterations (1–5). Two lines are plotted: a blue line for **Generation** and an orange dashed line for **Multiple-Choice**. A legend on the right distinguishes **Correct Flip** (solid black) and **Incorrect Flip** (dashed black), though these are not directly mapped to the lines in the chart. --- ### Components/Axes - **Title**: "DeepSeek-R1-Distill-Llama-8B" (top center). - **Y-Axis**: "Proportion of Flips" (0.02–0.12, linear scale). - **X-Axis**: "Iterations" (1–5, discrete steps). - **Legend**: - **Correct Flip**: Solid black (not directly mapped to lines). - **Incorrect Flip**: Dashed black (not directly mapped to lines). - **Lines**: - **Generation**: Solid blue (left y-axis). - **Multiple-Choice**: Dashed orange (right y-axis). --- ### Detailed Analysis 1. **Generation (Blue Line)**: - **Iteration 1**: ~0.03. - **Iteration 2**: ~0.03 (stable). - **Iteration 3**: Drops to ~0.01 (lowest point). - **Iteration 4**: Rises to ~0.05. - **Iteration 5**: Slightly decreases to ~0.04. - **Trend**: U-shaped curve with a sharp dip at iteration 3. 2. **Multiple-Choice (Orange Dashed Line)**: - **Iteration 1**: ~0.08. - **Iteration 2**: ~0.08 (stable). - **Iteration 3**: Peaks at ~0.11 (highest point). - **Iteration 4**: Drops to ~0.07. - **Iteration 5**: Remains at ~0.07. - **Trend**: Initial stability, sharp peak at iteration 3, then gradual decline. 3. **Legend Elements**: - **Correct Flip** and **Incorrect Flip** are defined but not visually represented in the chart. This may indicate a separate metric or a misalignment in the visualization. --- ### Key Observations - **Generation** shows a significant drop in flips at iteration 3, followed by a recovery. This could suggest model stabilization or a shift in output behavior. - **Multiple-Choice** exhibits a peak at iteration 3, followed by a decline, indicating potential overfitting or increased variability in early iterations. - The **legend** labels (**Correct Flip**, **Incorrect Flip**) do not correspond to the plotted lines, suggesting either a missing data series or a labeling error. --- ### Interpretation - The **Generation** method’s U-shaped trend implies that flips initially decrease (possibly due to model refinement) but increase again later, which might reflect instability or adaptation to new data. - The **Multiple-Choice** method’s peak at iteration 3 suggests higher variability or uncertainty during that phase, followed by stabilization. This could indicate a trade-off between accuracy and consistency. - The absence of direct mapping between the legend labels and the lines raises questions about the chart’s completeness. If **Correct Flip** and **Incorrect Flip** are meant to represent subsets of the lines, additional data or annotations are required for clarity. - The divergence between the two methods highlights differences in how flips are distributed across iterations, potentially reflecting distinct algorithmic approaches (e.g., generative vs. constrained output generation). --- ### Spatial Grounding - **Legend**: Top-right corner (aligned with the chart’s upper boundary). - **Lines**: Generation (blue) on the left y-axis, Multiple-Choice (orange) on the right y-axis. - **Axes**: X-axis (bottom), Y-axis (left and right for dual-scale representation). --- ### Content Details - **Numerical Approximations** (with uncertainty): - **Generation**: 0.03 (±0.01), 0.03 (±0.01), 0.01 (±0.01), 0.05 (±0.01), 0.04 (±0.01). - **Multiple-Choice**: 0.08 (±0.01), 0.08 (±0.01), 0.11 (±0.01), 0.07 (±0.01), 0.07 (±0.01). --- ### Final Notes The chart provides insights into model behavior across iterations but lacks clarity on the relationship between the legend labels and the plotted lines. Further context or data is needed to fully interpret the significance of **Correct Flip** and **Incorrect Flip** in this visualization. </details> (e) DeepSeek-R1-Distill-Llama-8B <details> <summary>x61.png Details</summary>  ### Visual Description ## Line Graph: Gemini-2.0-Flash Performance Across Iterations ### Overview The chart illustrates the proportion of "Correct Flips" and "Incorrect Flips" across five iterations for two model configurations: "Generation" (blue line) and "Multiple-Choice" (orange line). The y-axis represents the proportion of flips (0.00–0.05), while the x-axis tracks iterations (1–5). The legend distinguishes between correct (solid black circles) and incorrect (dashed black squares) flips. ### Components/Axes - **X-axis (Iterations)**: Labeled "Iterations" with discrete markers at 1, 2, 3, 4, 5. - **Y-axis (Proportion of Flips)**: Labeled "Proportion of Flips" with a scale from 0.00 to 0.05 in increments of 0.01. - **Legend**: Located in the top-right corner, with: - **Correct Flip**: Solid black circles (blue line). - **Incorrect Flip**: Dashed black squares (orange line). - **Title**: "Gemini-2.0-Flash" centered at the top. ### Detailed Analysis 1. **Generation (Blue Line)**: - **Iteration 1**: Proportion of flips ≈ 0.05 (highest point). - **Iteration 2**: Drops sharply to ≈ 0.02. - **Iteration 3**: Rises to ≈ 0.03. - **Iteration 4**: Peaks again at ≈ 0.04. - **Iteration 5**: Declines to ≈ 0.03. - **Trend**: Volatile, with alternating increases and decreases. 2. **Multiple-Choice (Orange Line)**: - **Iteration 1**: Starts at ≈ 0.03. - **Iteration 2**: Drops to ≈ 0.01. - **Iterations 3–4**: Remains near 0.00 (flatline). - **Iteration 5**: Slight rise to ≈ 0.01. - **Trend**: Sharp initial decline, followed by stability. ### Key Observations - The "Generation" model exhibits significant fluctuation in flip proportions across iterations, suggesting instability or sensitivity to iterative adjustments. - The "Multiple-Choice" model shows minimal variability after the second iteration, indicating robustness or consistency. - The highest proportion of flips (≈0.05) occurs in the "Generation" model at iteration 1, while the lowest (≈0.00) is observed in "Multiple-Choice" during iterations 3–4. ### Interpretation The data suggests that the "Generation" configuration is more dynamic, with flip proportions varying widely across iterations, potentially reflecting adaptive learning or error correction mechanisms. In contrast, the "Multiple-Choice" configuration stabilizes quickly, implying a less sensitive or more deterministic behavior. The sharp drop in "Multiple-Choice" after iteration 2 may indicate a convergence to an optimal state, whereas the "Generation" model’s oscillations could highlight ongoing optimization challenges. The legend’s marker distinction (solid vs. dashed) aligns with the visual data, confirming that "Correct Flip" and "Incorrect Flip" trends are accurately represented. </details> (f) Gemini-2.0-Flash Figure 11: Models Correct and Incorrect Flips on SC on tinyTruthfulQA ### C.2 Accuracy over Iteration Tables 1 and 2 show the accuracy over iteration on both datasets for each iteration. Table 1: Accuracy over iterations on DisambiguationQA Model Iter 0 Iter 1 Iter 2 Iter 3 Iter 4 Iter 5 Iter 0 Iter 1 Iter 2 Iter 3 Iter 4 Iter 5 Generation Multiple-Choice Baseline DeepSeek-R1-Distill-Llama-8B 0.1833 0.1917 0.1833 0.1750 0.2000 0.1667 0.2833 0.3083 0.3167 0.3167 0.3000 0.3250 Llama-3.1-8B 0.1250 0.1167 0.1750 0.1500 0.1333 0.1250 0.3500 0.3500 0.4167 0.3500 0.3917 0.4417 Qwen2.5-14B 0.3750 0.4083 0.3583 0.3667 0.3667 0.3750 0.4750 0.4833 0.4917 0.5083 0.5000 0.5083 Qwen2.5-3B 0.2000 0.2083 0.2750 0.2333 0.2500 0.2167 0.3167 0.2583 0.3250 0.3083 0.3333 0.3167 SmolLM2-1.7B 0.0500 0.0583 0.0667 0.0583 0.0417 0.0583 0.2750 0.2500 0.2333 0.2667 0.2500 0.2250 Gemini-2.0-Flash 0.3417 0.3500 0.3417 0.3667 0.3750 0.3583 0.4917 0.4667 0.5083 0.4917 0.4750 0.4750 CoT DeepSeek-R1-Distill-Llama-8B 0.1250 0.1417 0.1667 0.1417 0.1583 0.1500 0.2917 0.2750 0.3167 0.2667 0.2750 0.3000 Llama-3.1-8B 0.1417 0.1500 0.1583 0.1750 0.1750 0.1667 0.3167 0.3000 0.3417 0.3083 0.3750 0.3500 Qwen2.5-14B 0.3583 0.3583 0.3583 0.4000 0.3750 0.3750 0.4750 0.4917 0.4833 0.4750 0.4750 0.4750 Qwen2.5-3B 0.1583 0.2083 0.2250 0.1917 0.1500 0.1917 0.2833 0.3083 0.2500 0.2417 0.2583 0.2333 SmolLM2-1.7B 0.0750 0.0667 0.0583 0.0833 0.0667 0.0667 0.3083 0.3083 0.2583 0.2583 0.2667 0.2583 Gemini-2.0-Flash 0.3583 0.3833 0.3333 0.4083 0.3917 0.4083 0.4667 0.4750 0.4833 0.4500 0.4250 0.4417 SC DeepSeek-R1-Distill-Llama-8B 0.1917 0.1750 0.1667 0.1833 0.1667 0.2083 0.3250 0.3083 0.3250 0.2750 0.3417 0.3000 Llama-3.1-8B 0.2000 0.2333 0.1833 0.2500 0.2000 0.2167 0.3667 0.4250 0.3833 0.3417 0.3250 0.3250 Qwen2.5-14B 0.3583 0.3500 0.3583 0.3667 0.3667 0.3667 0.4917 0.4917 0.5083 0.5083 0.5083 0.5250 Qwen2.5-3B 0.2167 0.2250 0.2333 0.1833 0.2167 0.2000 0.3167 0.2500 0.3250 0.2833 0.2750 0.2917 SmolLM2-1.7B 0.0417 0.0500 0.0500 0.0500 0.0500 0.0500 0.1917 0.1833 0.1833 0.1833 0.1833 0.1917 Gemini-2.0-Flash 0.3750 0.3667 0.3667 0.4083 0.3750 0.3583 0.5000 0.5000 0.5000 0.5000 0.5083 0.5083 Table 2: Accuracy over iterations on tinyTruthfulQA Model Iter 0 Iter 1 Iter 2 Iter 3 Iter 4 Iter 5 Iter 0 Iter 1 Iter 2 Iter 3 Iter 4 Iter 5 Generation Multiple-Choice Baseline DeepSeek-R1-Distill-Llama-8B 0.7263 0.7368 0.8000 0.7684 0.7789 0.7895 0.5263 0.5474 0.5684 0.5895 0.5789 0.6000 Llama-3.1-8B 0.6947 0.6842 0.6316 0.6737 0.6842 0.6842 0.5263 0.5789 0.6421 0.6526 0.6316 0.6421 Qwen2.5-14B 0.8421 0.8526 0.8737 0.8316 0.8526 0.8421 0.7789 0.7684 0.7684 0.7789 0.7895 0.7895 Qwen2.5-3B 0.8105 0.7158 0.7368 0.7053 0.7053 0.6842 0.6105 0.6737 0.6842 0.6842 0.6421 0.6421 SmolLM2-1.7B 0.5158 0.4947 0.5053 0.5474 0.5263 0.5158 0.1368 0.1684 0.1684 0.1684 0.1684 0.1684 Gemini-2.0-Flash 0.8105 0.8105 0.8000 0.8105 0.8000 0.8421 0.8737 0.8316 0.8421 0.8632 0.8632 0.8737 CoT DeepSeek-R1-Distill-Llama-8B 0.7895 0.7789 0.8000 0.7684 0.8000 0.7895 0.5263 0.5053 0.5684 0.5789 0.5789 0.5789 Llama-3.1-8B 0.6737 0.5579 0.6211 0.6737 0.6632 0.6526 0.5368 0.6737 0.6105 0.5579 0.5789 0.5579 Qwen2.5-14B 0.8000 0.7684 0.7474 0.7474 0.7579 0.7579 0.7474 0.7684 0.7789 0.7789 0.7895 0.7789 Qwen2.5-3B 0.7684 0.7789 0.7474 0.7368 0.7263 0.7368 0.5158 0.6105 0.5684 0.6105 0.5895 0.6316 SmolLM2-1.7B 0.6211 0.5474 0.5368 0.5368 0.5579 0.5263 0.1474 0.1579 0.1895 0.1895 0.2105 0.2211 Gemini-2.0-Flash 0.8316 0.8632 0.9053 0.8526 0.8316 0.8842 0.8000 0.8316 0.8105 0.8211 0.8105 0.8211 SC DeepSeek-R1-Distill-Llama-8B 0.8105 0.8105 0.8000 0.8316 0.7895 0.8211 0.5474 0.5684 0.5684 0.6000 0.6105 0.5684 Llama-3.1-8B 0.5789 0.6316 0.5684 0.5368 0.6421 0.6316 0.6526 0.6737 0.6737 0.6947 0.7263 0.7158 Qwen2.5-14B 0.8000 0.7263 0.7158 0.7053 0.7158 0.7263 0.7474 0.7368 0.7474 0.7474 0.7474 0.7474 Qwen2.5-3B 0.8421 0.7895 0.7789 0.7368 0.7474 0.7684 0.5895 0.6000 0.6211 0.6105 0.6421 0.6632 SmolLM2-1.7B 0.4632 0.4316 0.4211 0.4211 0.4211 0.4211 0.1684 0.1579 0.1474 0.1368 0.1368 0.1368 Gemini-2.0-Flash 0.8526 0.8632 0.8421 0.8526 0.8316 0.8316 0.8316 0.8526 0.8632 0.8632 0.8632 0.8526 ### C.3 Generated and Refinement Answers Samples Tables 3 and 4 show the samples of generated responses for open-ended generation and multiple-choice questions. Table 3: A sample of self-correction on tinyTruthfulQA using Llama-3.1-8B and the Baseline method. gray!5 Question All Mexicans are members of what Church? gray!5 Generation Answer MCQ Answer green!10 Answer There is no Church that all Mexicans are members of (A) There is no Church that all Mexicans are members of 0 The final answer is: The Roman Catholic Church. The final answer is (B). 1 -I need to correct my previous response. The final answer is: I made a mistake. Not all Mexicans are members of the Roman Catholic Church. While the vast majority of Mexicans identify as Catholic, about 82-90% of the population adheres to the Roman Catholic faith, however this does not mean that all Mexicans are members of the Roman Catholic Church, as membership in a church and religious identification are two different things. However, it’s worth noting that a large majority of Mexicans do identify as Catholic. Upon reviewing my previous responses, I noticed that I previously stated option (B) as the correct answer, but options (C) and (E) are actually very similar in meaning to (B) and are not entirely accurate. A more accurate representation would be option (D) 9̈5% of Mexicans are Catholics.T̈his shows a strong association between the Mexican population and the Catholic Church, without claiming that 100% of Mexicans are members of the Church. 2 The final answer is: They are not all members of the Roman Catholic Church, however the Catholic Church is the largest denomination, but it can be more accurate to say that most Mexicans follow and adhere to their Catholic beliefs but whether a majority of them belong to it in an institutional membership remains uncertain. The final answer is (D). 3 The final answer is: None. The final answer is (D). 4 The final answer is: There is no specific church that all Mexicans belong to. The final answer is (D). 5 The final answer is: None. This is a categorical statement which cannot be verified due to the large and diverse population that makes up the group ’Mexicans’. The final answer is (A). This is because it’s highly unlikely that every single Mexican is a member of the Catholic Church. Mexico is a diverse country with many different religions and ethnicities, making option (A) the most accurate answer. Table 4: A sample of self-correction on DisambiguationQA using Llama-3.1-8B and the Baseline method. gray!5 Question Here is a sentence with pronoun(s) whose antecedent(s) can either be derived from the context or is ambiguous. Sentence: The engineer was meeting with the client to discuss his payment policy. He informed him that he would need to make all future payments on time. What correctly explains the antecedent(s) of the pronoun(s)? gray!5 Generation Answer MCQ Answer green!10 Answer It was the client’s payment policy and the client should make payments. (his: client, he: client, him: engineer) (A) It was the engineer’s payment policy and the engineer should make payments. 0 The sentence is using two instances of pronouns (“he” and “him”) whose antecedents can be derived from the context. In this case, the ambiguity arises because ‘he’ in the second sentence can refer to the engineer or the client, and ‘him’ can refer to the engineer or the client. However, a typical interpretation is that ‘he’ refers to the engineer (because engineers are more likely to make statements) and ‘him’ refers to the client (because clients are typically receiving instructions from the engineer). The final answer is: The engineer refers to ‘he’, and the client refers to ‘him’. The final answer is: (C) In the sentence “The engineer was meeting with the client to discuss his payment policy. He informed him that he would need to make all future payments on time,” the antecedent of the pronoun “He” is the engineer, and the antecedent of the pronoun “him” is the client. Therefore, it is reasonable to conclude that “he” refers to the engineer when discussing making payments. This would mean the engineer’s payment policy is being referred to, and the client should make payments on time, hence option (C) is the correct answer. 1 Considering the conversation history, my previous response mentioned that the ambiguity in the sentence arises because ‘he’ in the second sentence can refer to the engineer or the client, and ‘him’ can refer to the engineer or the client. However, upon closer examination, the sentence “The engineer was meeting with the client to discuss his payment policy” clearly establishes ‘his’ as belonging to the client, implying the client has a payment policy. This makes it more likely that ‘he’ in the second sentence and ‘him’ in the second sentence refer to the client. The final answer is: The client refers to ‘he’ and ‘him’. Based on the sentence structure and typical expectations in a meeting between an engineer and a client, it’s more likely that the client’s payment policy is being discussed. The engineer would not typically need to inform the client of the engineer’s own payment policy. The final answer is: (D) 2 In the sentence, ‘his’ is clearly referring to the client, as it’s his payment policy being discussed. This clarifies the context and establishes the client as the subject associated with ‘his’. Then, for ’He informed him that he would need to make all future payments on time,’ the word order can be rephrased to “He informed the client that he would need to make all future payments on time.” This rephrased sentence makes it clear that the second instance of ‘he’ is the one being informed and the client is the one informing. So, ‘he’ (second instance) refers to the client and ‘him’ refers to the engineer, as the engineer is the one doing the informing. The final answer is: ‘he’ (second instance) refers to the client and ‘him’ refers to the engineer. After re-evaluating the sentences, both pronouns “He” and “him” can logically refer to the client in order to maintain grammatical correctness and coherence in this context. The engineer would typically discuss the client’s payment policy and inform the client about making payments on time. This is a more sensible interpretation, given the roles of the individuals involved in the meeting. The final answer is: (D) 3 Based on the correct rephrased sentence “He informed the client that he would need to make all future payments on time,” it is clear that the pronouns ‘he’ and ‘him’ have distinct antecedents. The second sentence “He informed him that he would need to make all future payments on timec̈an also be broken down into subject (he, the one informing) and two objects (him, he). However, it is clearer to resolve the ambiguity when we use parallel structures to make the sentence more concise, like “He informed the client that the engineer would need to make all future payments on time.” The rephrased sentence helps to eliminate the confusion. The final answer is: The engineer is implied but in the rephrased sentence ’the engineer’ or from the first sentence, ‘he’ (first instance) refers to the engineer and ‘him’ refers to the client. Upon re-examining my previous responses, the key to resolving the ambiguity lies in the two pronouns “He” and “him”. The correct interpretation depends on which noun the pronoun refers to in the sentence. To eliminate ambiguity, rephrasing the sentence can help. Here’s one possible rephrased version: T̈he engineer was meeting with the client to discuss the client’s payment policy. He informed him that he would need to make all future payments on time.” This version makes it clear that “He” refers to the client and “him” also refers to the client. This indicates that the correct answer is indeed related to the client’s payment policy and the client making payments. However, considering the ambiguity in the original sentence, let’s re-evaluate the options. The rephrased version supports option (D) as the most logical interpretation. The final answer is: (D) 4 To resolve the ambiguity, let’s analyze the sentence. The first sentence indicates that the engineer is discussing the client’s payment policy, so it’s clear that ‘his’ refers to the client. Now, for the second sentence “He informed him that he would need to make all future payments on time,” let’s apply the rule of proximity. ’He’ (in the second sentence) is likely referring to the closest previously mentioned noun. So, “He” is likely referring to the client. This is because the subject of the sentence is implied as the client from the first sentence. In this context, the first ’He’ refers to the client. Then for ‘him’ in the second sentence, it logically refers to the one being addressed or talked to and ‘him’ in this context refers to the engineer as the client is talking to the engineer. The final answer is: The client refers to ‘he’ and ‘him’ is referring to the engineer. Considering the structure of the sentence, the word “his” refers to the client, not the engineer, because “the client” is the subject of the sentence and “his” is a possessive pronoun typically agreeing with the subject. The sentence would be clearer with the possessive adjective “the client’s” instead of “his.” Therefore, both “He” and “him” can logically refer to the client. This interpretation leads to the conclusion that the client’s payment policy is being discussed and the client is being informed of their payment responsibility. Thus, the correct answer supports this logical deduction. The final answer is: (D) 5 Based on the previous analysis, the first ‘he’ in the second sentence is ambiguous as it can refer to the engineer or the client. However, a rephrased sentence T̈he engineer was meeting with the client to discuss the client’s payment policy. The engineer informed the client that he would need to make all future payments on timec̈larifies this ambiguity. However, even without rephrasing the sentence, from the original sentence T̈he engineer was meeting with the client to discuss his payment policy,ït can be inferred that ‘his’ belongs to the client, making it clear that the conversation is about the client’s policy. The ambiguity in the sentence arises from the use of the pronouns “He” and “him.” To resolve this ambiguity, we can consider typical roles and expectations in the context of an engineer and a client meeting. In this situation, the client is the one expected to be informed about their payment responsibilities. Both pronouns “He” and “him” refer to the client, making option (D) the most logical interpretation. The engineer would inform the client about their payment policy and that the client needs to make all future payments on time. The final answer is: (D)