2510.16062

# Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs > Equal contributions. † Corresponding authors: Pan Zhou and Lixing Chen. ## Abstract Self-correction of large language models (LLMs) emerges as a critical component for enhancing their reasoning performance. Although various self-correction methods have been proposed, a comprehensive evaluation of these methods remains largely unexplored, and the question of whether LLMs can truly correct themselves is a matter of significant interest and concern. In this study, we introduce CorrectBench, a benchmark developed to evaluate the effectiveness of self-correction strategies, including intrinsic, external, and fine-tuned approaches, across three tasks: commonsense reasoning, mathematical reasoning, and code generation. Our findings reveal that: 1) Self-correction methods can improve accuracy, especially for complex reasoning tasks; 2) Mixing different self-correction strategies yields further improvements, though it reduces efficiency; 3) Reasoning LLMs have limited optimization under additional self-correction methods and have high time costs. Interestingly, a comparatively simple chain-of-thought (CoT) baseline demonstrates competitive accuracy and efficiency. These results underscore the potential of self-correction to enhance LLM’s reasoning performance while highlighting the ongoing challenge of improving their efficiency. Consequently, we advocate for further research focused on optimizing the balance between reasoning capabilities and operational efficiency. Project Page: https://correctbench.github.io/ ## 1 Introduction The rapid advancement of large language models (LLMs), exemplified by GPT-3.5 Ye2023ACC and LLaMA 3 Dubey2024TheL3 , has precipitated a transformative shift in artificial intelligence (AI), yielding state-of-the-art performance across diverse tasks tie2025survey . Specifically, these tasks include content generation Agossah2023LLMBasedIF , natural language understanding Kuang2024NaturalLU , and complex decision-making Yang2024EvaluatingWM , all of which have been revolutionized by the extensive pretraining and sophisticated architectures of LLMs. Notably, the introduction of frameworks like Chain-of-Thought (CoT) Wei2022ChainOT has further expanded LLM’s capacity for multi-step reasoning, enabling them to tackle more intricate tasks. Despite these advancements, ensuring the reliability and accuracy of model outputs, especially for reasoning-intensive tasks, remains a formidable challenge. In response, recent works have focused on self-correction strategies aimed at refining LLMs’ decision-making processes Kamoi2024WhenCL ; Lin2024CriticBenchBL through iterative revision. Pioneering approaches such as RARR Gao2022RARRRA , Refiner Paul2023REFINERRF , and CRITIC Gou2023CRITICLL illustrate the potential of integrating feedback loops and corrective components into model architectures. However, these approaches often yield inconsistent gains across different tasks, prompting deeper questions about their capability of correction and generalizability. This observation motivates the central question: Can LLMs truly correct themselves? Moreover, it remains unclear whether more intricate self-correction schemes necessarily translate into superior overall performance. To address these issues, this paper presents CorrectBench, a systematic benchmark for investigating how self-correction methods affect LLMs performance across multiple tasks. Building on a recent survey of self-correction approaches Kamoi2024WhenCL , we categorize such methods into three primary categories (i.e., intrinsic correction, external correction and fine-tuned correction), and select 11 representative methods from them. Additionally, we establish comparative baselines using both the widely adopted CoT Wei2022ChainOT and a standard prompting strategy denoted as ‘Base’ (detailed in Appendix G.1). <details> <summary>x1.png Details</summary>  ### Visual Description ## Radar Chart Comparison: AI Model Performance Across Benchmarks ### Overview The image displays three radar charts (spider plots) comparing the performance of 11 different large language models (LLMs) across six standardized benchmarks. The charts are organized to show performance under three different conditions: a baseline ("Base"), an "Intrinsic Correction" condition ("S1"), and an "External Correction" condition ("S2"). Each chart plots the same set of models on the same six axes, allowing for a direct visual comparison of how model performance changes across the three conditions. ### Components/Axes * **Chart Titles (Top Center):** * Left Chart: `Base (Baseline)` * Middle Chart: `S1 (Intrinsic Correction)` * Right Chart: `S2 (External Correction)` * **Axes (Radial Spokes):** Six benchmarks are plotted as axes radiating from the center. The axes are labeled at their outer ends. Clockwise from the top: 1. `GPQA` 2. `CS-QA` 3. `HotpotQA` 4. `HumanEval` 5. `MATH` 6. `AQUA` * **Scale (Concentric Circles):** The radial scale is marked by concentric circles representing performance scores. The innermost circle is labeled `0.2`, the next `0.4`, the next `0.6`, and the outermost labeled `0.8`. The center point represents a score of 0. * **Legend (Bottom):** A comprehensive legend is provided below the three charts, mapping line colors and styles to specific AI models. The legend is organized in three columns. * **Column 1:** * `LLaMA3.1-8B-Instruct` (Solid, light teal line) * `LLaMA3.1-70B-Instruct` (Solid, light yellow-green line) * **Column 2:** * `Qwen2.5-7B-Instruct` (Solid, light purple line) * `Qwen2.5-72B-Instruct` (Solid, salmon pink line) * **Column 3:** * `Claude3.5-Sonnet` (Solid, medium blue line) * `GPT-3.5` (Solid, orange line) * **Column 4:** * `GPT-4o` (Solid, light green line) * `QWQ-32B-Instruct` (Dashed, pink line) * **Column 5:** * `DeepSeek-V3` (Dashed, purple line) * `DeepSeek-R1` (Dashed, light blue line) * **Column 6:** * `o3-mini` (Dashed, light pink line) ### Detailed Analysis **1. Base (Baseline) Chart:** * **Trend:** Most models show a similar, somewhat irregular pentagonal shape, indicating varied performance across benchmarks. Performance is generally strongest on `GPQA` and `CS-QA` (closer to the 0.8 ring) and weakest on `MATH` and `AQUA` (often between 0.4 and 0.6). * **Key Data Points (Approximate):** * **Top Performers (Outermost lines):** `Claude3.5-Sonnet` (blue) and `GPT-4o` (green) consistently form the outermost shape, indicating the highest overall scores. They approach or exceed 0.8 on `GPQA` and `CS-QA`. * **Mid-Tier:** `Qwen2.5-72B-Instruct` (salmon), `LLaMA3.1-70B-Instruct` (yellow-green), and `GPT-3.5` (orange) form a cluster just inside the top performers. * **Lower-Tier:** `LLaMA3.1-8B-Instruct` (teal) and `Qwen2.5-7B-Instruct` (purple) are generally the innermost lines, indicating lower scores, particularly on `MATH` and `AQUA` where they dip near or below 0.4. * **Notable Outlier:** The dashed pink line for `QWQ-32B-Instruct` shows a very distinct shape. It has a pronounced spike towards `HotpotQA` (near 0.8) but is the innermost line on `MATH` and `AQUA` (below 0.4), indicating highly specialized performance. **2. S1 (Intrinsic Correction) Chart:** * **Trend:** The overall shapes expand outward compared to the Base chart, suggesting a general improvement in scores across most models and benchmarks after intrinsic correction. The relative ordering of models remains similar. * **Key Changes:** * The gap between the top performers (`Claude3.5-Sonnet`, `GPT-4o`) and the mid-tier narrows slightly. * The lower-tier models (`LLaMA3.1-8B-Instruct`, `Qwen2.5-7B-Instruct`) show noticeable improvement, moving further from the center. * The specialized shape of `QWQ-32B-Instruct` (dashed pink) becomes less extreme; its low scores on `MATH`/`AQUA` improve, while its high score on `HotpotQA` remains strong. **3. S2 (External Correction) Chart:** * **Trend:** This chart shows the most significant expansion and convergence of shapes. The performance of nearly all models improves further, and the differences between them become much smaller. The lines are tightly clustered near the outer edge of the chart. * **Key Changes:** * **Massive Convergence:** Almost all models now score between approximately 0.7 and 0.9 on all six benchmarks. The distinct performance profiles seen in the Base chart are largely erased. * **Top Cluster:** `Claude3.5-Sonnet`, `GPT-4o`, `Qwen2.5-72B-Instruct`, `LLaMA3.1-70B-Instruct`, and `GPT-3.5` are nearly indistinguishable at the top. * **Dramatic Improvement:** The smaller models (`LLaMA3.1-8B-Instruct`, `Qwen2.5-7B-Instruct`) and the specialized `QWQ-32B-Instruct` show the most dramatic gains, now performing at a level comparable to the much larger models in the baseline. * **New Entrants:** The dashed lines for `DeepSeek-V3`, `DeepSeek-R1`, and `o3-mini` are also present in this cluster, indicating high performance under the S2 condition. ### Key Observations 1. **Performance Hierarchy:** In the baseline, a clear hierarchy exists: proprietary models (Claude, GPT-4o) > large open-source models (70B/72B) > smaller open-source models (7B/8B). 2. **Benchmark Difficulty:** `MATH` and `AQUA` appear to be the most challenging benchmarks for all models in the baseline, as scores are consistently lowest on these axes. 3. **Specialization:** The `QWQ-32B-Instruct` model exhibits a unique performance profile in the baseline, excelling at `HotpotQA` but struggling with `MATH` and `AQUA`. 4. **Correction Impact:** Both "Intrinsic" (S1) and especially "External" (S2) correction methods lead to substantial performance gains. The S2 condition acts as a powerful equalizer, dramatically reducing the performance gap between model sizes and architectures. 5. **Diminishing Returns:** The improvement from Base to S1 is significant, but the leap from S1 to S2 is even more pronounced, suggesting the external correction method is highly effective. ### Interpretation This visualization demonstrates the profound impact of correction techniques on LLM benchmark performance. The data suggests that: * **Raw Capability vs. Corrected Performance:** The baseline ("Base") chart reflects the raw, unaided reasoning and knowledge capabilities of the models, where scale (parameter count) and training data quality create a clear performance stratification. * **The Power of External Tools/Methods:** The dramatic convergence in the "S2 (External Correction)" chart implies that when models are augmented with external correction mechanisms (which could involve tools, retrieval-augmented generation, or specialized verification modules), their inherent limitations in specific domains (like mathematical reasoning) can be largely overcome. This narrows the gap between smaller and larger models. * **Benchmark Sensitivity:** The consistent difficulty of `MATH` and `AQUA` in the baseline highlights these as areas where model reasoning is most fragile without assistance. The fact that correction methods most dramatically improve scores on these axes underscores their value for practical applications requiring robust reasoning. * **Strategic Implication:** For developers, this indicates that investing in external correction systems (S2) may yield greater performance improvements and cost-efficiency (by enabling smaller models to perform like larger ones) than simply scaling up model size alone. The charts argue for a paradigm where model capability is a combination of base model intelligence and the sophistication of its supporting correction ecosystem. </details> Figure 1: Comparison of different LLMs across various self-correction types and task scenarios. For a rigorous and comprehensive assessment, we construct two curated datasets: CorrectBench-base, which integrates diverse subtasks with 3,825 question-answer pairs drawn from 7 distinct subdatasets, and CorrectBench-test, a curated collection of question-answer pairs specifically tailored for correction-oriented experiments. These subdatasets cover three principal tasks: commonsense reasoning Putri2024CanLG , mathematical reasoning Hendrycks2021MeasuringMP , and code generation Chen2021EvaluatingLL . We then apply these subdatasets to both instruction-based LLMs (e.g., LLaMA 3.1-8B-Instruct meta2024llama3 , Qwen 2.5-7B-Instruct Yang2024Qwen25TR , GPT-4o hurst2024gpt , Claude 3.5-Sonnet AhtropicClaude ) and reasoning LLMs This paper defines “reasoning LLMs” as those models that are specifically enhanced with complex reasoning capabilities through a post-training optimization process. (e.g., DeepSeek-V3 DeepSeekAI2024DeepSeekV3TR ). Figure 1 compares the baseline (‘Base’) performance with the mean performances of intrinsic and external correction methods separately, revealing that self-correction bolsters overall accuracy (detailed in Figure 8). Key insights. First, self-correction methods substantially enhance accuracy, particularly in complex reasoning tasks. Meanwhile, mixing multiple methods, while improving accuracy, incurs higher computational costs and reduced efficiency. For reasoning LLMs, these methods offer only marginal gains with increased time usage. Interestingly, the CoT Wei2022ChainOT strategy demonstrates a favorable trade-off between operational efficiency and overall accuracy, challenging the prevailing assumption that more sophisticated correction frameworks inherently produce superior outcomes. To summarize, our work provides three key contributions: - A Comprehensive Benchmark. We propose CorrectBench, the first benchmark devised to systematically evaluate the impact of self-correction on LLMs inference. Spanning multiple tasks and model categories, CorrectBench offers a robust, reproducible platform for methodological comparisons. - Two Datasets. We present CorrectBench-base and CorrectBench-test, both meticulously constructed to encompass a broad range of question-answer formats and reasoning scenarios, facilitating thorough assessments of different correction methods. - Insights and Implications. Our empirical findings show that self-correction substantially advances LLMs’ performance, especially on tasks demanding extensive reasoning. However, the increased computational load of mixing multiple correction strategies must be weighed against potential accuracy gains. Moreover, for reasoning LLMs, additional correction methods provide limited improvements, emphasizing critical cost-efficiency concerns for practical applications. ## 2 CorrectBench: A Benchmark of Self-Correction in LLMs CorrectBench is a systematically designed benchmark that quantifies the extent to which various correction methods improve model outputs in reasoning-intensive scenarios. As illustrated in Figure 2, CorrectBench characterizes self-correction along three principal dimensions: Task Scenario, Self-Correction Type, and LLM Type. The evaluation pipeline begins with selecting a specific task scenario and dataset, followed by applying a chosen correction method, and concludes with assessing the model’s iterative self-correction process across diverse LLMs. <details> <summary>x2.png Details</summary>  ### Visual Description ## Diagram: LLM Correction Methods and Task Examples ### Overview The image is a technical diagram illustrating three different correction methods (S1, S2, S3) for Large Language Models (LLMs), alongside three example tasks (T1, T2, T3) that demonstrate the types of problems these methods might address. The diagram is divided into two primary sections: a left panel with task examples and a right panel detailing the correction workflows. A legend and model categorization are provided at the bottom. ### Components/Axes **Left Panel (Task Examples):** * **T1: Commonsense Reasoning** * **Question:** "Which of the following is the largest planet in our solar system? A) Earth B) Mars C) Jupiter D) Venus?" * **Answer:** "C." * **T2: Mathematical Reasoning** * **Question:** "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?" * **Answer:** "72." * **T3: Code Generation** * **Question:** "Write a Python function that takes two integers as input and returns their sum." * **Answer:** "def add_numbers(a, b): return a + b:" **Right Panel (Correction Methods):** * **S1: Intrinsic Correction** * A cyclical process involving a "Base LLM" (black icon), a "Refine LLM" (blue icon), a "Response" (clipboard icon), and a "Feedback LLM" (two red figures icon). Arrows indicate a closed-loop data flow between these components. * **S2: External Correction** * Similar cyclical process to S1, but includes an external dashed box containing a "Search Engine" and a "Code Executor." An "Extra Data Flow" (dashed arrow) connects these external tools to the "Refine LLM." * **S3: Fine-tuned Correction** * Similar cyclical process, but the "Refine LLM" is replaced by a "Fine-tuned Refine LLM" (green icon). A dashed arrow labeled "Fine-tuning" points from the "Base LLM" to the "Fine-tuned Refine LLM." **Legend (Bottom Right of Right Panel):** * **Icons & Labels:** * Black spiral icon: "Base LLM" * Blue spiral icon: "Refine LLM" * Green spiral icon: "Fine-tuned Refine LLM" * Clipboard icon: "Response" * Two red figures icon: "Feedback LLM" * Three small squares icon: "Generated Feedback" * **Arrows:** * Solid black arrow: "Data Flow" * Dashed black arrow: "Extra Data Flow" **Model Categorization (Bottom of Image):** * **M1: Instruction-based LLM** (Blue background) * Logos/Names: Meta, Qwen, Claude, OpenAI * **M2: Reasoning LLM** (Green background) * Logos/Names: QwQ-32B, deepseek ### Detailed Analysis The diagram presents a comparative view of LLM self-correction paradigms. 1. **Task Context (Left Panel):** The three tasks (T1-T3) represent common LLM evaluation benchmarks: factual recall, multi-step arithmetic, and functional code synthesis. They establish the problem domain where correction methods are applied. 2. **Correction Workflows (Right Panel):** * **S1 (Intrinsic):** Shows a purely internal loop. The Base LLM generates a response, which is evaluated by a Feedback LLM. The generated feedback is then used by a Refine LLM to improve the response, creating a closed system. * **S2 (External):** Augments the intrinsic loop by incorporating external tools (Search Engine, Code Executor). The Refine LLM can query these tools for factual verification or code execution, with the results flowing back as extra data to inform the refinement. * **S3 (Fine-tuned):** Replaces the general Refine LLM with a specialized, fine-tuned version. The fine-tuning process is explicitly shown as a separate step originating from the Base LLM, suggesting the refinement model is a specialized derivative of the base model. 3. **Model Landscape (Bottom):** The diagram categorizes contemporary LLMs into two groups: "Instruction-based LLM" (M1), which includes major commercial and open-source models, and "Reasoning LLM" (M2), which lists specific models like QwQ-32B and DeepSeek, implying a focus on enhanced reasoning capabilities. ### Key Observations * The core cyclical structure (Base LLM -> Response -> Feedback LLM -> Refine LLM -> back to Base LLM) is consistent across all three methods (S1, S2, S3), indicating a common architectural pattern for iterative correction. * The primary differentiator between the methods is the source of refinement intelligence: internal models (S1), augmented by external tools (S2), or a specialized fine-tuned model (S3). * The "Generated Feedback" icon (three squares) appears in all three workflows, positioned between the Feedback LLM and the Refine LLM, signifying the output of the evaluation step. * The model categorization (M1 vs. M2) is presented separately from the correction methods, suggesting these are the types of base models that could be subjected to the S1/S2/S3 correction processes. ### Interpretation This diagram serves as a conceptual framework for understanding how LLMs can be made to self-correct and improve their outputs. It argues that correction is not a single technique but a spectrum of approaches with increasing complexity and resource requirements. * **S1 (Intrinsic)** represents a baseline, self-contained approach relying on the model's own knowledge and a separate feedback mechanism. * **S2 (External)** acknowledges the limitations of an LLM's static knowledge and internal reasoning by integrating dynamic, real-world information and execution capabilities, making it suitable for tasks requiring up-to-date facts or verified code execution. * **S3 (Fine-tuned)** suggests a more permanent, integrated solution where the refinement capability is baked into a specialized model, potentially offering more efficient and consistent corrections for specific domains. The inclusion of the task examples (T1-T3) grounds these abstract methods in practical applications, showing the types of errors (factual, arithmetic, syntactic) each correction method might be designed to address. The model categorization at the bottom hints that the choice of base model (Instruction-based vs. Reasoning-focused) may influence the effectiveness of these correction strategies. Overall, the image maps a technical pathway from problem identification (tasks) to solution architectures (correction methods) within the current LLM ecosystem. </details> Figure 2: An overview of the CorrectBench framework. Iterative Self-Correction Paradigm. In a standard LLM evaluation, the model generates an initial response $r_0$ to a question $q$ given an initial prompt $p_0$ , formally $r_0=M(q,p_0)$ , where $M$ denotes the LLM. While this process becomes iterative in the self-correction paradigm. Specifically, for the $k$ -th iteration, $p_k=p_k-1∪ r_k-1,r_k=M(q,p_k)$ , where $p_k$ is the updated prompt that includes the previous response $r_k-1$ . After $K$ iterations, the final output $r_K$ reflects the model’s corrected response. This iterative mechanism enables the model to continually refine its output based on newly revealed errors or inconsistencies. <details> <summary>x3.png Details</summary>  ### Visual Description ## Diagram: Iterative Correction Method for AI Responses ### Overview The image is a technical diagram illustrating an iterative process for refining AI-generated responses. It depicts a workflow where an initial response to a question undergoes multiple stages of correction, with each stage producing a new response that builds upon the previous one. The diagram uses symbolic icons, labeled components, and directional arrows to convey the flow and methodology. ### Components/Axes The diagram is divided into two primary regions: 1. **Main Flowchart (Top Region):** This is the core process flow, arranged horizontally from left to right. 2. **Legend (Bottom Region):** A dashed box containing explanations for the symbols used in the flowchart. **Main Flowchart Components & Labels:** * **Input (Top-Left):** The word "Question" with a downward arrow pointing to an AI model icon. * **AI Model Icon (Left):** A stylized, black knot-like symbol representing an AI or language model. * **Initial Output (Left):** The text "Initial Response" below the AI model icon. * **Process Nodes (Center to Right):** Three laptop icons, each labeled with a gear and a checkmark (✓) and cross (✗) above it. These represent the "Correction Method" at each stage. * **Response Labels (Below Laptops):** * `Response₁` (below the first laptop) * `Responseₙ₋₁` (below the second laptop, in blue text) * `Responseₙ` (below the third laptop, in green text) * **Connection Symbols:** * **Addition (⊕):** A circle with a plus sign inside, used to combine elements. * **Iteration Arrow:** A dashed blue arrow (`- - ->`). * **Flow Arrows:** Solid black and green arrows indicating the direction of the process. **Legend Components (Bottom Region):** * A small laptop icon with ✓/✗, labeled **"Correction Method"**. * The addition symbol (⊕), labeled **"Addition"**. * A dashed blue arrow, labeled **"Iteration"**. ### Detailed Analysis The process flow is as follows: 1. **Stage 1 (Left):** A "Question" is fed into an AI model, which generates an "Initial Response." 2. **First Correction Cycle:** * The "Initial Response" and the original "Question" are combined via an **Addition (⊕)** operation. * This combined input is processed by the first **Correction Method** (laptop icon). * The output is `Response₁`. 3. **Iterative Refinement (Center):** * `Response₁` is fed back into the process via an **Iteration** (dashed blue arrow). * It is combined with another element (implied to be the original question or context) via an **Addition (⊕)** operation. * This is processed by the second **Correction Method**. * The output is `Responseₙ₋₁` (indicating the penultimate response in a sequence). 4. **Final Stage (Right):** * `Responseₙ₋₁` is combined via an **Addition (⊕)** operation. * This is processed by the third **Correction Method**. * The final output is `Responseₙ` (the nth, or final, corrected response), indicated by a green arrow and green text. **Spatial Grounding:** The legend is positioned at the bottom, centered horizontally. The main flowchart progresses linearly from left to right. The "Iteration" dashed arrows create feedback loops between the correction stages, visually connecting the output of one stage back to the input of the next. ### Key Observations * **Iterative Nature:** The core pattern is a loop of correction and refinement, explicitly labeled as "Iteration." * **Symbolic Consistency:** The "Correction Method" icon (laptop with ✓/✗) is identical at each stage, suggesting the same type of process is applied repeatedly. * **Progressive Labeling:** The response labels (`Response₁`, `Responseₙ₋₁`, `Responseₙ`) clearly denote a sequence from the first to the final output. * **Color Coding:** Blue is used for the intermediate iteration step and its associated response label (`Responseₙ₋₁`), while green is used for the final output (`Responseₙ`), visually distinguishing the process stages. * **Combination Operation:** The "Addition (⊕)" symbol is used at every stage before a correction method is applied, indicating that inputs are always aggregated or combined before processing. ### Interpretation This diagram models a **self-correction or refinement pipeline for large language models (LLMs)**. It demonstrates a methodology where an AI's initial output is not final but serves as a draft. This draft is systematically improved through multiple passes of a "Correction Method." The process suggests a framework where: 1. The model's own output is treated as a candidate for improvement. 2. Each correction cycle likely involves evaluating the response (symbolized by the ✓/✗) and generating a better version. 3. The "Addition" step implies that the correction method doesn't work on the response in isolation; it likely incorporates the original question, previous context, or error signals to guide the refinement. The use of `ₙ₋₁` and `ₙ` is a mathematical convention indicating this process can be repeated for an arbitrary number of iterations (`n` times) until a satisfactory result is achieved. The diagram's purpose is to visually communicate a technical approach for enhancing AI response quality through iterative, algorithmic correction rather than relying on a single generation pass. </details> Figure 3: Mixture of different correction methods. Mixture Framework. While individual self-correction methods can improve model responses, it is plausible that integrating multiple methods may yield further improvements in accuracy. To examine these potential synergies, we propose the mixture framework, illustrated in Figure 3. The response of one correction method serves as input to the next, forming a dynamic pipeline of iterative refinements. This setup enables us to analyze how distinct self-correction methods interact, thereby guiding the development of optimal configurations for improving LLM’s performance. ### 2.1 Self-Correction Method CorrectBench comprehensively evaluates three distinct categories of self-correction methods: S1: Intrinsic Correction. This category focuses on the LLMs’ capacity to internally identify and correct errors without external tools. Methods such as RCI Kim2023LanguageMC , Self-Refine Madaan2023SelfRefineIR , CoVe Dhuliawala2023ChainofVerificationRH , and Reflexion Shinn2023ReflexionLA enable the LLMs to re-evaluate its prior reasoning steps and resolve inconsistencies based on its internal knowledge. S2: External Correction. In contrast to S1, S2 (e.g., Reflexion Shinn2023ReflexionLA , RARR Gao2022RARRRA , RATT Zhang2024RATTAT , CRITIC Gou2023CRITICLL ) leverages external resources, such as knowledge bases or Google search tools, to address gaps in the internal representation. This external support can correct factual inaccuracies or logical oversights, though it may constrain the model’s capacity for divergent reasoning. S3: Fine-tuned Correction. Fine-tuned correction methods including DCoT Puerto2024FineTuningWD , SCORE Zhang2024SmallLM , and SuperCorrect Yang2024SuperCorrectSA enhance LLMs’ self-correction performance through targeted fine-tuning. Although potentially effective, these methods require substantial training and are often limited by the scope and quality of the fine-tuning data. ### 2.2 Task Scenario CorrectBench investigates self-correction methods across three representative task scenarios: T1: Commonsense Reasoning. This scenario probes the model’s capacity to address factual or logical inconsistencies within everyday knowledge domains. Relevant datasets include HotpotQA Yang2018HotpotQAAD , CommonsenseQA Putri2024CanLG CommonsenseQA is represented as CS-QA in the following., and the more challenging GPQA Rein2023GPQAAG , which emphasizes complex reasoning. T2: Mathematical Reasoning. Datasets in this task scenario emphasize the detection and correction of errors in mathematical derivations, algebraic manipulations, and multi-step reasoning. Representative datasets include GSM8K Cobbe2021TrainingVT , AQUA Chen2019AQuAAA , and MATH Hendrycks2021MeasuringMP . T3: Code Generation. This scenario assesses the LLM’s ability to generate functionally correct and logically consistent code from natural language prompts. For instance, HumanEval Chen2021EvaluatingLL measures the LLM’s ability to detect and correct syntax errors, logical flaws, and other coding mistakes. ### 2.3 LLM Type To ensure a broad and realistic appraisal, CorrectBench encompasses two categories of LLMs: M1: Instruction-Based LLMs. LLMs are designed to follow user-provided instructions to generate relevant outputs, making them highly versatile across diverse tasks. This category includes both open-source and closed-source LLMs, distinguished by their accessibility and design paradigms. Open-source LLMs, such as Qwen 2.5-7B/70B-Instruct Yang2024Qwen25TR , as well as LLaMA 3.1-8B/70B-Instruct meta2024llama3 , offer transparency and flexibility for modification, enabling detailed analysis and fine-tuning. Conversely, closed-source LLMs, including OpenAI’s GPT-3.5 ouyang2022training , GPT-4o hurst2024gpt , and Anthropic’s Claude 3.5-Sonnet AhtropicClaude , excel in real-world tasks due to proprietary optimizations, but restrict direct access and customization for research purposes. M2: Reasoning LLMs. Reasoning LLMs are models specifically enhanced with advanced reasoning capabilities through targeted post-training optimization processes. These models are designed to excel in tasks requiring multi-step logical reasoning, often incorporating integrated self-correction mechanisms to refine their outputs. Representative examples include QWQ-32B-Instruct qwen2 , o3-mini and DeepSeek-R1 DeepSeekAI2024DeepSeekV3TR . DeepSeek-V3 DeepSeekAI2024DeepSeekV3TR , in particular, adopts an innovative approach to distilling reasoning capabilities from long chain-of-thought models, leveraging its predecessor. By integrating verification and reflection patterns from R1, DeepSeek-V3 achieves substantial improvements in reasoning accuracy while maintaining precise control over output style and length. ### 2.4 Research Question This study aims to elucidate the effectiveness of different self-correction methods in enhancing LLMs’ performance, addressing the following core research questions: [RQ1] To what extent can LLMs achieve accurate results by leveraging intrinsic (S1) and external (S2) self-correction methods S3 is analyzed separately due to dataset-specific constraints. without requiring further intervention? [RQ2] How does mixing multiple self-correction methods influence model accuracy and robustness, and what are the associated computational trade-offs? [RQ3] For reasoning LLMs with built-in correction mechanisms, to what extent can the above self-correction methods provide additional benefits? ## 3 Experiment Settings Dataset Preparation. To ensure consistency and reproducibility, we employ CorrectBench-test for experimental evaluations. For each dataset within CorrectBench-test, we randomly select 100 samples and subsequently refine this selection by excluding a minimal number of outliers or irrelevant instances, thereby ensuring a more precise representation of error patterns. Comprehensive details regarding the datasets and preprocessing steps are provided in Appendix B.1, thereby promoting transparency and reproducibility for subsequent research. Task and Model Selection. CorrectBench adopts a hierarchical strategy to evaluate self-correction across diverse task scenarios and LLM types. Specific datasets are selected to match the characteristics and objectives of each self-correction method, ensuring that the benchmark captures comprehensive error types and correction challenges. Further specifications regarding selections of datasets and LLMs are included in Appendix B.2, ensuring full reproducibility of the evaluation methodology. Evaluation Metrics. We employ both task-specific and judgment-based metrics to evaluate the self-correction capabilities of diverse tasks: 1) Task-Specific Metrics. These metrics are tailored to evaluate model performance across different tasks. For T1, accuracy is computed as: $ACC=\frac{1}{N}∑_i=1^NI(\hat{y}_i=y_i)$ , where $N$ is the total number of samples, $y_i$ is the ground truth, $\hat{y}_i$ is the predicted answer, and $I(·)$ is the indicator function. For T2, the solve rate represents the percentage of problems correctly solved by the model out of the total number of problems. For T3, pass@k evaluates whether at least one of the $k$ generated solutions for a problem passes all test cases. The final pass@k score is calculated as the average pass rate across all problems. 2) Judgment-Based Metrics. In cases where the model’s response is ambiguous or incomplete, we conduct human evaluations, where human evaluators apply stringent criteria to ensure an impartial and thorough assessment of the judgments. Additionally, we employ GPT-4o as an LLM-as-a-Judge paradigm, providing an automated yet consistent scoring mechanism for large-scale experimental runs. ## 4 Empirical Results and Analysis Table 1: Main results on CorrectBench for the average of multiple LLMs. Values in () indicate the change from the baseline. Blue signifies improvements, and orange indicates declines, where darker shades reflect larger magnitudes. Further details are given in Appendix H. | Type - CoT | Method Base 83.29 ( +2.53 ) | HotpotQA( $↑$ ) 80.76 78.03 ( -1.93 ) | CS-QA( $↑$ ) 79.96 16.52 ( -2.04 ) | GPQA( $↑$ ) 18.56 91.96 ( +5.50 ) | GSM8K( $↑$ ) 86.46 60.24 ( -0.99 ) | AQUA( $↑$ ) 61.23 72.59 ( -2.53 ) | MATH( $↑$ ) 75.12 60.10 ( -12.61 ) | HumanEval( $↑$ ) 72.71 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | S1 | RCI | 79.67 ( -1.09 ) | 76.29 ( -3.67 ) | 19.98 ( +1.42 ) | 87.00 ( +0.54 ) | 67.12 ( +5.89 ) | 74.92 ( -0.20 ) | 67.46 ( -5.25 ) | | CoVe | 83.04 ( +2.28 ) | 78.54 ( -1.42 ) | 37.41 ( +18.85 ) | 92.23 ( +5.77 ) | 71.12 ( +9.89 ) | 79.30 ( +4.18 ) | 76.96 ( +4.25 ) | | | Self-Refine | 85.49 ( +4.73 ) | 81.06 ( +1.10 ) | 40.69 ( +22.13 ) | 91.74 ( +5.28 ) | 69.46 ( +8.23 ) | 81.77 ( +6.65 ) | - | | | Reflexion-v1 | 69.52 ( -11.24 ) | 63.89 ( -16.07 ) | 19.25 ( +0.69 ) | 67.64 ( -18.82 ) | 48.33 ( -12.90 ) | 65.01 ( -10.11 ) | - | | | S2 | Reflexion-v2 | 87.98 ( +7.22 ) | 82.21 ( +2.25 ) | 26.85 ( +8.29 ) | 89.87 ( +3.41 ) | 68.23 ( +7.00 ) | 81.36 ( +6.24 ) | - | | RARR | 85.47 ( +4.71 ) | 80.57 ( +0.61 ) | 36.82 ( +18.26 ) | 88.92 ( +2.46 ) | 66.81 ( +5.58 ) | 82.78 ( +7.66 ) | 77.35 ( +4.64 ) | | | RATT | 79.59 ( -1.17 ) | 80.81 ( +0.85 ) | 25.90 ( +7.34 ) | 88.08 ( +1.62 ) | 68.06 ( +6.83 ) | 80.74 ( +5.62 ) | 73.44 ( +0.73 ) | | | CRITIC | - | 81.77 ( +1.81 ) | - | 77.46 ( -9.00 ) | - | - | - | | | - | Average | 83.54 ( +2.78 ) | 80.18 ( +0.22 ) | 31.28 ( +12.72 ) | 85.04 ( -1.42 ) | 68.47 ( +7.24 ) | 80.15 ( +5.03 ) | 73.80 ( +1.09 ) | ### 4.1 Main Results Table 1 summarizes the average performance improvements attained by various self-correction methods over the ‘Base’. The results reveal that each self-correction method demonstrates performance improvements over the ‘Base’ to varying degrees, with particularly pronounced gains in more complex tasks such as GPQA and MATH. For instance, CoVe from S1 yields an improvement of +23.24% on GPQA. However, simpler tasks like GSM8K exhibit more modest gains (e.g., +5.28% for CoVe). By contrast, external correction methods S2 generally achieve higher average gains than S1. For example, Reflexion-v1 Reflexion-v1 denotes reflexion without external tools experiences declines on tasks such as HotpotQA (-11.13%) and AQUA (-12.90%). However, Reflexion-v2 Reflexion-v2 denotes reflexion with external tools increases its effectiveness, yielding improvements of +7.33% on HotpotQA and +7.00% on AQUA. We analyze that is because Reflexion Shinn2023ReflexionLA was initially designed to leverage external tools for enhanced correction, but Reflexion-v1, stripped of these tools, lacks optimization. This leaves weaker LLMs prone to getting "stuck," producing persistent incorrect outputs and lowering the average score. Finally, S3 generally lags behind these methods, likely due to smaller model sizes and narrower training objectives (Details in Section 4.5). <details> <summary>x4.png Details</summary>  ### Visual Description ## Scatter Plot: Language Model Performance Delta Across Datasets ### Overview This image is a scatter plot comparing the performance delta (Δ, in percentage) of various large language models (LLMs) across seven different benchmark datasets. Each data point represents a specific model's performance relative to a baseline (Δ=0). The plot categorizes models as Open LLMs, Closed LLMs, or Reasoning LLMs using distinct marker shapes. ### Components/Axes * **Y-Axis:** Labeled "Δ (%)". The scale ranges from -15 to 25, with major tick marks every 5 units. A horizontal dashed green line at Δ=0 is labeled "Baseline (Δ=0)". * **X-Axis:** Labeled "Dataset". It lists seven categorical datasets: `HotpotQA`, `CS-QA`, `GPQA`, `AQUA`, `GSM8K`, `MATH`, and `HumanEval`. Vertical dashed lines separate each dataset category. * **Legend (Positioned on the right side):** * **Models (by color):** * `LLaMA3.1-8B` (Teal circle) * `LLaMA3.1-70B` (Light green circle) * `Qwen2.5-7B` (Light blue circle) * `Qwen2.5-72B` (Red circle) * `Claude3.5` (Dark blue circle) * `GPT-3.5` (Orange pentagon) * `GPT-4o` (Yellow-green pentagon) * `QWQ-32B` (Pink diamond) * `DeepSeek-V3` (Purple diamond) * **Model Type (by shape):** * `Open LLM` (Circle) * `Closed LLM` (Pentagon) * `Reasoning LLM` (Diamond) * **Baseline:** `Baseline (Δ=0)` (Green dashed line) ### Detailed Analysis Performance deltas are approximate, read from the chart's grid. **1. HotpotQA:** * `LLaMA3.1-8B` (Teal, Circle): ~ -13% * `Qwen2.5-7B` (Light blue, Circle): ~ -2% * `LLaMA3.1-70B` (Light green, Circle): ~ +3% * `Qwen2.5-72B` (Red, Circle): ~ +4% * `Claude3.5` (Dark blue, Circle): ~ +1% * `GPT-3.5` (Orange, Pentagon): ~ +5% * `GPT-4o` (Yellow-green, Pentagon): ~ +3% * `QWQ-32B` (Pink, Diamond): ~ -1% * `DeepSeek-V3` (Purple, Diamond): ~ +17% **2. CS-QA:** * `LLaMA3.1-8B` (Teal, Circle): ~ -8% * `Qwen2.5-7B` (Light blue, Circle): ~ -9% * `LLaMA3.1-70B` (Light green, Circle): ~ -6% * `Qwen2.5-72B` (Red, Circle): ~ 0% * `Claude3.5` (Dark blue, Circle): ~ +7% * `GPT-3.5` (Orange, Pentagon): ~ +8% * `GPT-4o` (Yellow-green, Pentagon): ~ +6% * `QWQ-32B` (Pink, Diamond): ~ -8% * `DeepSeek-V3` (Purple, Diamond): ~ -4% **3. GPQA:** * `LLaMA3.1-8B` (Teal, Circle): ~ +9% * `Qwen2.5-7B` (Light blue, Circle): ~ +20% * `LLaMA3.1-70B` (Light green, Circle): ~ +20% * `Qwen2.5-72B` (Red, Circle): ~ +16% * `Claude3.5` (Dark blue, Circle): ~ +9% * `GPT-3.5` (Orange, Pentagon): ~ +14% * `GPT-4o` (Yellow-green, Pentagon): ~ +16% * `QWQ-32B` (Pink, Diamond): ~ +11% * `DeepSeek-V3` (Purple, Diamond): ~ +12% **4. AQUA:** * `LLaMA3.1-8B` (Teal, Circle): ~ -5% * `Qwen2.5-7B` (Light blue, Circle): ~ -2% * `LLaMA3.1-70B` (Light green, Circle): ~ +1% * `Qwen2.5-72B` (Red, Circle): ~ +3% * `Claude3.5` (Dark blue, Circle): ~ +1% * `GPT-3.5` (Orange, Pentagon): ~ +16% * `GPT-4o` (Yellow-green, Pentagon): ~ +15% * `QWQ-32B` (Pink, Diamond): ~ +1% * `DeepSeek-V3` (Purple, Diamond): ~ +5% **5. GSM8K:** * `LLaMA3.1-8B` (Teal, Circle): ~ -1% * `Qwen2.5-7B` (Light blue, Circle): ~ +9% * `LLaMA3.1-70B` (Light green, Circle): ~ +5% * `Qwen2.5-72B` (Red, Circle): ~ +9% * `Claude3.5` (Dark blue, Circle): ~ +2% * `GPT-3.5` (Orange, Pentagon): ~ +22% * `GPT-4o` (Yellow-green, Pentagon): ~ +11% * `QWQ-32B` (Pink, Diamond): ~ +10% * `DeepSeek-V3` (Purple, Diamond): ~ +4% **6. MATH:** * `LLaMA3.1-8B` (Teal, Circle): ~ +9% * `Qwen2.5-7B` (Light blue, Circle): ~ +4% * `LLaMA3.1-70B` (Light green, Circle): ~ +4% * `Qwen2.5-72B` (Red, Circle): ~ +1% * `Claude3.5` (Dark blue, Circle): ~ +2% * `GPT-3.5` (Orange, Pentagon): ~ +9% * `GPT-4o` (Yellow-green, Pentagon): ~ +10% * `QWQ-32B` (Pink, Diamond): ~ -4% * `DeepSeek-V3` (Purple, Diamond): ~ -4% **7. HumanEval:** * `LLaMA3.1-8B` (Teal, Circle): ~ -5% * `Qwen2.5-7B` (Light blue, Circle): ~ -5% * `LLaMA3.1-70B` (Light green, Circle): ~ +7% * `Qwen2.5-72B` (Red, Circle): ~ -10% * `Claude3.5` (Dark blue, Circle): ~ -1% * `GPT-3.5` (Orange, Pentagon): ~ +1% * `GPT-4o` (Yellow-green, Pentagon): ~ +4% * `QWQ-32B` (Pink, Diamond): ~ +10% * `DeepSeek-V3` (Purple, Diamond): ~ -4% ### Key Observations 1. **High Variance:** Performance deltas vary dramatically across both models and datasets. No single model dominates all benchmarks. 2. **Dataset Difficulty:** Models show the widest spread of performance on `GPQA` and `GSM8K`, suggesting these datasets may differentiate model capabilities more sharply. 3. **Top Performers:** `GPT-3.5` (Orange pentagon) achieves the single highest delta on the chart (~+22% on GSM8K). `Qwen2.5-7B` and `LLaMA3.1-70B` also show strong peaks (~+20% on GPQA). 4. **Notable Underperformance:** `LLaMA3.1-8B` (Teal circle) has the lowest delta (~-13% on HotpotQA). `Qwen2.5-72B` (Red circle) shows a significant drop on HumanEval (~-10%). 5. **Reasoning LLMs (Diamonds):** `DeepSeek-V3` (Purple) shows high variance, with a strong positive outlier on HotpotQA (~+17%) but negative performance on MATH and HumanEval. `QWQ-32B` (Pink) is generally closer to the baseline. 6. **Open vs. Closed:** Closed LLMs (Pentagons: GPT-3.5, GPT-4o) tend to cluster in the upper half of the chart for most datasets, but are not universally superior. ### Interpretation This chart visualizes the **non-uniform progress and specialization** in the current LLM landscape. The data suggests: * **Benchmark Sensitivity:** A model's "capability" is not a single number but a profile across tasks. Strengths in mathematical reasoning (GSM8K, MATH) do not guarantee strength in question answering (HotpotQA, CS-QA) or code generation (HumanEval). * **The "No Free Lunch" Theorem in AI:** The absence of a model that excels in all categories indicates that architectural choices, training data, and optimization targets create trade-offs. For example, a model fine-tuned for math may see regressions on other tasks. * **The Baseline is Key:** The Δ=0 baseline is critical for interpretation. It likely represents the performance of a reference model (e.g., an earlier version or a standard baseline). Points above the line indicate improvement over this reference; points below indicate regression. The chart therefore measures **relative advancement**, not absolute accuracy. * **Strategic Implications:** For a practitioner, this chart argues for **model selection based on the specific task**. Choosing a model requires consulting its performance profile on benchmarks analogous to the intended application, rather than relying on aggregate scores or reputation. The high variance, especially among open models, highlights the rapid and divergent evolution in this field. </details> Figure 4: Average performance improvements achieved by S1 across multiple LLMs. <details> <summary>x5.png Details</summary>  ### Visual Description ## Scatter Plot: LLM Performance Delta Across Datasets ### Overview The image is a scatter plot comparing the performance delta (Δ, in percentage) of various Large Language Models (LLMs) across seven different benchmark datasets. The plot uses distinct symbols and colors to represent different models and model types (Open, Closed, Reasoning). A horizontal dashed green line at Δ=0 represents the baseline performance. ### Components/Axes * **Y-Axis:** Label is "Δ (%)". Scale ranges from -15 to 25, with major ticks at intervals of 5. * **X-Axis:** Label is "Dataset". Seven categorical datasets are listed, separated by vertical dashed lines: 1. HotpotQA 2. CS-QA 3. GPQA 4. AQUA 5. GSM8K 6. MATH 7. HumanEval * **Legend (Position: Top-Right):** Contains 11 entries, defining model names and their corresponding symbols/colors, plus a baseline indicator. * **Models & Symbols:** * LLaMA3.1-8B: Light green circle * LLaMA3.1-70B: Yellow circle * Qwen2.5-7B: Light blue circle * Qwen2.5-72B: Red circle * Claude3.5: Blue star * GPT-3.5: Orange pentagon * GPT-4o: Yellow pentagon * QWQ-32B: Pink diamond * DeepSeek-V3: Purple diamond * **Model Type Indicators:** * Open LLM: Circle symbol * Closed LLM: Pentagon symbol * Reasoning LLM: Diamond symbol * **Baseline:** Green dashed line labeled "Baseline (Δ=0)" ### Detailed Analysis Performance deltas are plotted for each model within each dataset column. Values are approximate based on visual positioning. **1. HotpotQA:** * LLaMA3.1-8B: ~+7% * LLaMA3.1-70B: ~+6% * Qwen2.5-7B: ~+3% * Qwen2.5-72B: ~+1% * Claude3.5: ~0% * GPT-3.5: ~+2% * QWQ-32B: ~+17% (Significant outlier, highest in this group) * DeepSeek-V3: ~-3% **2. CS-QA:** * LLaMA3.1-8B: ~0% * LLaMA3.1-70B: ~+1% * Qwen2.5-7B: ~+2% * Qwen2.5-72B: ~+3% * Claude3.5: ~+1% * GPT-3.5: ~+3% * QWQ-32B: ~-2% * DeepSeek-V3: ~+2% * *Trend:* Most models cluster near the baseline (0-3% range). **3. GPQA:** * LLaMA3.1-8B: ~+14% * LLaMA3.1-70B: ~+13% * Qwen2.5-7B: ~+18% (Highest in this group) * Qwen2.5-72B: ~+17% * Claude3.5: ~+15% * GPT-3.5: ~+5% * GPT-4o: ~+4% * QWQ-32B: ~+13% * DeepSeek-V3: ~+3% * *Trend:* Most models show a strong positive delta, with several above +13%. **4. AQUA:** * LLaMA3.1-8B: ~-3% * LLaMA3.1-70B: ~-5% * Qwen2.5-7B: ~0% * Qwen2.5-72B: ~-1% * Claude3.5: ~0% * GPT-3.5: ~+11% * GPT-4o: ~+5% * QWQ-32B: ~+18% (Highest in this group) * DeepSeek-V3: ~0% * *Trend:* Mixed performance. Open-source models (circles) are at or below baseline, while some closed/reasoning models show high positive deltas. **5. GSM8K:** * LLaMA3.1-8B: ~+3% * LLaMA3.1-70B: ~+2% * Qwen2.5-7B: ~+8% * Qwen2.5-72B: ~+2% * Claude3.5: ~+4% * GPT-3.5: ~+13% * GPT-4o: ~+13% * QWQ-32B: ~+13% * DeepSeek-V3: ~-1% * *Trend:* Generally positive deltas, with GPT-3.5, GPT-4o, and QWQ-32B clustered at the high end (~+13%). **6. MATH:** * LLaMA3.1-8B: ~+7% * LLaMA3.1-70B: ~+8% * Qwen2.5-7B: ~+9% * Qwen2.5-72B: ~+11% * Claude3.5: ~+8% * GPT-3.5: ~+5% * GPT-4o: ~0% * QWQ-32B: ~0% * DeepSeek-V3: ~0% * *Trend:* Open-source models (circles) show a clear upward trend in performance delta, while closed/reasoning models (pentagons, diamonds) are at or near baseline. **7. HumanEval:** * LLaMA3.1-8B: ~-5% * LLaMA3.1-70B: ~+7% * Qwen2.5-7B: ~+3% * Qwen2.5-72B: ~0% * Claude3.5: ~0% * GPT-3.5: ~-1% * GPT-4o: ~+9% * QWQ-32B: ~+6% * DeepSeek-V3: ~-3% * *Trend:* Highly variable performance with no clear grouping by model type. ### Key Observations 1. **Dataset Sensitivity:** Model performance delta is highly dependent on the dataset. No single model dominates across all benchmarks. 2. **GPQA & AQUA Outliers:** The QWQ-32B (Reasoning LLM) shows exceptionally high positive deltas on GPQA (~+13%) and AQUA (~+18%), making it a standout on these specific tasks. 3. **MATH Trend:** A clear pattern emerges on the MATH dataset where open-source models (circles) consistently show positive performance deltas, while closed and reasoning models cluster near zero. 4. **Baseline Clustering:** On CS-QA, most models, regardless of type, cluster within a narrow band near the baseline (Δ=0 to +3%). 5. **Symbol Consistency:** The legend's model type indicators (Open/Closed/Reasoning) are consistently applied to the symbols in the plot, allowing for categorical analysis. ### Interpretation This chart visualizes the **relative performance change (Δ)** of various LLMs compared to an unspecified baseline across diverse reasoning and knowledge benchmarks. The data suggests: * **Specialization Over Generalization:** The significant variance in Δ across datasets indicates that LLM capabilities are not uniform. Models excel in specific domains (e.g., QWQ-32B on AQUA/GPQA, open-source models on MATH) rather than demonstrating blanket superiority. * **Model Type Insights:** The grouping by symbol type reveals potential trends. For instance, on the MATH dataset, open-source models (circles) appear to have a performance advantage over closed models (pentagons) in this specific comparison. Conversely, on AQUA, reasoning models (diamonds) show the highest deltas. * **Benchmark Diversity:** The choice of benchmark critically affects the perceived performance of a model. A model like Qwen2.5-72B shows a high positive delta on GPQA (~+17%) but a negative delta on HumanEval (~0% or slightly below), highlighting the importance of multi-faceted evaluation. * **Investigative Lead:** The most notable anomaly is the performance of QWQ-32B on AQUA and GPQA. This warrants further investigation into the model's architecture or training data that might give it a specific advantage on these particular types of questions (likely scientific or complex reasoning QA). The chart effectively argues against using a single benchmark to rank LLMs. </details> Figure 5: Average performance improvements achieved by S2 across multiple LLMs. ### 4.2 Results of Intrinsic Correction Figure 4 illustrates the mean performance gains realized by all S1 methods across nine LLMs and multiple datasets, where y=0 denotes the baseline. For detailed results of each method, refer to Appendix E.1. Although S1 improves accuracy overall, the degree of improvement varies across instruction-based and reasoning LLMs. 1) Instruction-based LLMs. Closed-source LLMs exhibit uneven performance gains. For instance, LLaMA3.1-8B-Instruct shows significant declines on AQUA and HumanEval, whereas Qwen2.5-7B-Instruct demonstrates modest gains. These discrepancies likely stem from smaller parameter sizes and weaker instruction-following capabilities. In contrast, open-source LLMs offer more consistent and stable performance improvements. Notably, GPT-4o shows substantial gains on GPQA, whereas Claude3.5 achieves similar enhancements, highlighting the robust instruction-following adaptability of open-source architectures. 2) Reasoning LLMs. DeepSeek-V3’s performance remains close to the baseline across most tasks. While it demonstrates slight improvements on datasets such as CS-QA, GPQA, and MATH, it exhibits marginal declines on others. To further investigate this phenomenon, we conducted additional experiments (see Section 4.8), revealing that DeepSeek-V3’s built-in correction mechanism delivers a strong baseline performance. This high initial performance likely limits the impact of other correction methods. Conversely, QWQ attains considerable improvements on most datasets except CS-QA and MATH, likely reflecting the constraints posed by its smaller parameter size. ### 4.3 Results of External Correction Figure 5 illustrates the average performance improvements achieved by each LLM utilizing external correction methods (S2) across various datasets. The trends observed align closely with those depicted in Figure 4, indicating a consistent enhancement in overall performance. Notably, external correction methods demonstrate greater stability, which can be attributed to their reliance on authoritative external resources. By referencing these resources, S2 effectively mitigates the occurrence of incorrect responses. However, this reliance on external inputs may also limit the LLM’s capacity for divergent or creative reasoning, resulting in steadier but less flexible performance compared to intrinsic correction methods. For comprehensive results for each method, please refer to Appendix E.2. [RQ1] To what extent can LLMs achieve accurate results by leveraging intrinsic (S1) and external (S2) self-correction methods without requiring further intervention? Conclusion: Both S1 and S2 enable significant performance gains, particularly for complex tasks requiring multi-step reasoning or domain-specific knowledge. By iteratively refining responses, these methods effectively correct themselves even without additional fine-tuning. <details> <summary>x6.png Details</summary>  ### Visual Description ## [Multi-Panel Bar Chart]: AI Model Performance Comparison on GPQA and MATH Tasks ### Overview The image displays a set of four stacked bar charts comparing the performance of various large language models (LLMs) on two different tasks (GPQA and MATH) using two different evaluation metrics (F1 and F2). The performance is broken down into three components: Base, Intrinsic Correction, and External Correction. The charts are arranged in a 2x2 grid, labeled a) through d). ### Components/Axes * **Overall Structure:** Four subplots arranged horizontally. * **Subplot Titles:** * a) GPQA (F1) * b) MATH (F1) * c) GPQA (F2) * d) MATH (F2) * **Y-Axis (All Subplots):** Labeled "Percentage (%)". The scale runs from 0 to 100 in increments of 20. * **X-Axis (All Subplots):** Lists the same set of 9 AI models in the same order. The labels are rotated approximately 45 degrees for readability. The models are: 1. LLaMA3.1-8B 2. LLaMA3.1-70B 3. Qwen2.5-7B 4. Qwen2.5-72B 5. Claude3.5 6. GPT-3.5 7. GPT-4o 8. QWQ-32B 9. DeepSeek-V3 * **Legend:** Positioned at the bottom center of the entire figure. It defines the three stacked components of each bar: * **Teal/Green:** Base * **Salmon/Red:** Intrinsic Correction * **Blue:** External Correction ### Detailed Analysis The following analysis provides approximate percentage values for each component of each bar. Values are estimated based on visual alignment with the y-axis grid lines. **a) GPQA (F1)** * **Trend:** Performance varies significantly. Claude3.5 and GPT-4o show the highest total scores. QWQ-32B has the lowest total score. * **Data Points (Approximate):** * LLaMA3.1-8B: Base ~15%, Intrinsic ~15%, External ~0%. Total ~30%. * LLaMA3.1-70B: Base ~15%, Intrinsic ~10%, External ~10%. Total ~35%. * Qwen2.5-7B: Base ~10%, Intrinsic ~5%, External ~5%. Total ~20%. * Qwen2.5-72B: Base ~10%, Intrinsic ~5%, External ~25%. Total ~40%. * Claude3.5: Base ~30%, Intrinsic ~20%, External ~5%. Total ~55%. * GPT-3.5: Base ~15%, Intrinsic ~10%, External ~10%. Total ~35%. * GPT-4o: Base ~15%, Intrinsic ~10%, External ~15%. Total ~40%. * QWQ-32B: Base ~10%, Intrinsic ~5%, External ~0%. Total ~15%. * DeepSeek-V3: Base ~15%, Intrinsic ~10%, External ~10%. Total ~35%. **b) MATH (F1)** * **Trend:** Overall scores are higher than in GPQA (F1). Claude3.5 and GPT-4o again lead. The "External Correction" component is more prominent for many models. * **Data Points (Approximate):** * LLaMA3.1-8B: Base ~70%, Intrinsic ~5%, External ~5%. Total ~80%. * LLaMA3.1-70B: Base ~70%, Intrinsic ~10%, External ~5%. Total ~85%. * Qwen2.5-7B: Base ~65%, Intrinsic ~10%, External ~5%. Total ~80%. * Qwen2.5-72B: Base ~60%, Intrinsic ~5%, External ~5%. Total ~70%. * Claude3.5: Base ~75%, Intrinsic ~10%, External ~10%. Total ~95%. * GPT-3.5: Base ~65%, Intrinsic ~5%, External ~10%. Total ~80%. * GPT-4o: Base ~65%, Intrinsic ~5%, External ~20%. Total ~90%. * QWQ-32B: Base ~65%, Intrinsic ~10%, External ~5%. Total ~80%. * DeepSeek-V3: Base ~75%, Intrinsic ~5%, External ~10%. Total ~90%. **c) GPQA (F2)** * **Trend:** Scores are generally lower than in GPQA (F1). The "External Correction" component is very significant for Qwen2.5-72B and GPT-4o. * **Data Points (Approximate):** * LLaMA3.1-8B: Base ~10%, Intrinsic ~10%, External ~5%. Total ~25%. * LLaMA3.1-70B: Base ~10%, Intrinsic ~5%, External ~15%. Total ~30%. * Qwen2.5-7B: Base ~10%, Intrinsic ~5%, External ~10%. Total ~25%. * Qwen2.5-72B: Base ~10%, Intrinsic ~5%, External ~20%. Total ~35%. * Claude3.5: Base ~20%, Intrinsic ~25%, External ~5%. Total ~50%. * GPT-3.5: Base ~15%, Intrinsic ~5%, External ~10%. Total ~30%. * GPT-4o: Base ~15%, Intrinsic ~5%, External ~20%. Total ~40%. * QWQ-32B: Base ~10%, Intrinsic ~5%, External ~0%. Total ~15%. * DeepSeek-V3: Base ~15%, Intrinsic ~20%, External ~10%. Total ~45%. **d) MATH (F2)** * **Trend:** Scores are high, similar to MATH (F1). Claude3.5 and GPT-4o are top performers. The "External Correction" component is substantial for several models. * **Data Points (Approximate):** * LLaMA3.1-8B: Base ~65%, Intrinsic ~5%, External ~5%. Total ~75%. * LLaMA3.1-70B: Base ~65%, Intrinsic ~10%, External ~10%. Total ~85%. * Qwen2.5-7B: Base ~65%, Intrinsic ~5%, External ~10%. Total ~80%. * Qwen2.5-72B: Base ~60%, Intrinsic ~5%, External ~15%. Total ~80%. * Claude3.5: Base ~70%, Intrinsic ~10%, External ~10%. Total ~90%. * GPT-3.5: Base ~60%, Intrinsic ~5%, External ~15%. Total ~80%. * GPT-4o: Base ~65%, Intrinsic ~5%, External ~15%. Total ~85%. * QWQ-32B: Base ~65%, Intrinsic ~5%, External ~10%. Total ~80%. * DeepSeek-V3: Base ~70%, Intrinsic ~5%, External ~15%. Total ~90%. ### Key Observations 1. **Model Hierarchy:** Claude3.5 and GPT-4o consistently achieve the highest total percentages across all four metrics. QWQ-32B consistently shows the lowest or among the lowest total scores. 2. **Task Difficulty:** Models score significantly higher on the MATH task (subplots b and d) than on the GPQA task (subplots a and c), suggesting GPQA is a more challenging benchmark for these models. 3. **Correction Impact:** The "External Correction" (blue) component is often the largest contributor to the total score for mid-performing models, especially on the GPQA task. For top models like Claude3.5, the "Base" (teal) component is dominant. 4. **Metric Difference (F1 vs. F2):** For the same task, the F2 metric generally yields lower scores than the F1 metric, particularly for the GPQA task. This indicates the F2 evaluation is stricter. ### Interpretation This chart provides a comparative analysis of LLM capabilities on reasoning-heavy tasks (GPQA, likely a science QA benchmark, and MATH). The breakdown into Base, Intrinsic, and External Correction suggests an evaluation framework that measures not just raw model output ("Base"), but also the model's ability to self-correct ("Intrinsic") and benefit from external feedback or tools ("External"). The data demonstrates that model scale (e.g., LLaMA3.1-70B vs. 8B) and architecture (Claude, GPT, Qwen, etc.) lead to significant performance differences. The consistent lead of Claude3.5 and GPT-4o indicates their superior reasoning capabilities on these benchmarks. The high contribution of "External Correction" for many models implies that their standalone reasoning is less reliable, but they can effectively utilize additional mechanisms to improve accuracy. The stark difference between GPQA and MATH scores highlights that model performance is highly task-dependent; excelling in mathematical reasoning does not guarantee equivalent performance in scientific question answering. </details> Figure 6: Comparison of different LLMs with mixture methods. ### 4.4 Results of Mixture Method We further explored whether mixing multiple self-correction methods results in additive or synergistic performance improvements. Specifically, the responses generated by the baseline method (Base) are utilized as auxiliary prompts for an intrinsic method (S1), an external method (S2), or a mixture of both (S1 + S2). As shown in Figure 6, we evaluated two representative configurations (e.g., F1: ‘Base to S1 to S2 ’ and F2: ‘Base to S2 to S1 ’) on two benchmark tasks: GPQA and MATH. The results revealed two key findings. Firstly, applying one or more correction methods consistently improves model performance to varying degrees. Secondly, S2 generally yields larger performance gains compared to S1. Notably, complex reasoning tasks, such as GPQA, benefit the most from these mixed methods. However, these mixtures often introduce additional computational overhead. To address the associated efficiency and accuracy trade-offs, we conducted a complementary analysis of response times under different correction methods and model configurations, as detailed in Section 4.7. ### 4.5 Results of Fine-tuned Correction Table 2: Performance of fine-tuned methods on selected datasets. | Fine-tuned LLM - LLaMA2-7B-hf | Method Base DCoT | CS-QA( $↑$ ) 31.40 29.65(-1.75) | GSM8K( $↑$ ) 56.75 41.20(-15.55) | | --- | --- | --- | --- | | Gemma-7B-it | SCORE | 43.26(+11.86) | 75.30(+18.55) | | LLaMA2-13B-chat | SCORE | 41.45(+10.05) | 72.10(+15.35) | | Qwen2.5-Math-7B-Instruct | SuperCorrect | 46.25(+14.85) | 84.30(+27.55) | | MATH( $↑$ ) | HumanEval( $↑$ ) | | | | - | Base | 41.71 | 26.25 | | Qwen2.5-Math-7B-Instruct | SuperCorrect | 70.16(+28.45) | 39.30(+13.05) | Table 2 summarizes the performance of fine-tuned correction methods (S3), revealing two main observations. First, S3 (e.g., DCoT) often exhibits inconsistent outcomes across diverse tasks. This variability stems from their reliance on fine-tuning with narrowly focused datasets, which restricts their broader applicability. Second, domain-specific fine-tuning proves especially promising for models tailored to specialized tasks. For example, SuperCorrect, fine-tuned on Qwen2.5-Math-7B-Instruct, demonstrates marked improvements in mathematical reasoning (e.g., on GSM8K and MATH), outperforming other methods by a wide margin. This underscores the effectiveness of leveraging task-aligned models, particularly when fine-tuning objectives closely align with the target domain requirements. Table 3: Performance of correction rate and misjudgment rate. | Cove MR RARR | CR 8.1 CR | 31.6 8.0 30.7 | 36.0 6.7 49.3 | 52.1 7.5 51.3 | 40.8 47.1 | | --- | --- | --- | --- | --- | --- | | MR | 5.5 | 4.3 | 4.5 | 4.5 | | ### 4.6 Results of Correction and Misjudgment In order to further assess the correction ability, we divide the responses of different models to questions in the three most challenging tasks (GPQA, AQUA, and HotpotQA) into error-based dataset and correction-based dataset, corresponding to the wrong and correct question-response pairs, respectively. We select CoVe and RARR from S1 and S2 respectively to evaluate on Claude 3.5-Sonnet. We hereby define two new metrics: Correction Rate (CR) indicates the proportion of incorrect responses that are successfully corrected, and Misjudgment Rate (MR) refers to the proportion of correct responses that are misjudged to be corrected wrongly. The results shown in Table 3 reflects that both methods achieve high CRs and low MRs, which shows that self-correction methods can effectively correct the wrong examples with less misjudgment. <details> <summary>x7.png Details</summary>  ### Visual Description ## Bar Chart: Performance Comparison of Self-Correction Methods Across Three Large Language Models ### Overview The image is a grouped bar chart comparing the execution time (in seconds) of eight different self-correction methods when applied to three distinct large language models (LLMs). The chart visually demonstrates the computational cost associated with each method for each model. ### Components/Axes * **Chart Type:** Grouped Bar Chart. * **X-Axis (Horizontal):** Labeled **"Self-correction Methods"**. It lists eight categorical methods: 1. Base 2. CoT (Chain-of-Thought) 3. RCI (Recursive Critique and Improve) 4. CoVe (Chain-of-Verification) 5. SelfRefine 6. Reflexion 7. RARR (Rethink and Rewrite) 8. RATT (Recursive Abstractive Thought Tree) * **Y-Axis (Vertical):** Labeled **"Time (s)"**. It represents execution time in seconds, with a linear scale from 0 to 400, marked at intervals of 100 (0, 100, 200, 300, 400). * **Legend:** Located in the **top-left corner** of the chart area. It defines the three data series by color: * **Teal/Green:** LLaMA3.1-70B * **Salmon/Orange:** GPT-4o * **Blue:** DeepSeek-V3 ### Detailed Analysis The following values are approximate, derived from visual inspection of the bar heights against the y-axis scale. **Trend Verification:** For all three models, the general trend is that time increases significantly for the more complex, iterative methods (CoVe, SelfRefine, Reflexion, RATT) compared to the baseline and simpler prompting methods (Base, CoT, RCI, RARR). The RATT method is a clear outlier, consuming the most time by a large margin for every model. **Data Points by Method (Approximate Time in Seconds):** 1. **Base:** * LLaMA3.1-70B: ~10s * GPT-4o: ~5s * DeepSeek-V3: ~20s 2. **CoT:** * LLaMA3.1-70B: ~15s * GPT-4o: ~10s * DeepSeek-V3: ~25s 3. **RCI:** * LLaMA3.1-70B: ~20s * GPT-4o: ~15s * DeepSeek-V3: ~60s 4. **CoVe:** * LLaMA3.1-70B: ~30s * GPT-4o: ~25s * DeepSeek-V3: ~150s 5. **SelfRefine:** * LLaMA3.1-70B: ~40s * GPT-4o: ~50s * DeepSeek-V3: ~160s 6. **Reflexion:** * LLaMA3.1-70B: ~70s * GPT-4o: ~50s * DeepSeek-V3: ~110s 7. **RARR:** * LLaMA3.1-70B: ~10s * GPT-4o: ~10s * DeepSeek-V3: ~60s 8. **RATT:** * LLaMA3.1-70B: ~430s (exceeds the 400s axis mark) * GPT-4o: ~290s * DeepSeek-V3: ~420s (exceeds the 400s axis mark) ### Key Observations 1. **Dominant Outlier:** The **RATT** method is dramatically more time-intensive than all others for every model, with times roughly 3-10x higher than the next most expensive method (SelfRefine/CoVe for DeepSeek-V3). 2. **Model-Specific Patterns:** * **DeepSeek-V3 (Blue)** consistently shows the highest or near-highest execution time for most methods, particularly for CoVe, SelfRefine, and RARR. * **GPT-4o (Salmon)** generally exhibits the lowest execution times across the board, with the notable exception of the RATT method. * **LLaMA3.1-70B (Teal)** often falls between the other two models, but it records the single highest time for RATT. 3. **Method Clustering:** Methods can be loosely grouped by computational cost: * **Low Cost:** Base, CoT, RARR (all models < ~60s). * **Medium Cost:** RCI, Reflexion (times vary by model, up to ~110s). * **High Cost:** CoVe, SelfRefine (significantly higher for DeepSeek-V3). * **Very High Cost:** RATT (extremely high for all models). ### Interpretation This chart provides a clear quantitative comparison of the **computational overhead** introduced by various self-correction techniques in LLMs. The data suggests a fundamental trade-off: more sophisticated, recursive, or multi-step self-correction methods (like RATT, CoVe, SelfRefine) incur a substantial time penalty. The significant variation between models (e.g., DeepSeek-V3's high cost for CoVe vs. GPT-4o's relative efficiency) implies differences in model architecture, inference optimization, or the specific implementation of these methods for each model. The extreme cost of RATT indicates it is likely a highly complex, tree-based search or deliberation process. For a practitioner, this information is critical for selecting a self-correction method. If latency is a primary constraint, methods like CoT or RARR are preferable. If maximum accuracy is the goal and computational resources are abundant, a method like RATT might be justified, but its cost must be weighed against potential gains. The chart effectively visualizes that "better" self-correction is not free—it is paid for in time. </details> Figure 7: Average response times for LLaMA3.1-70B, GPT-4o, and DeepSeek-V3. Intrinsic (S1) and external (S2) methods generally increase inference duration relative to Base and CoT. ### 4.7 Results of Response Time Figure 7 compares the average response times across representative models (e.g., LLaMA3.1-70B and GPT-4o from M1, DeepSeek-V3 from M2) under various correction methods. In general, both intrinsic and external methods extend inference times relative to baseline approaches (Base and CoT), largely due to increased reasoning complexity or reliance on external services (especially for RATT). Moreover, reasoning LLMs, such as DeepSeek-V3, exhibit notably longer execution times than instruction-based models, likely attributable to their built-in correction mechanisms. In contrast, the baseline CoT method achieves notably shorter response times while maintaining a reasonable accuracy (combined with Table 1). This observation underscores that more complex correction strategies do not always yield superior outcomes, highlighting the critical trade-off between model accuracy and computational efficiency. As shown in Table 4, RARR offers a balanced trade-off between efficiency and accuracy, with only 533 tokens and 2 API calls. Reflexion-v2 and RATT achieve the highest accuracies, reflecting the benefit of external retrieval or code execution, while their overhead remains manageable (below 15% additional search tokens). Bootstrap-based confidence intervals for these results are reported in Appendix B.3. [RQ2] How does mixing multiple self-correction methods influence model accuracy and robustness, and what are the associated computational trade-offs? Conclusion: Mixing self-correction methods typically results in accuracy improvements, though at the cost of increased computational overhead. Such mixtures are particularly beneficial for high-precision tasks where the trade-off of more runtime is justifiable. Table 4: Comprehensive resource cost analysis on the MATH dataset (150 samples). Values in green indicate the best trade-off between cost and accuracy, while red highlights the least efficient results. “Efficiency Rank” represents the ratio of Accuracy / (Token Count × API Calls). | Base CoT CoVe | - - S1 | 791 1804 2019 | 1.0 1.0 1.0 | 0 (0%) 0 (0%) 0 (0%) | 68.5 69.5 75.0 | 0.0866 0.0385 0.0371 | | --- | --- | --- | --- | --- | --- | --- | | RCI | S1 | 1780 | 1.2 | 0 (0%) | 70.2 | 0.0328 | | Reflexion-v1 | S1 | 1460 | 3.5 | 0 (0%) | 72.8 | 0.0143 | | Reflexion-v2 | S2 | 1712 | 4.0 | 154 (8.25%) | 74.5 | 0.0109 | | RARR | S2 | 533 | 2.0 | 89 (14.31%) | 76.3 | 0.0716 | | RATT | S2 | 2185 | 3.0 | 162 (6.9%) | 78.7 | 0.0120 | ### 4.8 Results of Reasoning LLMs Table 5: Comparison of baseline performance among instruction-based and reasoning LLMs. Per-column maxima are bolded; per-column minima are underlined. Per-row maxima are highlighted with blue; per-row minima are highlighted with orange. | Open-source LLaMA3.1-70B-Instruct Qwen2.5-7B-Instruct | LLaMA3.1-8B-Instruct 81.28 74.05 | 75.80 81.88 74.75 | 76.16 15.62 7.53 | 9.74 90.63 90.23 | 81.55 62.65 47.50 | 53.88 78.21 74.28 | 74.37 62.18 79.11 | 73.44 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Qwen2.5-72B-Instruct | 83.63 | 81.92 | 9.85 | 91.11 | 57.58 | 66.91 | 86.13 | | | Closed-source | Claude3.5-Sonnet | 88.29 | 80.25 | 32.34 | 95.81 | 81.26 | 83.51 | 84.69 | | GPT-3.5 | 82.94 | 77.92 | 27.29 | 79.14 | 55.15 | 70.44 | 80.29 | | | GPT-4o | 89.16 | 80.65 | 22.49 | 91.15 | 65.82 | 69.54 | 77.04 | | | Reasoning | QWQ-32B | 62.43 | 82.78 | 10.85 | 63.41 | 52.42 | 73.78 | 19.86 | | DeepSeek-V3 | 89.29 | 83.35 | 31.35 | 95.12 | 74.79 | 85.02 | 91.67 | | | DeepSeek-R1 | 88.92 | 79.93 | 41.15 | 92.63 | 80.23 | 84.21 | 89.06 | | | o3-mini | 81.24 | 74.28 | 27.17 | 92.45 | 78.26 | 67.97 | 85.75 | | To further investigate why DeepSeek-V3 shows limited improvement from self-correction, we compare instruction-based LLMs and reasoning LLMs on the Base alone. Table 5 shows that DeepSeek-V3 consistently achieves top-2 or even top-1 performance across all datasets. Combined with the results of Section 4.2, we find that this may be because reasoning LLMs already incorporate robust intrinsic correction mechanisms, limiting additional gains from external correction steps. In particular, DeepSeek-V3 integrates advanced reflection modules and comprehensive error-detection routines distilled from its earlier R1 series, enabling thorough multi-step reasoning at the outset. This high baseline effectively reduces the scope for further improvement through additional self-correction. Consequently, attempts to augment DeepSeek-V3 with further self-correction methods produce minimal net gains while incurring additional computational overhead. [RQ3] For reasoning LLMs with built-in correction mechanisms, to what extent can the above self-correction methods provide additional benefits? Conclusion: Reasoning LLMs (e.g., DeepSeek-V3) already embed sophisticated error-detection and correction processes. As a result, additional self-correction methods confer only marginal gains and may increase computational overhead, highlighting a performance ceiling in highly reasoning LLMs. ### 4.9 Failure Mode Taxonomy and Case Analysis To better understand why different self-correction strategies succeed or fail, we conducted a supplemental failure-mode analysis on the GPQA (250 samples) and MATH (500 samples) datasets. Six major categories of failure were identified, alongside a residual “Other” category, as summarized in Table 6. Logical Oversight (32.9%) and Factual Inaccuracy (22.0%) dominate, implying that intrinsic corrections (S1) are suitable for reasoning-related errors, while external corrections (S2) excel at factual validation. These findings motivate our adaptive correction controller (Sec. 6), which dynamically selects correction strategies based on detected failure types. Table 6: Error taxonomy of LLM self-correction failures across GPQA and MATH. Logical and factual errors dominate, suggesting distinct correction strategies (S1 vs. S2). | Logical Oversight Factual Inaccuracy Over-Reliance on Tools | 32.9 22.0 14.6 | Reasoning step errors (e.g., misapplied formula) Outdated or incorrect retrieved evidence Excessive external API calls causing inefficiency | S1: CoT, RCI S2: RARR, RATT S2 (bounded) | | --- | --- | --- | --- | | Ambiguous Output | 14.2 | Incomplete or vague final answer | S1 refinement | | Contextual Misunderstanding | 10.8 | Misinterpreted question or missing context | S1+S2 hybrid | | Computational Error | 3.5 | Faulty code execution or symbolic computation | S2 verification | | Other | 2.0 | Miscellaneous or formatting issues | - | ## 5 Related Work Self-Correction Methods. With the continuous development of self-correction techniques wei2025unsupervised ; xu2024llava ; fan2025missing ; li2025vision ; wei2025advancing , researchers have proposed various approaches to enhance the performance of large language models. Intrinsic methods, such as CoVe Dhuliawala2023ChainofVerificationRH and RCI Kim2023LanguageMC , improve the precision and consistency of generated content through self-supervised mechanisms within the model. At the same time, extrinsic methods, such as CRITIC Gou2023CRITICLL , RATT Zhang2024RATTAT , and RARR Gao2022RARRRA , rely on an external tool to evaluate and provide feedback on the generated outputs, guiding the model towards optimization. Fine-tuned methods, such as DCoT Puerto2024FineTuningWD , Supercorrect Yang2024SuperCorrectSA , and SCORE Zhang2024SmallLM , further enhance the performance of the model by fine-tuning it for specific tasks, enabling more accurate and efficient handling of complex tasks. The continuous evolution of these methods provides diverse options and techniques for self-correction. Detailed discussions on the related word are provided in Appendix C. Correction Benchmarks. Benchmarking the LLMs’ self-correction ability tie2025mmmr ; duan2024gtbench ; zheng2024thinking ; duan2024reta ; cao2025survey ; zhou2024comprehensive ; shi2024optimization ; liu2025agentic has prompted the development of specialized benchmarks for different tasks. For instance, CriticBench Lin2024CriticBenchBL evaluates critique ability using discrimination results, but it struggles with task-specific fine-grained metrics and reliance on costly human annotations or potentially biased GPT-4 outputs. In the realm of vision-language models, VISCO Wu2024VISCOBF focuses on self-correction in multimodal tasks, while Beyond Correctness Zheng2024BeyondCB specifically targets self-correction in large models for code generation. Our CorrectBench focused on striking a trade-off between reasoning capability and efficiency, proposing more generalized and nuanced evaluation methods for complex reasoning tasks. ## 6 Future Improvements in Self-Correction Looking ahead, several promising directions can further enhance the robustness and adaptability of self-correction in large language models. (1) Dynamic Adjustment: reinforcement learning or meta-controller mechanisms could dynamically select among correction strategies (S1–S3) based on task complexity or confidence levels, reducing redundant computation through early stopping. (2) Task-Specific Optimization: domain-oriented fine-tuning, such as the 10.2% improvement achieved by SuperCorrect on MATH, suggests the value of adaptive pipelines that align with domain reasoning depth and structure. (3) Human-in-the-Loop Integration: in sensitive fields like medicine or law, coupling automated correction with limited expert feedback could improve factual reliability and ensure accountable model behavior. (4) Meta-Controller Framework: developing a lightweight controller to detect and correct intermediate reasoning inconsistencies may help refine the chain-of-thought process and prevent logical drift. Further analysis and discussions of these future directions are provided in Appendix D. ## 7 Conclusion This paper presents CorrectBench, a comprehensive and extensible benchmark for evaluating the self-correction capabilities of large language models (LLMs) across diverse reasoning-intensive tasks, including commonsense inference, mathematical problem-solving, and code generation. Through systematic evaluation, we demonstrate that modern LLMs are increasingly capable of genuine self-correction, with reasoning-oriented models such as DeepSeek-R1 achieving substantial baseline accuracy and showing consistent improvement through iterative refinement. CorrectBench not only reveals the effectiveness of various correction paradigms (S1, S2, S3) but also exposes critical limitations—such as diminishing returns in deeper correction chains and resource inefficiencies in web-augmented methods. These insights emphasize the necessity for adaptive, cost-aware correction mechanisms that balance efficiency and reasoning depth. Overall, this study provides a unified foundation for understanding and benchmarking LLM self-correction. We hope this work serves as a stepping stone toward more trustworthy and self-improving language models. ## Acknowledgments This work is supported by National Natural Science Foundation of China (NSFC) under grant No. 62476107. ## References - [1] Alexandre Agossah, Frédérique Krupa, Matthieu Perreira da Silva, and Patrick Le Callet. Llm-based interaction for content generation: A case study on the perception of employees in an it department. Proceedings of the 2023 ACM International Conference on Interactive Media Experiences, 2023. - [2] Claude Ahtropic. Claude. [Online]. Available: https://www.anthropic.com/claude, 2024. - [3] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, et al. Constitutional ai: Harmlessness from ai feedback. ArXiv, abs/2212.08073, 2022. - [4] Su Lin Blodgett, Solon Barocas, Hal Daum’e, and Hanna M. Wallach. Language (technology) is power: A critical survey of “bias” in nlp. ArXiv, abs/2005.14050, 2020. - [5] Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip Yu, and Lichao Sun. A survey of ai-generated content (aigc). ACM Computing Surveys, 57(5):1–38, 2025. - [6] Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? ArXiv, abs/2309.13788, 2023. - [7] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, et al. Evaluating large language models trained on code. ArXiv, abs/2107.03374, 2021. - [8] Michael Chen, Mike D’Arcy, Alisa Liu, Jared Fernandez, and Doug Downey. Aqua: An adversarially authored question-answer dataset for common sense. 2019. - [9] Yuxuan Chen, Rongpeng Li, Xiaoxue Yu, Zhifeng Zhao, and Honggang Zhang. Adaptive layer splitting for wireless llm inference in edge computing: A model-based reinforcement learning approach. arXiv preprint arXiv:2406.02616, 2024. - [10] Ethan Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, and Pengfei Liu. Factool: Factuality detection in generative ai - a tool augmented framework for multi-task and multi-domain scenarios. ArXiv, abs/2307.13528, 2023. - [11] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. ArXiv, abs/2110.14168, 2021. - [12] DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bing-Li Wang, Bochao Wu, et al. Deepseek-v3 technical report. 2024. - [13] Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. Chain-of-verification reduces hallucination in large language models. ArXiv, abs/2309.11495, 2023. - [14] Jinhao Duan, Shiqi Wang, James Diffenderfer, Lichao Sun, Tianlong Chen, Bhavya Kailkhura, and Kaidi Xu. Reta: Recursively thinking ahead to improve the strategic reasoning of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2232–2246, 2024. - [15] Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. Gtbench: Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations. Advances in Neural Information Processing Systems, 37:28219–28253, 2024. - [16] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, et al. The llama 3 herd of models. ArXiv, abs/2407.21783, 2024. - [17] Esin Durmus, He He, and Mona T. Diab. Feqa: A question answering evaluation framework for faithfulness assessment in abstractive summarization. ArXiv, abs/2005.03754, 2020. - [18] Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514, 2025. - [19] Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. In Annual Meeting of the Association for Computational Linguistics, 2023. - [20] Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, N. Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. Rarr: Researching and revising what language models say, using language models. In Annual Meeting of the Association for Computational Linguistics, 2022. - [21] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive critiquing. ArXiv, abs/2305.11738, 2023. - [22] Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alexa Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, A. Doucet, Orhan Firat, and Nando de Freitas. Reinforced self-training (rest) for language modeling. ArXiv, abs/2308.08998, 2023. - [23] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Xiaodong Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. ArXiv, abs/2103.03874, 2021. - [24] Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. ArXiv, abs/2210.11610, 2022. - [25] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. ArXiv, abs/2310.01798, 2023. - [26] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. - [27] Dongwei Jiang, Jingyu Zhang, Orion Weller, Nathaniel Weir, Benjamin Van Durme, and Daniel Khashabi. Self-[in]correct: Llms struggle with discriminating self-generated responses. 2024. - [28] Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, and Rui Zhang. Evaluating llms at detecting errors in llm responses. ArXiv, abs/2404.03602, 2024. - [29] Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can llms actually correct their own mistakes? a critical survey of self-correction of llms. Transactions of the Association for Computational Linguistics, 12:1417–1440, 2024. - [30] Geunwoo Kim, Pierre Baldi, and Stephen Marcus McAleer. Language models can solve computer tasks. ArXiv, abs/2303.17491, 2023. - [31] Jiayi Kuang, Jingyou Xie, Haohao Luo, Ronghao Li, Zhe Xu, Xianfeng Cheng, Yinghui Li, Xika Lin, and Ying Shen. Natural language understanding and inference with mllm in visual question answering: A survey. ArXiv, abs/2411.17558, 2024. - [32] Yuting Li, Lai Wei, Kaipeng Zheng, Jingyuan Huang, Linghe Kong, Lichao Sun, and Weiran Huang. Vision matters: Simple visual perturbations can boost multimodal math reasoning. arXiv preprint arXiv:2506.09736, 2025. - [33] Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, and Xiang Ren. Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. Advances in Neural Information Processing Systems, 36, 2024. - [34] Zicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo, Haowei Liu, and Yujiu Yang. Criticbench: Benchmarking llms for critique-correct reasoning. In Annual Meeting of the Association for Computational Linguistics, 2024. - [35] Dancheng Liu, Amir Nassereldine, Ziming Yang, Chenhui Xu, Yuting Hu, Jiajie Li, Utkarsh Kumar, Changjae Lee, and Jinjun Xiong. Large language models have intrinsic self-correction ability. ArXiv, abs/2406.15673, 2024. - [36] Guang-Da Liu, Haitao Mao, Bochuan Cao, Zhiyu Xue, Kristen Marie Johnson, Jiliang Tang, and Rongrong Wang. On the intrinsic self-correction capability of llms: Uncertainty and latent concept. ArXiv, abs/2406.02378, 2024. - [37] Yixin Liu, Yonghui Wu, Denghui Zhang, and Lichao Sun. Agentic autosurvey: Let llms survey llms. arXiv preprint arXiv:2509.18661, 2025. - [38] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. ArXiv, abs/2303.17651, 2023. - [39] Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. Generating training data with language models: Towards zero-shot language understanding. ArXiv, abs/2202.04538, 2022. - [40] Meta. Meta Llama 3. https://llama.meta.com/docs/model-cards-andprompt-formats/meta-llama-3/, 2024. - [41] Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. Fine-grained hallucination detection and editing for language models. ArXiv, abs/2401.06855, 2024. - [42] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022. - [43] Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. Language model self-improvement by reinforcement learning contemplation. ArXiv, abs/2305.14483, 2023. - [44] Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning feedback on intermediate representations. In Conference of the European Chapter of the Association for Computational Linguistics, 2023. - [45] Haritz Puerto, Tilek Chubakov, Xiaodan Zhu, Harish Tayyar Madabushi, and Iryna Gurevych. Fine-tuning with divergent chains of thought boosts reasoning through self-correction in language models. ArXiv, abs/2407.03181, 2024. - [46] Zac Pullar-Strecker, Katharina Dost, Eibe Frank, and Jörg Wicker. Hitting the target: stopping active learning at the cost-based optimum. Machine Learning, 113(4):1529–1547, 2024. - [47] Rifki Afina Putri, Faiz Ghifari Haznitrama, Dea Adhista, and Alice Oh. Can llm generate culturally relevant commonsense qa data? case study in indonesian and sundanese. In Conference on Empirical Methods in Natural Language Processing, 2024. - [48] Biqing Qi, Xinquan Chen, Junqi Gao, Dong Li, Jianxing Liu, Ligang Wu, and Bowen Zhou. Interactive continual learning: Fast and slow thinking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12882–12892, 2024. - [49] Maribeth Rauh, John F. J. Mellor, Jonathan Uesato, Po-Sen Huang, Johannes Welbl, Laura Weidinger, Sumanth Dathathri, Amelia Glaese, Geoffrey Irving, Iason Gabriel, William S. Isaac, and Lisa Anne Hendricks. Characteristics of harmful text: Towards rigorous benchmarking of language models. ArXiv, abs/2206.08325, 2022. - [50] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark. ArXiv, abs/2311.12022, 2023. - [51] Carl Orge Retzlaff, Srijita Das, Christabel Wayllace, Payam Mousavi, Mohammad Afshari, Tianpei Yang, Anna Saranti, Alessa Angerschmid, Matthew E Taylor, and Andreas Holzinger. Human-in-the-loop reinforcement learning: A survey and position on requirements, challenges, and opportunities. Journal of Artificial Intelligence Research, 79:359–415, 2024. - [52] Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, and Alex Wang. Questeval: Summarization asks for fact-based evaluation. In Conference on Empirical Methods in Natural Language Processing, 2021. - [53] Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, and Neil Zhenqiang Gong. Optimization-based prompt injection attack to llm-as-a-judge. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 660–674, 2024. - [54] Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In Neural Information Processing Systems, 2023. - [55] Chuanneng Sun, Songjun Huang, and Dario Pompili. Llm-based multi-agent reinforcement learning: Current and future directions. arXiv preprint arXiv:2405.11106, 2024. - [56] Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang, Jiangyue Yan, Yao Su, et al. A survey on post-training of large language models. arXiv e-prints, pages arXiv–2503, 2025. - [57] Guiyao Tie, Xueyang Zhou, Tianhe Gu, Ruihang Zhang, Chaoran Hu, Sizhe Zhang, Mengqu Sun, Yan Zhang, Pan Zhou, and Lichao Sun. Mmmr: Benchmarking massive multi-modal reasoning tasks. arXiv preprint arXiv:2505.16459, 2025. - [58] Gladys Tyen, Hassan Mansoor, Peter Chen, Tony Mak, and Victor Carbune. Llms cannot find reasoning errors, but can correct them! In Annual Meeting of the Association for Computational Linguistics, 2023. - [59] Alex Wang, Kyunghyun Cho, and Mike Lewis. Asking and answering questions to evaluate the factual consistency of summaries. ArXiv, abs/2004.04228, 2020. - [60] Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, and Yisen Wang. A theoretical understanding of self-correction through in-context alignment. ArXiv, abs/2405.18634, 2024. - [61] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, F. Xia, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903, 2022. - [62] Lai Wei, Yuting Li, Chen Wang, Yue Wang, Linghe Kong, Weiran Huang, and Lichao Sun. Unsupervised post-training for multi-modal llm reasoning via grpo. arXiv preprint arXiv:2505.22453, 2025. - [63] Lai Wei, Yuting Li, Kaipeng Zheng, Chen Wang, Yue Wang, Linghe Kong, Lichao Sun, and Weiran Huang. Advancing multimodal reasoning via reinforcement learning with cold start. arXiv preprint arXiv:2505.22334, 2025. - [64] Xueqing Wu, Yuheng Ding, Bingxuan Li, Pan Lu, Da Yin, Kai-Wei Chang, and Nanyun Peng. Visco: Benchmarking fine-grained critique and correction towards self-improvement in visual reasoning. ArXiv, abs/2412.02172, 2024. - [65] Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440, 2024. - [66] An Yang, Baosong Yang, Binyuan Hui, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024. - [67] Chang Yang, Xinrun Wang, Junzhe Jiang, Qinggang Zhang, and Xiao Huang. Evaluating world models with llm for decision making. ArXiv, abs/2411.08794, 2024. - [68] Hanqing Yang, Marie Siew, and Carlee Joe-Wong. An llm-based digital twin for optimizing human-in-the loop systems. arXiv preprint arXiv:2403.16809, 2024. - [69] Ling Yang, Zhaochen Yu, Tianjun Zhang, Minkai Xu, Joseph E. Gonzalez, Bin Cui, and Shuicheng Yan. Supercorrect: Supervising and correcting language models with error-driven insights. ArXiv, abs/2410.09008, 2024. - [70] Qwen An Yang, Baosong Yang, Beichen Zhang, et al. Qwen2.5 technical report. 2024. - [71] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing, 2018. - [72] Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, and Xuanjing Huang. A comprehensive capability analysis of gpt-3 and gpt-3.5 series models. ArXiv, abs/2303.10420, 2023. - [73] E. Zelikman, Yuhuai Wu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning. 2022. - [74] Jinghan Zhang, Xiting Wang, Weijieying Ren, Lu Jiang, Dongjie Wang, and Kunpeng Liu. Ratt: A thought structure for coherent and correct llm reasoning. ArXiv, abs/2406.02746, 2024. - [75] Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. How language model hallucinations can snowball. ArXiv, abs/2305.13534, 2023. - [76] Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers to self-correct reasoning. In Annual Meeting of the Association for Computational Linguistics, 2024. - [77] Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, and Lichao Sun. Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination. arXiv preprint arXiv:2411.12591, 2024. - [78] Jiasheng Zheng, Boxi Cao, Zhengzhao Ma, Ruotong Pan, Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. Beyond correctness: Benchmarking multi-dimensional code generation for large language models. ArXiv, abs/2407.11470, 2024. - [79] Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. International Journal of Machine Learning and Cybernetics, pages 1–65, 2024. ## NeurIPS Paper Checklist 1. Claims 1. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? 1. Answer: [Yes] 1. Justification: The abstract and introduction clearly state the main contributions of the paper, including evaluating the performance of three types of self-correction methods on three types of tasks, exploring the effects of mixture-based methods, and testing the correction performance of the reasoning model. These contributions are supported by the theoretical analysis and experimental results in the main text. In order to avoid over-generalization, the scope and limitations of the study are also discussed in the text. 1. Guidelines: - The answer NA means that the abstract and introduction do not include the claims made in the paper. - The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. - The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. - It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 1. Limitations 1. Question: Does the paper discuss the limitations of the work performed by the authors? 1. Answer: [Yes] 1. Justification: Section LABEL:sec:future discusses the limitations of our approach and future directions for scalability, including adaptive correction pipelines, integration in agents, and human-in-the-loop correction. 1. Guidelines: - The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. - The authors are encouraged to create a separate "Limitations" section in their paper. - The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. - The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. - The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. - The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. - If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. - While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 1. Theory assumptions and proofs 1. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? 1. Answer: [N/A] 1. Justification: This paper does not include theoretical results. 1. Guidelines: - The answer NA means that the paper does not include theoretical results. - All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced. - All assumptions should be clearly stated or referenced in the statement of any theorems. - The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. - Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. - Theorems and Lemmas that the proof relies upon should be properly referenced. 1. Experimental result reproducibility 1. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? 1. Answer: [Yes] 1. Justification: We provide all necessary details to reproduce the main experimental results. This includes complete descriptions of datasets, models, evaluation metrics, and prompt implementation details in Section LABEL:app:1 and Section G. 1. Guidelines: - The answer NA means that the paper does not include experiments. - If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. - If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. - Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. - While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example 1. If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. 1. If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. 1. If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). 1. We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 1. Open access to data and code 1. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? 1. Answer: [Yes] 1. Justification: We provide anonymized access to key code and data used in the experiments, with detailed instructions for reproducing the main results, including environment setup, running commands, and datasets. 1. Guidelines: - The answer NA means that paper does not include experiments requiring code. - Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details. - While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). - The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details. - The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc. - The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. - At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). - Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 1. Experimental setting/details 1. Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? 1. Answer: [Yes] 1. Justification: We specify all necessary experimental details to understand and interpret our results. This includes the data splits and model configuration. 1. Guidelines: - The answer NA means that the paper does not include experiments. - The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. - The full details can be provided either with the code, in appendix, or as supplemental material. 1. Experiment statistical significance 1. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? 1. Answer: [No] 1. Justification: While we provide quantitative results for all experiments, we do not include error bars or statistical significance tests. 1. Guidelines: - The answer NA means that the paper does not include experiments. - The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. - The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). - The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) - The assumptions made should be given (e.g., Normally distributed errors). - It should be clear whether the error bar is the standard deviation or the standard error of the mean. - It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. - For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). - If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 1. Experiments compute resources 1. Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? 1. Answer: [Yes] 1. Justification: We report the response time cost of calling API by different methods in Section 4.7. 1. Guidelines: - The answer NA means that the paper does not include experiments. - The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. - The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. - The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper). 1. Code of ethics 1. Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 1. Answer: [Yes] 1. Justification: We have carefully reviewed the NeurIPS Code of Ethics and confirm that our research complies with all relevant ethical guidelines. Our work does not involve human subjects, private or sensitive data, or potentially harmful applications. 1. Guidelines: - The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics. - If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. - The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 1. Broader impacts 1. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 1. Answer: [Yes] 1. Justification: We discuss the potential broader societal impacts of our work in the Impact Statement section. On the positive side, our method can help researchers understand the correction ability of LLMs better. 1. Guidelines: - The answer NA means that there is no societal impact of the work performed. - If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. - Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. - The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. - The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. - If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 1. Safeguards 1. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? 1. Answer: [No] 1. Justification: The paper does not introduce or release any models or datasets that pose significant risks of misuse. Therefore, no specific safeguards are necessary. 1. Guidelines: - The answer NA means that the paper poses no such risks. - Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. - Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. - We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 1. Licenses for existing assets 1. Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? 1. Answer: [Yes] 1. Justification: We make use of publicly available datasets and code assets, all of which are properly cited in the main paper. For each asset, we explicitly state the license (e.g., MIT, Apache 2.0, CC-BY 4.0) and ensure our use complies with the terms. Version information and source URLs are also provided where applicable. 1. Guidelines: - The answer NA means that the paper does not use existing assets. - The authors should cite the original paper that produced the code package or dataset. - The authors should state which version of the asset is used and, if possible, include a URL. - The name of the license (e.g., CC-BY 4.0) should be included for each asset. - For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. - If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. - For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. - If this information is not available online, the authors are encouraged to reach out to the asset’s creators. 1. New assets 1. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? 1. Answer: [Yes] 1. Justification: We introduce new assets in the paper, including the collected datasets for evaluating self-correction of LLMs. We provide complete documentation alongside these assets, including descriptions of their structure, usage instructions, licensing terms, known limitations, and guidelines for responsible use. All release materials are anonymized and hosted in accordance with NeurIPS submission policies. 1. Guidelines: - The answer NA means that the paper does not release new assets. - Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. - The paper should discuss whether and how consent was obtained from people whose asset is used. - At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 1. Crowdsourcing and research with human subjects 1. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? 1. Answer: [N/A] 1. Justification: The paper does not involve crowdsourcing or research with human subjects. All experiments are performed using synthetic or publicly available machine-generated datasets. 1. Guidelines: - The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. - Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. - According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 1. Institutional review board (IRB) approvals or equivalent for research with human subjects 1. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? 1. Answer: [N/A] 1. Justification: The paper does not involve crowdsourcing or research with human subjects, and thus IRB approval is not applicable. 1. Guidelines: - The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. - Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. - We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. - For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 1. Declaration of LLM usage 1. Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required. 1. Answer: [Yes] 1. Justification: This work involves the use of LLMs as a core component of our methodology. Specifically, we use instruction-based LLMs and reasoning-based LLMs for evaluations. The role of the LLM in our pipeline is described in detail in Sections 2.3. 1. Guidelines: - The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components. - Please refer to our LLM policy (https://neurips.cc/Conferences/2025/LLM) for what should or should not be described. ## Appendix A Overview of Correction Performance Figure 8 shows the comparison between the performance of the baseline and the average performance of the intrinsic correction and external correction methods on different tasks on different LLMs. It can be observed that both intrinsic correction and external correction outperform the baseline on most models and tasks. <details> <summary>x8.png Details</summary>  ### Visual Description ## Radar Chart Comparison: AI Model Performance Across Benchmarks with Correction Methods ### Overview The image displays a 3x3 grid of nine radar charts (spider charts). Each chart compares the performance of a specific Large Language Model (LLM) across seven different evaluation benchmarks. For each model, three performance lines are plotted, representing a baseline ("Base") and two correction methods ("S1" and "S2"). The charts collectively visualize how different correction techniques impact model performance across a standardized set of tasks. ### Components/Axes * **Chart Type:** Radar Charts (Spider Charts). * **Grid Layout:** 3 rows x 3 columns. * **Model Titles (Top of each chart, reading left-to-right, top-to-bottom):** 1. LLaMA3.1-8B-Instruct 2. LLaMA3.1-70B-Instruct 3. Qwen2.5-7B-Instruct 4. Qwen2.5-72B-Instruct 5. Claude3.5-Sonnet 6. GPT-3.5 7. GPT-4o 8. QWQ-32B-Instruct 9. DeepSeek-V3 * **Axes (Benchmarks):** Each chart has seven axes radiating from the center, labeled clockwise from the top: 1. GPQA (Graduate-level Google-Proof Q&A) 2. CS-QA (Computer Science Q&A) 3. HotpotQA (Multi-hop reasoning Q&A) 4. HumanEval (Code generation) 5. MATH (Mathematical problem-solving) 6. AQUA (Algebraic word problems) 7. GSM8K (Grade School Math) * **Scale:** Each axis has a concentric grid with values marked at 0.2, 0.4, 0.6, and 0.8. The center represents 0.0, and the outermost ring represents 1.0 (implied). * **Legend (Bottom center of the entire image):** * **Blue Line:** Base (Baseline) * **Orange Line:** S1 (Intrinsic Correction) * **Green Line:** S2 (External Correction) ### Detailed Analysis Below is an approximate data extraction for each model. Values are estimated based on the line's intersection with each axis. The trend for each correction method (S1, S2) is described relative to the Base (Blue) line. **1. LLaMA3.1-8B-Instruct** * **Trend:** S1 (Orange) shows minor improvements over Base. S2 (Green) shows significant improvement, especially on GSM8K and MATH. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.35 / ~0.38 / ~0.40 * CS-QA: ~0.55 / ~0.58 / ~0.60 * HotpotQA: ~0.50 / ~0.52 / ~0.55 * HumanEval: ~0.45 / ~0.48 / ~0.50 * MATH: ~0.30 / ~0.35 / ~0.55 * AQUA: ~0.40 / ~0.42 / ~0.45 * GSM8K: ~0.50 / ~0.55 / ~0.75 **2. LLaMA3.1-70B-Instruct** * **Trend:** S1 and S2 provide modest, consistent improvements across most benchmarks over the Base. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.50 / ~0.52 / ~0.55 * CS-QA: ~0.65 / ~0.68 / ~0.70 * HotpotQA: ~0.60 / ~0.62 / ~0.65 * HumanEval: ~0.55 / ~0.58 / ~0.60 * MATH: ~0.45 / ~0.50 / ~0.60 * AQUA: ~0.50 / ~0.52 / ~0.55 * GSM8K: ~0.65 / ~0.70 / ~0.80 **3. Qwen2.5-7B-Instruct** * **Trend:** S1 shows slight gains. S2 shows more pronounced gains, particularly on GSM8K and MATH. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.40 / ~0.42 / ~0.45 * CS-QA: ~0.60 / ~0.62 / ~0.65 * HotpotQA: ~0.55 / ~0.57 / ~0.60 * HumanEval: ~0.50 / ~0.52 / ~0.55 * MATH: ~0.35 / ~0.40 / ~0.60 * AQUA: ~0.45 / ~0.47 / ~0.50 * GSM8K: ~0.55 / ~0.60 / ~0.80 **4. Qwen2.5-72B-Instruct** * **Trend:** Both S1 and S2 show clear improvements over Base across all axes, with S2 generally outperforming S1. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.55 / ~0.58 / ~0.60 * CS-QA: ~0.70 / ~0.72 / ~0.75 * HotpotQA: ~0.65 / ~0.68 / ~0.70 * HumanEval: ~0.60 / ~0.62 / ~0.65 * MATH: ~0.50 / ~0.55 / ~0.65 * AQUA: ~0.55 / ~0.58 / ~0.60 * GSM8K: ~0.70 / ~0.75 / ~0.85 **5. Claude3.5-Sonnet** * **Trend:** High baseline performance. S1 and S2 provide incremental improvements, with S2 showing the largest gains on GSM8K and MATH. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.65 / ~0.68 / ~0.70 * CS-QA: ~0.75 / ~0.78 / ~0.80 * HotpotQA: ~0.70 / ~0.72 / ~0.75 * HumanEval: ~0.65 / ~0.68 / ~0.70 * MATH: ~0.55 / ~0.60 / ~0.70 * AQUA: ~0.60 / ~0.62 / ~0.65 * GSM8K: ~0.75 / ~0.80 / ~0.90 **6. GPT-3.5** * **Trend:** S1 and S2 corrections yield noticeable improvements across all benchmarks, with S2 consistently higher. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.45 / ~0.48 / ~0.50 * CS-QA: ~0.60 / ~0.63 / ~0.65 * HotpotQA: ~0.55 / ~0.58 / ~0.60 * HumanEval: ~0.50 / ~0.53 / ~0.55 * MATH: ~0.40 / ~0.45 / ~0.55 * AQUA: ~0.45 / ~0.48 / ~0.50 * GSM8K: ~0.60 / ~0.65 / ~0.80 **7. GPT-4o** * **Trend:** Strong baseline. S1 provides small gains. S2 provides more substantial gains, especially on GSM8K and MATH, pushing performance near the outer edge of the chart. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.60 / ~0.62 / ~0.65 * CS-QA: ~0.70 / ~0.72 / ~0.75 * HotpotQA: ~0.65 / ~0.68 / ~0.70 * HumanEval: ~0.60 / ~0.62 / ~0.65 * MATH: ~0.50 / ~0.55 / ~0.70 * AQUA: ~0.55 / ~0.58 / ~0.60 * GSM8K: ~0.70 / ~0.75 / ~0.90 **8. QWQ-32B-Instruct** * **Trend:** S1 shows minimal change from Base. S2 shows a dramatic expansion, particularly on GSM8K, MATH, and AQUA, creating a much larger area. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.40 / ~0.40 / ~0.45 * CS-QA: ~0.55 / ~0.55 / ~0.60 * HotpotQA: ~0.50 / ~0.50 / ~0.55 * HumanEval: ~0.45 / ~0.45 / ~0.50 * MATH: ~0.30 / ~0.35 / ~0.60 * AQUA: ~0.35 / ~0.38 / ~0.55 * GSM8K: ~0.50 / ~0.55 / ~0.85 **9. DeepSeek-V3** * **Trend:** S1 provides slight improvements. S2 provides more significant improvements, with the largest gains on GSM8K and MATH. * **Approximate Values (Base / S1 / S2):** * GPQA: ~0.50 / ~0.52 / ~0.55 * CS-QA: ~0.65 / ~0.68 / ~0.70 * HotpotQA: ~0.60 / ~0.62 / ~0.65 * HumanEval: ~0.55 / ~0.58 / ~0.60 * MATH: ~0.45 / ~0.50 / ~0.65 * AQUA: ~0.50 / ~0.52 / ~0.55 * GSM8K: ~0.65 / ~0.70 / ~0.85 ### Key Observations 1. **Universal Improvement from S2:** In every single chart, the Green line (S2 - External Correction) encloses the largest area, indicating it provides the most significant performance boost across the board compared to the Base and S1 methods. 2. **GSM8K and MATH are Most Responsive:** The benchmarks GSM8K (Grade School Math) and MATH show the most dramatic improvements when applying the S2 correction method across nearly all models. The green line spikes sharply outward on these axes. 3. **Model Size Correlation:** Larger models (e.g., LLaMA3.1-70B, Qwen2.5-72B, Claude3.5-Sonnet, GPT-4o) generally have higher baseline (blue) performance and maintain a more balanced, larger shape across all benchmarks. 4. **S1 vs. S2 Efficacy:** The S1 (Intrinsic Correction) method typically provides modest, incremental gains over the baseline. The S2 method is consistently more powerful, often transforming the performance profile, especially on reasoning-heavy tasks (MATH, GSM8K). 5. **Specialized Performance:** Some models show more pronounced "spikes" on certain axes even after correction (e.g., QWQ-32B's massive GSM8K spike with S2), suggesting the corrections may be particularly effective for specific types of reasoning tasks. ### Interpretation This visualization demonstrates the efficacy of two different correction techniques ("Intrinsic" S1 and "External" S2) on a suite of diverse AI models. The data strongly suggests that **External Correction (S2) is a universally more effective method** for boosting performance across a wide range of cognitive tasks, from mathematical reasoning (MATH, GSM8K) to code generation (HumanEval) and complex Q&A (GPQA, HotpotQA). The consistent, dramatic improvement on GSM8K and MATH indicates that these mathematical reasoning benchmarks may be particularly susceptible to the type of enhancement provided by the S2 method. This could imply that S2 involves techniques like improved tool use, external knowledge retrieval, or specialized reasoning chains that are highly beneficial for quantitative tasks. Furthermore, the charts reveal that while larger, more capable models start with a higher baseline, they still benefit significantly from these corrections, especially S2. This suggests that even state-of-the-art models have room for improvement through post-hoc or integrated correction mechanisms. The relative shapes of the polygons also hint at model "specializations"—some models may have inherent strengths in certain domains (like code or math) that are amplified or balanced by the correction methods. In essence, the image provides a compelling argument for the development and application of advanced correction techniques (particularly the "External" S2 approach) as a key strategy for enhancing the reliability and capability of AI systems across a broad spectrum of challenging tasks. </details> Figure 8: Comparative performance of different LLMs across various self-correction types and task scenarios. ## Appendix B Dataset ### B.1 Data Setting This benchmark includes datasets from various domains as shown in Table 7: GSM8K, AQuA, and MATH for mathematical reasoning, HotpotQA, CommonsenseQA, and GPQA for commonsense reasoning, and HumanEval for code generation. GSM8K and AQuA feature high school-level math and quantitative reasoning problems, while MATH provides a broader set of mathematical challenges. HotpotQA and CommonsenseQA (CS-QA) test multi-hop and commonsense reasoning, respectively, with GPQA expanding on the latter by including more diverse questions. HumanEval consists of programming problems to assess code generation abilities. Table 7: Statistics of the datasets used in CorrectBench. | Type | Dataset | Samples | License | | --- | --- | --- | --- | | Commonsense | HotpotQA | 300 | CC BY-SA 4.0 | | CommonsenseQA | 300 | MIT License | | | GPQA | 250 | Apache License 2.0 | | | Math | GSM8K | 250 | MIT License | | AQuA | 254 | Apache License 2.0 | | | MATH | 500 | MIT License | | | Coding | HumanEval | 164 | MIT License | | All | - | 2018 | - | To ensure consistency and reproducibility, each dataset is sampled to include 100 examples, selected through a randomized process. To refine the dataset quality, we remove outliers or irrelevant samples, ensuring a more accurate representation of typical error patterns. For mathematical reasoning datasets such as GSM8K, AQuA, and MATH, we ensure that selected problems span diverse difficulty levels to capture a comprehensive assessment of model performance. Similarly, commonsense reasoning datasets (HotpotQA, CS-QA, and GPQA) are curated to include a balanced mix of multi-hop and diverse reasoning tasks. For HumanEval, programming problems are filtered to maintain relevance to standard coding scenarios while avoiding overly specialized or ambiguous cases. ### B.2 Dataset and LLM Selection Table 8 summarizes the experimental evaluation of various self-correction methods across multiple datasets. The ‘✓’ indicates that the corresponding method is evaluated on the dataset, whereas the ‘-’ signifies that there are no experiments. Table 8: Selection of different datasets and LLMs for all self-correction methods. | Type | Methods | T1 | T2 | T3 | LLM Type | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | HotpotQA | CS-QA | GPQA | GSM8K | AQUA | MATH | HumanEval | M1 | M2 | | | | S1 | RCI | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | CoVe | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | Self-Refine | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - | ✓ | ✓ | | | Reflexion-v1 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - | ✓ | ✓ | | | S2 | Reflexion-v2 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - | ✓ | ✓ | | RARR | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | RATT | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | CRITIC | - | ✓ | - | ✓ | - | - | - | ✓ | ✓ | | | S3 | DCoT | - | ✓ | - | ✓ | - | - | - | - | - | | SCORE | - | ✓ | - | ✓ | - | - | - | - | - | | | Supercorrect | - | ✓ | - | ✓ | - | ✓ | ✓ | - | - | | ### B.3 Bootstrap Confidence Intervals for Resource Cost Results To validate the robustness of the trade-offs in Table 4, we report 95% bootstrap confidence intervals for each method’s accuracy across five random subsamples. The mean accuracy difference between S1 and S2 methods is statistically significant ( $p<0.05$ ), confirming that cost-normalized accuracy scales with correction depth. ## Appendix C Comprehensive Related Works ### C.1 Self-Correction Methods Theoretical Perspectives. Recent research has delved into the theoretical foundations of self-correction in large language models (LLMs), particularly examining how these models refine their outputs through iterative self-examination. Key transformer components, such as softmax attention and multi-head attention, have been identified as central mechanisms in enabling this self-correction process [60]. Several studies have highlighted the limitations of intrinsic self-correction. For instance, LLMs often encounter challenges when attempting to self-correct reasoning tasks without external feedback, resulting in degraded performance in specific scenarios [25]. Nevertheless, other research has demonstrated that intrinsic self-correction can be effective under certain conditions, such as employing zero-temperature settings and fair prompts. These conditions help LLMs enhance accuracy across various tasks by providing a more structured and deterministic framework for response refinement [35]. Further investigations reveal that intrinsic self-correction processes can converge over multiple iterations, yielding stable performance improvements, particularly in iterative and complex tasks [36]. However, some studies challenge the notion that LLMs can consistently enhance their outputs through self-correction alone. These findings suggest that LLMs often struggle to differentiate between previously generated alternatives, which limits the effectiveness of their self-correction mechanisms [27]. Additionally, innovative approaches such as the Divergent CoT (DCoT) method have been proposed. By generating and comparing multiple divergent reasoning chains, this method improves reasoning accuracy and facilitates more effective self-correction during complex reasoning tasks [45]. Self-Detection of Mistakes. Self-detection of mistakes in LLM responses, often with the aid of external information, has been widely explored across several domains. In misinformation detection, numerous studies have investigated how LLMs can identify and correct errors in the information they generate [75, 10, 6, 41]. Similarly, context-faithfulness, which examines whether LLMs maintain consistency with the context in which they are deployed, has also been a focal point in recent research [59, 17, 52]. Other works have concentrated on harmful content detection, where LLMs are tasked with identifying potentially harmful or offensive outputs [49], as well as bias detection, which aims to identify and mitigate biases in LLM responses [4, 19]. Despite significant progress, recent studies have shown that even state-of-the-art LLMs struggle to reliably detect their own mistakes across a variety of tasks [58, 28]. For instance, research demonstrates that LLMs often fail to identify errors in their outputs, even when performing complex reasoning or content generation tasks. These findings highlight a crucial gap in the current self-correction capabilities of LLMs, underscoring the need for further research into more robust error detection and correction mechanisms. Fine-tuning Methods. Self-training, or self-improvement, involves models utilizing their own responses to enhance performance. Several studies have explored the use of self-evaluation or self-correction for generating training data. For example, [3] and [22] leverage self-correction as a means to create training datasets, while [43] employ self-evaluation as a training signal to improve model performance. Another direction within self-training focuses on improving reasoning in LLMs by selecting high-quality generated outputs. [73] enhance reasoning by selecting outputs based on ground-truth final answers, whereas [24] emphasize self-consistency as a method for refining reasoning. [39] adopt a different approach by using high-confidence sentences generated by LLMs to train classifiers, demonstrating the potential of leveraging model confidence in improving task performance. ### C.2 Correction Benchmarks. Benchmarking the LLMs’ self-correction ability has prompted the development of specialized benchmarks for different tasks. For instance, CriticBench [34] evaluates critique ability using discrimination results, but it struggles with task-specific fine-grained metrics and reliance on costly human annotations or potentially biased GPT-4 outputs. In the realm of vision-language models, VISCO [64] focuses on self-correction in multimodal tasks, while Beyond Correctness [78] specifically targets self-correction in large models for code generation. Our CorrectBench focused on striking a trade-off between reasoning capability and efficiency, proposing more generalized and nuanced evaluation methods for complex reasoning tasks. ## Appendix D Limitations and Future Directions Adaptive Correction Pipelines. The iterative nature of self-correction in LLMs presents an opportunity to develop adaptive correction pipelines that dynamically determine when and how to refine model outputs. Current self-correction methods often employ a fixed number of refinement steps, which may not be optimal for all tasks or inputs. By investigating optimal stopping criteria [46], researchers can design systems that allocate computational resources more efficiently, thereby balancing accuracy and efficiency. Techniques such as reinforcement learning and meta-learning could be leveraged to train models that autonomously decide the appropriate number of correction iterations based on the complexity and confidence of their responses [9]. Furthermore, adaptive pipelines can incorporate uncertainty estimation to identify instances where additional refinement is necessary, potentially reducing unnecessary computation for straightforward queries while allocating more resources to complex or ambiguous cases [48]. Integration for Agents. Incorporating self-correction mechanisms into autonomous LLM-based agents can significantly enhance their functionality beyond static conversational roles. Agents equipped with self-correction capabilities are better suited to perform complex, multi-step tasks that require continuous adaptation and error mitigation. This integration can enable agents to engage in more sophisticated interactions, such as dynamic problem-solving, real-time data analysis, and interactive decision-making in diverse domains [55, 33]. By embedding self-correction within the agent’s operational framework, these systems can achieve higher levels of autonomy and reliability, making them more effective in real-world applications. Additionally, the ability to self-correct allows agents to better handle unforeseen scenarios and maintain performance consistency across varying contexts, thereby broadening their applicability and utility. Human-in-the-Loop Correction. While automated self-correction methods offer significant improvements in model accuracy and reliability, integrating human feedback can further enhance these outcomes, especially in high-stakes or sensitive applications. Human-in-the-loop (HITL) correction involves leveraging expert knowledge to validate and refine model outputs, ensuring that the responses meet stringent quality and safety standards [51]. Effective HITL systems can combine the strengths of automated refinement with the nuanced understanding of human experts, thereby addressing limitations inherent in purely algorithmic approaches. For instance, in domains such as medical diagnostics, legal reasoning, or financial analysis, expert oversight can prevent critical errors and ensure that the model adheres to ethical guidelines and regulatory requirements. Future research should focus on developing seamless interfaces for human-AI collaboration, optimizing the balance between automation and manual intervention, and exploring scalable methods for incorporating diverse expert inputs without compromising efficiency [68]. ## Appendix E Additional Experiments for Performance Improvement This section evaluates performance gains from self-correction methods across various LLMs and datasets. Figure 9 shows the performance gains of the CoT method across models and datasets. Most models surpass the baseline (‘y=0’ for Base), though some fall short. For instance, LLaMA 3.1-8B-Instruct performs poorly overall, and no model achieves improvements on the HumanEval dataset. ### E.1 Performance Gains for Intrinsic Correction methods Performance Gains for RCI. Figure 10 illustrates the performance gains from the RCI method across all LLMs on the evaluated datasets. Over half the data points surpass the baseline, demonstrating its effectiveness. Notably, for GPT-4o, nearly all data points exceed the baseline, highlighting significant improvements. Performance Gains for CoVe. As depicted in Figure 11, the CoVe method delivers significant performance enhancements across all LLMs on the evaluated datasets. The majority of data points surpass the baseline, with substantial improvement magnitudes, underscoring the effectiveness of the CoVe method. Performance Gains for Self-Refine. Figure 12 demonstrates the performance gains achieved by the Self-Refine method across all LLMs on the selected datasets. Nearly all data points lie above the baseline. In particular for the GPQA dataset, all LLMs exhibit significant performance improvements. Performance Changes for Reflexion-v1. Figure 13 depicts the performance outcomes of the Reflexion-v1 method without tools across all LLMs on the evaluated datasets. In this scenario, nearly all data points fall below the baseline, indicating a performance decline across most models and datasets. ### E.2 Performance Gains for External Correction methods Performance Gains for Reflexion-v2. In contrast to the results without tools, Figure 14 highlights the performance improvements achieved by Reflexion-v2 with tools. The majority of data points surpass the baseline, demonstrating the effectiveness of tool integration in enhancing performance. Performance Gains for RARR. Figure 15 illustrates the performance improvements resulting from the RARR method across all LLMs on the evaluated datasets. Nearly all data points exceed the baseline. Specifically, the GPQA dataset shows significant performance enhancements across all models. Performance Gains for RATT. Figure 16 showcases the performance gains achieved by the RATT method across various LLMs on the evaluated datasets. Most data points lie above the baseline, reflecting the positive impact of the RATT method in improving model performance. <details> <summary>x9.png Details</summary>  ### Visual Description ## Scatter Plot: LLM Performance Delta Across Datasets ### Overview The image is a scatter plot (or dot plot) comparing the performance change (Δ%) of various Large Language Models (LLMs) across seven different benchmark datasets. Each model is represented by a unique colored shape, and its performance relative to a baseline (Δ=0) is plotted for each dataset. The plot is divided into vertical sections by dataset. ### Components/Axes * **X-Axis (Horizontal):** Labeled "Dataset". It lists seven distinct benchmark categories, separated by vertical dashed lines. From left to right: 1. HotpotQA 2. CS-QA 3. AQUA 4. GSM8K 5. MATH 6. GPQA 7. HumanEval * **Y-Axis (Vertical):** Labeled "Δ (%)". It represents the percentage change in performance. The scale ranges from -30 to +30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Baseline:** A horizontal dashed green line at Δ=0, indicating no change in performance. * **Legend:** Located on the right side of the chart. It maps symbols and colors to specific LLMs and categories: * **Open LLMs (Circle symbols):** * `LLaMA3.1-8B` (Teal circle) * `LLaMA3.1-70B` (Yellow circle) * `Qwen2.5-7B` (Light purple circle) * `Qwen2.5-72B` (Red circle) * **Close LLMs (Pentagon symbols):** * `Claude3.5` (Blue pentagon) * `GPT-3.5` (Orange pentagon) * `GPT-4o` (Green pentagon) * `QWQ-32B` (Light blue pentagon) * **Reasoning LLM (Diamond symbol):** * `DeepSeek-V3` (Purple diamond) * **Baseline:** `Baseline (Δ=0)` (Green dashed line) ### Detailed Analysis The following lists the approximate Δ (%) value for each model within each dataset section. Values are estimated based on the vertical position of each symbol relative to the y-axis scale. **1. HotpotQA** * `LLaMA3.1-8B` (Teal circle): ~ -4% * `LLaMA3.1-70B` (Yellow circle): ~ +5% * `Qwen2.5-7B` (Light purple circle): ~ -2% * `Qwen2.5-72B` (Red circle): ~ +6% * `Claude3.5` (Blue pentagon): ~ +7% * `GPT-3.5` (Orange pentagon): ~ +5% * `GPT-4o` (Green pentagon): ~ +7% * `QWQ-32B` (Light blue pentagon): ~ +1% * `DeepSeek-V3` (Purple diamond): ~ +1% **2. CS-QA** * `LLaMA3.1-8B` (Teal circle): ~ -2% * `LLaMA3.1-70B` (Yellow circle): ~ +1% * `Qwen2.5-7B` (Light purple circle): ~ -19% * `Qwen2.5-72B` (Red circle): ~ +7% * `Claude3.5` (Blue pentagon): ~ +3% * `GPT-3.5` (Orange pentagon): ~ +2% * `GPT-4o` (Green pentagon): ~ +3% * `QWQ-32B` (Light blue pentagon): ~ -4% * `DeepSeek-V3` (Purple diamond): ~ -8% **3. AQUA** * `LLaMA3.1-8B` (Teal circle): ~ -9% * `LLaMA3.1-70B` (Yellow circle): ~ 0% * `Qwen2.5-7B` (Light purple circle): ~ +3% * `Qwen2.5-72B` (Red circle): ~ 0% * `Claude3.5` (Blue pentagon): ~ -17% * `GPT-3.5` (Orange pentagon): ~ +10% * `GPT-4o` (Green pentagon): ~ -4% * `QWQ-32B` (Light blue pentagon): ~ +10% * `DeepSeek-V3` (Purple diamond): ~ +10% **4. GSM8K** * `LLaMA3.1-8B` (Teal circle): ~ -3% * `LLaMA3.1-70B` (Yellow circle): ~ +5% * `Qwen2.5-7B` (Light purple circle): ~ -3% * `Qwen2.5-72B` (Red circle): ~ +3% * `Claude3.5` (Blue pentagon): ~ +2% * `GPT-3.5` (Orange pentagon): ~ +18% * `GPT-4o` (Green pentagon): ~ +7% * `QWQ-32B` (Light blue pentagon): ~ +3% * `DeepSeek-V3` (Purple diamond): ~ +18% **5. MATH** * `LLaMA3.1-8B` (Teal circle): ~ -7% * `LLaMA3.1-70B` (Yellow circle): ~ -18% * `Qwen2.5-7B` (Light purple circle): ~ -6% * `Qwen2.5-72B` (Red circle): ~ -3% * `Claude3.5` (Blue pentagon): ~ +2% * `GPT-3.5` (Orange pentagon): ~ +5% * `GPT-4o` (Green pentagon): ~ +4% * `QWQ-32B` (Light blue pentagon): ~ +2% * `DeepSeek-V3` (Purple diamond): ~ -11% **6. GPQA** * `LLaMA3.1-8B` (Teal circle): ~ +3% * `LLaMA3.1-70B` (Yellow circle): ~ +8% * `Qwen2.5-7B` (Light purple circle): ~ +5% * `Qwen2.5-72B` (Red circle): ~ +6% * `Claude3.5` (Blue pentagon): ~ -16% * `GPT-3.5` (Orange pentagon): ~ -9% * `GPT-4o` (Green pentagon): ~ -9% * `QWQ-32B` (Light blue pentagon): ~ -7% * `DeepSeek-V3` (Purple diamond): ~ 0% **7. HumanEval** * `LLaMA3.1-8B` (Teal circle): ~ -22% * `LLaMA3.1-70B` (Yellow circle): ~ -16% * `Qwen2.5-7B` (Light purple circle): ~ -16% * `Qwen2.5-72B` (Red circle): ~ -24% * `Claude3.5` (Blue pentagon): ~ -4% * `GPT-3.5` (Orange pentagon): ~ -4% * `GPT-4o` (Green pentagon): ~ -12% * `QWQ-32B` (Light blue pentagon): ~ -4% * `DeepSeek-V3` (Purple diamond): ~ -12% ### Key Observations 1. **Dataset-Specific Performance:** Model performance is highly variable across datasets. No single model consistently outperforms others on all tasks. 2. **Strong Positive Outliers:** `GPT-3.5` and `DeepSeek-V3` show the largest positive gains on the **GSM8K** dataset (~+18%). `GPT-3.5` also performs well on **AQUA** (~+10%). 3. **Strong Negative Outliers:** The **HumanEval** dataset shows the most significant negative performance deltas across nearly all models, with `Qwen2.5-72B` showing the largest drop (~-24%). `LLaMA3.1-70B` shows a notable drop on **MATH** (~-18%). 4. **Model Category Trends:** * **Open LLMs (Circles):** Show mixed results. `Qwen2.5-72B` (Red) is often positive but has a severe drop on HumanEval. `LLaMA3.1-70B` (Yellow) is generally positive except on MATH and HumanEval. * **Close LLMs (Pentagons):** `GPT-3.5` (Orange) and `GPT-4o` (Green) are frequently above the baseline. `Claude3.5` (Blue) shows significant negative performance on AQUA and GPQA. * **Reasoning LLM (Diamond):** `DeepSeek-V3` (Purple) is highly variable, with large positive gains on GSM8K and AQUA, but negative performance on CS-QA and MATH. 5. **Baseline Comparison:** The majority of data points lie above the Δ=0 baseline, suggesting that, for these specific benchmarks and this comparison metric, most models show a performance increase. The notable exception is the **HumanEval** dataset. ### Interpretation This chart likely illustrates the performance change (Δ%) of various LLMs when subjected to a specific intervention, technique, or compared to a different baseline model, across a suite of standard benchmarks. The "Δ (%)" metric implies a relative improvement or degradation. * **Task Specialization:** The data strongly suggests that model capabilities are not monolithic. A model excelling in mathematical reasoning (e.g., GSM8K, AQUA) may not be the best for coding (HumanEval) or complex question answering (GPQA). This highlights the importance of selecting models based on the specific task domain. * **Impact of Scale and Type:** The comparison between different sizes of the same model family (e.g., LLaMA3.1-8B vs. 70B, Qwen2.5-7B vs. 72B) does not show a uniform "bigger is better" trend. The performance delta is context-dependent, indicating that the intervention or comparison being measured affects models of different scales in non-linear ways. * **Coding Benchmark Challenge:** The uniformly negative performance on **HumanEval** is a critical finding. It suggests that whatever change is being measured (e.g., a new training method, quantization, a different prompting strategy) has a detrimental effect on code generation capabilities across a wide range of model architectures and sizes. This could be a significant trade-off to consider. * **Volatility of "Reasoning" Models:** The high variance in `DeepSeek-V3`'s performance—being a top performer on some tasks and a poor performer on others—might indicate that its "reasoning" specialization makes it particularly sensitive to the conditions of the benchmark or the nature of the intervention being tested. In summary, the chart provides a nuanced view of LLM performance, moving beyond simple "state-of-the-art" rankings to show how different models react across a diverse set of challenges. The key takeaway is the lack of a universally superior model and the presence of significant task-specific trade-offs, particularly concerning code generation. </details> Figure 9: Performance Gains for CoT. <details> <summary>x10.png Details</summary>  ### Visual Description ## Scatter Plot: Performance Change (Δ%) of Various LLMs Across Datasets ### Overview The image is a scatter plot comparing the performance change (Δ%) of multiple Large Language Models (LLMs) across seven different benchmark datasets. The plot uses distinct markers (shapes and colors) to represent each model, with a horizontal baseline at Δ=0%. The data points are grouped by dataset along the x-axis. ### Components/Axes * **X-axis (Categorical):** Labeled "Dataset". The categories are, from left to right: `HotpotQA`, `CS-QA`, `AQUA`, `GSM8K`, `MATH`, `GPQA`, `HumanEval`. * **Y-axis (Numerical):** Labeled "Δ(%)". The scale ranges from -30 to +30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Baseline:** A horizontal, dashed, light green line at `Δ=0`, labeled "Baseline (Δ=0)" in the legend. * **Legend (Positioned on the right side of the chart):** Contains 12 entries, defining the model names and their corresponding marker styles. * **Models & Markers:** * `LLaMA3.1-8B`: Teal circle (○) * `LLaMA3.1-70B`: Yellow circle (○) * `Qwen2.5-7B`: Purple circle (○) * `Qwen2.5-72B`: Red circle (○) * `Claude3.5`: Blue pentagon (⬠) * `GPT-3.5`: Orange pentagon (⬠) * `GPT-4o`: Green pentagon (⬠) * `QWQ-32B`: Purple diamond (◇) * `DeepSeek-V3`: Pink diamond (◇) * **Model Type Indicators (Shapes):** * `Open LLM`: Circle (○) * `Close LLM`: Pentagon (⬠) * `Reasoning LLM`: Diamond (◇) ### Detailed Analysis The following table reconstructs the approximate Δ(%) values for each model on each dataset, based on visual inspection of the scatter plot. Values are approximate and carry uncertainty (±2-3%). | Dataset | LLaMA3.1-8B (Teal ○) | LLaMA3.1-70B (Yellow ○) | Qwen2.5-7B (Purple ○) | Qwen2.5-72B (Red ○) | Claude3.5 (Blue ⬠) | GPT-3.5 (Orange ⬠) | GPT-4o (Green ⬠) | QWQ-32B (Purple ◇) | DeepSeek-V3 (Pink ◇) | | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | **HotpotQA** | -15% | -3% | -7% | +1% | -2% | +4% | +3% | +16% | -4% | | **CS-QA** | -1% | -14% | -8% | -1% | +6% | +9% | +1% | -10% | -15% | | **AQUA** | +3% | -1% | +7% | +5% | +6% | +24% | +5% | -4% | +7% | | **GSM8K** | -10% | -3% | +4% | +4% | -2% | +14% | +5% | -6% | +1% | | **MATH** | +1% | -2% | -1% | -6% | +1% | -1% | +8% | -11% | +1% | | **GPQA** | +12% | +10% | -4% | +4% | -10% | -4% | -4% | +3% | -4% | | **HumanEval** | -6% | -21% | -1% | -6% | -1% | -4% | +4% | -6% | -2% | ### Key Observations 1. **High Variance:** Performance change (Δ%) varies dramatically both across models for a single dataset and for a single model across datasets. No single model consistently outperforms others. 2. **Dataset Difficulty:** `HumanEval` and `CS-QA` show the most negative trends overall, with several models showing significant performance drops (e.g., LLaMA3.1-70B at ~-21% on HumanEval). `AQUA` and `GPQA` show more positive trends. 3. **Model-Specific Trends:** * **GPT-3.5 (Orange ⬠):** Shows the highest single positive outlier, with a Δ of ~+24% on `AQUA`. It also performs well on `GSM8K` (+14%). * **QWQ-32B (Purple ◇):** Exhibits extreme volatility. It has the highest positive Δ on `HotpotQA` (~+16%) but one of the lowest on `MATH` (~-11%). * **LLaMA3.1-70B (Yellow ○):** Shows significant negative performance on `CS-QA` (~-14%) and `HumanEval` (~-21%). * **GPT-4o (Green ⬠):** Shows relatively stable, modest positive performance, with its highest gain on `MATH` (~+8%). 4. **Model Type Pattern:** There is no clear, consistent separation in performance trends between the indicated model types (Open LLM, Close LLM, Reasoning LLM). All three types have models showing both high positive and high negative Δ values. ### Interpretation This chart visualizes the **relative performance shift** of various LLMs compared to an unspecified baseline (likely a previous model version or a different prompting/evaluation setup). The key takeaway is the **highly dataset-dependent nature of LLM performance improvements or regressions**. * **What the data suggests:** A model's architecture or training that leads to a large gain on one type of reasoning task (e.g., GPT-3.5 on `AQUA`) does not guarantee similar gains on another (e.g., its performance on `GPQA` is slightly negative). This highlights the challenge of creating general-purpose AI and the importance of multi-dataset benchmarking. * **Relationship between elements:** The scatter plot format effectively shows the distribution and spread of performance changes. The clustering of points around the zero line for some datasets (e.g., `MATH`) versus the wide dispersion for others (e.g., `AQUA`, `GPQA`) indicates varying levels of consensus or divergence in how models handle different problem types. * **Notable anomalies:** The extreme positive outlier for GPT-3.5 on `AQUA` and the extreme negative outlier for LLaMA3.1-70B on `HumanEval` are significant. These could indicate specific strengths or weaknesses in the models' training data, reasoning capabilities, or sensitivity to the evaluation methodology of those particular benchmarks. The volatility of the "Reasoning LLM" QWQ-32B suggests its specialized training may yield high rewards on certain tasks but at the cost of robustness on others. </details> Figure 10: Performance Gains for RCI. <details> <summary>x11.png Details</summary>  ### Visual Description ## Scatter Plot: Performance Delta of Various Language Models Across Datasets ### Overview The image is a scatter plot comparing the performance change (Δ%) of multiple large language models (LLMs) across seven different benchmark datasets. Each data point represents a specific model's performance on a dataset relative to a baseline (Δ=0). The models are categorized as Open LLMs, Closed LLMs, or Reasoning LLMs, indicated by different marker shapes. ### Components/Axes * **Chart Type:** Scatter plot with categorical x-axis. * **X-Axis (Horizontal):** Labeled "Dataset". Categories from left to right: `HotpotQA`, `CS-QA`, `AQUA`, `GSM8K`, `MATH`, `GPQA`, `HumanEval`. * **Y-Axis (Vertical):** Labeled "Δ (%)". Scale ranges from -30 to 30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Baseline:** A horizontal dashed green line at `Δ=0`, labeled "Baseline (Δ=0)" in the legend. * **Legend (Right Side):** Lists models and marker types. * **Models (by color):** * `LLaMA3.1-8B` (Teal circle) * `LLaMA3.1-70B` (Light green circle) * `Qwen2.5-7B` (Light purple circle) * `Qwen2.5-72B` (Red circle) * `Claude3.5` (Blue pentagon) * `GPT-3.5` (Orange pentagon) * `GPT-4o` (Green pentagon) * `QWQ-32B` (Pink diamond) * `DeepSeek-V3` (Purple diamond) * **Marker Types (by shape):** * `Open LLM` (Circle) * `Close LLM` (Pentagon) * `Reasoning LLM` (Diamond) ### Detailed Analysis The chart shows the performance delta (Δ%) for each model on each dataset. Below is an approximate extraction of data points, grouped by dataset. Values are estimated from the plot's grid. **1. HotpotQA:** * LLaMA3.1-8B: ~ -8% * LLaMA3.1-70B: ~ +8% * Qwen2.5-7B: ~ -1% * Qwen2.5-72B: ~ +6% * Claude3.5: ~ +1% * GPT-3.5: ~ +3% * GPT-4o: ~ +2% * QWQ-32B: ~ +14% * DeepSeek-V3: ~ -1% **2. CS-QA:** * LLaMA3.1-8B: ~ -10% * LLaMA3.1-70B: ~ -5% * Qwen2.5-7B: ~ -16% * Qwen2.5-72B: ~ -1% * Claude3.5: ~ +10% * GPT-3.5: ~ +12% * GPT-4o: ~ +8% * QWQ-32B: ~ -10% * DeepSeek-V3: ~ 0% **3. AQUA:** * LLaMA3.1-8B: ~ -3% * LLaMA3.1-70B: ~ +14% * Qwen2.5-7B: ~ +10% * Qwen2.5-72B: ~ +18% * Claude3.5: ~ +2% * GPT-3.5: ~ +26% * GPT-4o: ~ +10% * QWQ-32B: ~ +7% * DeepSeek-V3: ~ +6% **4. GSM8K:** * LLaMA3.1-8B: ~ -1% * LLaMA3.1-70B: ~ -4% * Qwen2.5-7B: ~ +1% * Qwen2.5-72B: ~ +3% * Claude3.5: ~ +2% * GPT-3.5: ~ +16% * GPT-4o: ~ +5% * QWQ-32B: ~ +30% (Highest point on chart) * DeepSeek-V3: ~ 0% **5. MATH:** * LLaMA3.1-8B: ~ 0% * LLaMA3.1-70B: ~ +9% * Qwen2.5-7B: ~ +11% * Qwen2.5-72B: ~ -3% * Claude3.5: ~ +2% * GPT-3.5: ~ +14% * GPT-4o: ~ +16% * QWQ-32B: ~ -7% * DeepSeek-V3: ~ -12% **6. GPQA:** * LLaMA3.1-8B: ~ +6% * LLaMA3.1-70B: ~ +24% * Qwen2.5-7B: ~ +29% (Highest for this dataset) * Qwen2.5-72B: ~ +23% * Claude3.5: ~ +11% * GPT-3.5: ~ +21% * GPT-4o: ~ +23% * QWQ-32B: ~ +14% * DeepSeek-V3: ~ +21% **7. HumanEval:** * LLaMA3.1-8B: ~ -10% * LLaMA3.1-70B: ~ +25% * Qwen2.5-7B: ~ +6% * Qwen2.5-72B: ~ -23% (Lowest point on chart) * Claude3.5: ~ 0% * GPT-3.5: ~ +4% * GPT-4o: ~ +9% * QWQ-32B: ~ -9% * DeepSeek-V3: ~ -9% ### Key Observations 1. **High Variance in HumanEval:** The `HumanEval` dataset shows the widest spread of performance deltas, ranging from approximately -23% (Qwen2.5-72B) to +25% (LLaMA3.1-70B). 2. **Top Performer on GSM8K:** The reasoning model `QWQ-32B` achieves the single highest performance delta on the chart, at approximately +30% on the `GSM8K` dataset. 3. **Consistently Strong Closed LLMs:** Closed models like `GPT-4o` and `Claude3.5` show generally positive deltas across most datasets, with `GPT-4o` never dropping below the baseline. 4. **Model-Specific Strengths/Weaknesses:** * `LLaMA3.1-70B` performs strongly on `GPQA` and `HumanEval` but negatively on `CS-QA`. * `Qwen2.5-72B` shows a significant negative outlier on `HumanEval` but strong positive results on `AQUA` and `GPQA`. * `DeepSeek-V3` (Reasoning LLM) shows mixed results, with its best performance on `GPQA` and its worst on `MATH`. 5. **Baseline Comparison:** The majority of data points lie above the `Δ=0` baseline, suggesting that most models, on most datasets, show a performance change (likely improvement, though the direction of "delta" isn't explicitly defined as improvement) relative to the baseline. ### Interpretation This chart provides a comparative snapshot of LLM performance across diverse reasoning and knowledge tasks. The data suggests that: * **Task-Specific Performance:** No single model dominates across all datasets. Performance is highly contingent on the specific benchmark, indicating that model capabilities are specialized. For example, a model strong in mathematical reasoning (`GSM8K`, `MATH`) may not be the best at code generation (`HumanEval`) or complex question answering (`HotpotQA`). * **Scale vs. Architecture:** Larger open models (e.g., `LLaMA3.1-70B`) often outperform their smaller counterparts, but not universally. The presence of reasoning-specialized models (`QWQ-32B`, `DeepSeek-V3`) with high variance suggests that architectural focus can lead to exceptional performance in specific domains (like math for QWQ-32B) but may not generalize. * **The "Delta" Ambiguity:** The chart uses "Δ (%)" without specifying the reference point or direction. Assuming Δ=0 is a previous model version or a standard baseline, positive values indicate improvement. The widespread positive deltas could reflect recent advancements in the field. The significant negative outlier for `Qwen2.5-72B` on `HumanEval` warrants investigation—it could indicate a regression, a benchmarking anomaly, or a specific weakness in code generation for that model version. * **Benchmark Diversity:** The selection of datasets covers a broad spectrum: multi-hop reasoning (HotpotQA), science QA (CS-QA, GPQA), math (AQUA, GSM8K, MATH), and coding (HumanEval). This diversity is crucial for a holistic evaluation, as the chart clearly shows that model rankings shift dramatically depending on the task. In essence, the visualization underscores the complexity of evaluating LLMs, highlighting that "best" is a context-dependent label and that the field continues to exhibit rapid, uneven progress across different cognitive domains. </details> Figure 11: Performance Gains for CoVe. <details> <summary>x12.png Details</summary>  ### Visual Description ## Scatter Plot: Model Performance Δ (%) Across Datasets ### Overview The image is a scatter plot comparing the performance change (Δ, in percentage) of various large language models (LLMs) across seven different benchmark datasets. The plot uses distinct symbols and colors to represent different model families and types (Open, Closed, Reasoning). A horizontal dashed line at Δ=0 serves as the baseline. ### Components/Axes * **X-Axis (Categorical):** Labeled "Dataset". The categories from left to right are: 1. HotpotQA 2. CS-QA 3. AQUA 4. GSM8K 5. MATH 6. GPQA 7. HumanEval * **Y-Axis (Numerical):** Labeled "Δ (%)". The scale ranges from -30 to 30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Legend (Top-Right):** Positioned in the top-right corner of the plot area. It defines the following model symbols and colors: * **LLaMA3.1-8B:** Teal circle (○) * **LLaMA3.1-70B:** Light green circle (○) * **Qwen2.5-7B:** Purple circle (○) * **Qwen2.5-72B:** Red circle (○) * **Claude3.5:** Blue circle (○) * **GPT-3.5:** Orange pentagon (⬠) * **GPT-4o:** Green pentagon (⬠) * **QWQ-32B:** Pink diamond (◇) * **DeepSeek-V3:** Purple diamond (◇) * **Open LLM:** Circle symbol (○) - *This is a category marker, not a specific model.* * **Close LLM:** Pentagon symbol (⬠) - *This is a category marker, not a specific model.* * **Reasoning LLM:** Diamond symbol (◇) - *This is a category marker, not a specific model.* * **Baseline:** A horizontal, light green dashed line at Δ=0, labeled "Baseline (Δ=0)" in the legend. ### Detailed Analysis Data points are plotted for each model within each dataset column. Approximate Δ (%) values are extracted below, grouped by dataset. **1. HotpotQA** * **Trend:** Most models show positive Δ, clustered between 0% and 10%. One significant outlier is below -10%. * **Data Points (Approximate):** * LLaMA3.1-8B (Teal ○): ~ -14% * LLaMA3.1-70B (Light Green ○): ~ +4% * Qwen2.5-7B (Purple ○): ~ +1% * Qwen2.5-72B (Red ○): ~ +7% * Claude3.5 (Blue ○): ~ +6% * GPT-3.5 (Orange ⬠): ~ +8% * GPT-4o (Green ⬠): ~ +6% * QWQ-32B (Pink ◇): ~ +22% **2. CS-QA** * **Trend:** Models are tightly clustered near the baseline, with most between -5% and +10%. One model is a clear outlier below -10%. * **Data Points (Approximate):** * LLaMA3.1-8B (Teal ○): ~ -12% * LLaMA3.1-70B (Light Green ○): ~ +1% * Qwen2.5-7B (Purple ○): ~ -2% * Qwen2.5-72B (Red ○): ~ +3% * Claude3.5 (Blue ○): ~ +6% * GPT-3.5 (Orange ⬠): ~ +5% * GPT-4o (Green ⬠): ~ +8% * DeepSeek-V3 (Purple ◇): ~ 0% * QWQ-32B (Pink ◇): ~ 0% **3. AQUA** * **Trend:** A wider spread of performance. Several models show strong positive gains (10-20%), while others are near or slightly below baseline. * **Data Points (Approximate):** * LLaMA3.1-8B (Teal ○): ~ -1% * LLaMA3.1-70B (Light Green ○): ~ +4% * Qwen2.5-7B (Purple ○): ~ +11% * Qwen2.5-72B (Red ○): ~ +3% * Claude3.5 (Blue ○): ~ -1% * GPT-3.5 (Orange ⬠): ~ +15% * GPT-4o (Green ⬠): ~ +18% * DeepSeek-V3 (Purple ◇): ~ +11% * QWQ-32B (Pink ◇): ~ +17% **4. GSM8K** * **Trend:** Mixed results. Some models show strong positive Δ (~20%), while others are negative or near zero. * **Data Points (Approximate):** * LLaMA3.1-8B (Teal ○): ~ -5% * LLaMA3.1-70B (Light Green ○): ~ +1% * Qwen2.5-7B (Purple ○): ~ -2% * Qwen2.5-72B (Red ○): ~ +2% * Claude3.5 (Blue ○): ~ +3% * GPT-3.5 (Orange ⬠): ~ +19% * GPT-4o (Green ⬠): ~ +5% * DeepSeek-V3 (Purple ◇): ~ +4% * QWQ-32B (Pink ◇): ~ +20% **5. MATH** * **Trend:** Generally positive performance, with most models between 0% and +15%. One model is at the baseline. * **Data Points (Approximate):** * LLaMA3.1-8B (Teal ○): ~ -2% * LLaMA3.1-70B (Light Green ○): ~ +4% * Qwen2.5-7B (Purple ○): ~ +4% * Qwen2.5-72B (Red ○): ~ +12% * Claude3.5 (Blue ○): ~ +6% * GPT-3.5 (Orange ⬠): ~ +15% * GPT-4o (Green ⬠): ~ +7% * DeepSeek-V3 (Purple ◇): ~ 0% * QWQ-32B (Pink ◇): ~ +8% **6. GPQA** * **Trend:** This dataset shows the highest and most consistently positive Δ values across nearly all models, with a cluster between +20% and +30%. * **Data Points (Approximate):** * LLaMA3.1-8B (Teal ○): ~ +11% * LLaMA3.1-70B (Light Green ○): ~ +25% * Qwen2.5-7B (Purple ○): ~ +26% * Qwen2.5-72B (Red ○): ~ +21% * Claude3.5 (Blue ○): ~ +27% * GPT-3.5 (Orange ⬠): ~ +25% * GPT-4o (Green ⬠): ~ +30% * DeepSeek-V3 (Purple ◇): ~ +15% * QWQ-32B (Pink ◇): ~ +20% **7. HumanEval** * **Trend:** Only two data points are visible, both showing positive Δ. * **Data Points (Approximate):** * GPT-4o (Green ⬠): ~ +20% * QWQ-32B (Pink ◇): ~ +19% ### Key Observations 1. **Dataset Difficulty:** GPQA elicits the largest positive performance changes (Δ) across the board, suggesting it may be a benchmark where recent model improvements are most pronounced. Conversely, HotpotQA and CS-QA show more modest or even negative changes for some models. 2. **Model Performance:** GPT-4o (Green ⬠) and QWQ-32B (Pink ◇) are frequently among the top performers, often showing Δ > +15%. LLaMA3.1-8B (Teal ○) is the most consistent negative outlier, showing negative Δ in 5 out of 6 datasets where it appears. 3. **Model Type Trends:** "Reasoning LLMs" (Diamonds: QWQ-32B, DeepSeek-V3) generally show strong positive Δ, particularly on AQUA, GSM8K, and GPQA. "Closed LLMs" (Pentagons: GPT-3.5, GPT-4o) also show strong positive trends, especially on GPQA. 4. **Outliers:** The most significant negative outlier is LLaMA3.1-8B on HotpotQA (~ -14%). The most significant positive outlier is GPT-4o on GPQA (~ +30%). ### Interpretation This chart visualizes the **relative improvement or degradation (Δ)** of various LLMs compared to a baseline (likely a previous model version or a standard prompting method) across diverse reasoning and knowledge benchmarks. * **What the data suggests:** The positive Δ values for most models on most datasets indicate that the evaluated models generally outperform the baseline. The magnitude of improvement is highly dataset-dependent, with complex reasoning tasks (GPQA, MATH) showing larger gains than others (CS-QA). * **Relationships:** The plot allows for a direct comparison of model families (e.g., LLaMA vs. Qwen vs. GPT) and types (Open vs. Closed vs. Reasoning) on the same tasks. It highlights that model size alone (e.g., LLaMA3.1-70B vs. 8B) is not the sole determinant of performance gain, as architecture and training (implied by model family) play a crucial role. * **Anomalies & Insights:** The consistently negative Δ for LLaMA3.1-8B suggests it may be a weaker baseline or that the specific evaluation setup disadvantaged it. The exceptional performance of all models on GPQA is notable and may warrant investigation into the nature of that benchmark—whether it aligns particularly well with current model capabilities or training data. The absence of data for many models on HumanEval limits conclusions for that specific coding task. </details> Figure 12: Performance Gains for Self-Refine. <details> <summary>x13.png Details</summary>  ### Visual Description ## Scatter Plot: Performance Change (Δ%) of Various LLMs Across Multiple Datasets ### Overview The image is a scatter plot comparing the performance change (Δ%) of multiple Large Language Models (LLMs) across seven different benchmark datasets. The plot uses distinct symbols and colors to represent each model, with a horizontal baseline at Δ=0 indicating no change in performance. The data points are grouped by dataset along the x-axis. ### Components/Axes * **X-axis (Categorical):** Labeled "Dataset". The seven datasets listed from left to right are: 1. HotpotQA 2. CS-QA 3. AQUA 4. GSM8K 5. MATH 6. GPQA 7. HumanEval * **Y-axis (Numerical):** Labeled "Δ(%)". The scale ranges from -30 to +30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Legend (Top-Right):** Contains 12 entries, each pairing a model name with a unique symbol/color: * LLaMA3.1-8B: Light blue circle * LLaMA3.1-70B: Yellow circle * Qwen2.5-7B: Light purple circle * Qwen2.5-72B: Red circle * Claude3.5: Blue pentagon * GPT-3.5: Orange pentagon * GPT-4o: Green pentagon * QWQ-32B: Purple diamond * DeepSeek-V3: Pink diamond * Open LLM: Circle symbol (category) * Close LLM: Pentagon symbol (category) * Reasoning LLM: Diamond symbol (category) * **Baseline:** A horizontal, dashed, light green line at y=0, labeled "Baseline (Δ=0)". ### Detailed Analysis Performance change (Δ%) is plotted for each model on each dataset. Values are approximate based on visual positioning. **1. HotpotQA:** * All models show negative Δ% (performance decrease). * Values range from approximately -5% (LLaMA3.1-8B) to -18% (GPT-4o). * Cluster of points between -10% and -15% includes LLaMA3.1-70B, Qwen2.5-7B, Qwen2.5-72B, GPT-3.5, and Claude3.5. **2. CS-QA:** * All models show negative Δ%. * Values range from approximately -12% (Claude3.5) to -25% (LLaMA3.1-8B). * Most models cluster between -15% and -20%. **3. AQUA:** * All models show negative Δ%. * Significant outlier: LLaMA3.1-70B shows the largest decrease at approximately -31%. * Other models range from approximately -1% (Qwen2.5-7B) to -18% (Qwen2.5-72B). **4. GSM8K:** * All models show negative Δ%. * Values range from approximately -6% (GPT-3.5) to -29% (Qwen2.5-7B). * DeepSeek-V3 shows a decrease of approximately -25%. **5. MATH:** * All models show negative Δ%. * Values range from approximately -2% (Claude3.5) to -21% (DeepSeek-V3). * Most models cluster between -10% and -15%. **6. GPQA:** * Mixed performance. Some models show positive Δ%, others negative. * **Positive Δ%:** Claude3.5 (~+10%), DeepSeek-V3 (~+11%), Qwen2.5-72B (~+1%). * **Near Baseline:** LLaMA3.1-8B (~-3%), GPT-3.5 (~0%). * **Negative Δ%:** LLaMA3.1-70B (~-5%), GPT-4o (~-6%), QWQ-32B (~-2%). **7. HumanEval:** * Data is sparse. Only two data points are clearly visible. * Qwen2.5-72B: Approximately -2%. * DeepSeek-V3: Approximately -2%. * Other models are not plotted for this dataset. ### Key Observations 1. **Predominant Negative Trend:** Across the first five datasets (HotpotQA through MATH), every single model shows a negative performance change (Δ < 0), indicating a consistent decrease in performance on these benchmarks. 2. **GPQA as an Exception:** GPQA is the only dataset where multiple models (Claude3.5, DeepSeek-V3, Qwen2.5-72B) show a positive performance change. 3. **Model Performance Variability:** * **LLaMA3.1-70B** shows extreme variability, with the largest decrease on AQUA (~-31%) but a relatively smaller decrease on GPQA (~-5%). * **Claude3.5** and **DeepSeek-V3** (both "Reasoning LLMs" per the legend) are the top performers on GPQA, showing the only significant positive gains. * **Qwen2.5-7B** shows one of the largest decreases on GSM8K (~-29%). 4. **Symbol/Category Correlation:** The legend categorizes models by symbol shape: circles for "Open LLM", pentagons for "Close LLM", and diamonds for "Reasoning LLM". This categorization is visually consistent throughout the plot. ### Interpretation This chart likely illustrates the performance delta of various LLMs when subjected to a specific intervention, condition, or evaluation method compared to a baseline. The consistent negative Δ% across most datasets suggests the intervention generally hinders performance on tasks like question answering (HotpotQA, CS-QA, AQUA) and mathematical reasoning (GSM8K, MATH). The notable exception is the GPQA dataset, where "Reasoning LLMs" (Claude3.5, DeepSeek-V3) and one other model (Qwen2.5-72B) show improved performance. This suggests the intervention or evaluation condition may specifically benefit certain model architectures or training paradigms on this particular type of task (GPQA is a graduate-level science QA benchmark). The extreme negative outlier for LLaMA3.1-70B on AQUA indicates a severe and specific failure mode for that model under the tested condition. The sparse data for HumanEval limits conclusions but shows minimal negative impact for the two models plotted. In summary, the data demonstrates that the evaluated condition has a broadly negative impact on LLM performance across diverse benchmarks, with a specific, positive exception for a subset of models on the GPQA dataset. This highlights the importance of evaluating model robustness across multiple domains, as performance impacts can be highly dataset- and model-specific. </details> Figure 13: Performance Gains for Reflexion-v1. <details> <summary>x14.png Details</summary>  ### Visual Description ## Scatter Plot: Language Model Performance Comparison Across Datasets ### Overview This image is a scatter plot comparing the performance change (Δ%) of various large language models (LLMs) across seven different benchmark datasets. The chart evaluates both open-source and closed-source models, as well as specialized reasoning models, against a baseline performance level. ### Components/Axes * **Chart Type:** Scatter plot with categorical x-axis and numerical y-axis. * **X-Axis (Horizontal):** Labeled "Dataset". It lists seven benchmark categories from left to right: 1. HotpotQA 2. CS-QA 3. AQUA 4. GSM8K 5. MATH 6. GPQA 7. HumanEval * **Y-Axis (Vertical):** Labeled "Δ (%)". It represents the percentage change in performance, with a scale ranging from -30% to +30% in increments of 10%. A horizontal dashed green line at Δ=0 is labeled "Baseline (Δ=0)". * **Legend (Right Side):** The legend is positioned in the top-right quadrant of the chart area. It maps marker colors and shapes to specific models and model categories. * **Models (by color/shape):** * LLaMA3.1-8B (Teal circle) * LLaMA3.1-70B (Light green circle) * Qwen2.5-7B (Purple circle) * Qwen2.5-72B (Red circle) * Claude3.5 (Blue pentagon) * GPT-3.5 (Orange pentagon) * GPT-4o (Green pentagon) * QWQ-32B (Pink diamond) * DeepSeek-V3 (Purple diamond) * **Model Categories (by shape):** * Open LLM (Circle) * Close LLM (Pentagon) * Reasoning LLM (Diamond) ### Detailed Analysis The chart is divided into seven vertical sections by dashed grey lines, one for each dataset. Below are the approximate Δ(%) values for each model within each dataset, based on visual estimation of marker position. **1. HotpotQA** * **Trend:** Most models show positive Δ(%) values, indicating performance above the baseline. * **Data Points (Approximate):** * DeepSeek-V3 (Purple diamond): ~+21% (Highest) * LLaMA3.1-8B (Teal circle): ~+13% * Qwen2.5-7B (Purple circle): ~+13% * GPT-3.5 (Orange pentagon): ~+7% * LLaMA3.1-70B (Light green circle): ~+7% * Claude3.5 (Blue pentagon): ~+5% * GPT-4o (Green pentagon): ~+2% * Qwen2.5-72B (Red circle): ~0% * QWQ-32B (Pink diamond): ~-4% **2. CS-QA** * **Trend:** Mixed performance, with several models near or below the baseline. * **Data Points (Approximate):** * Claude3.5 (Blue pentagon): ~+11% (Highest) * GPT-3.5 (Orange pentagon): ~+5% * LLaMA3.1-70B (Light green circle): ~+4% * LLaMA3.1-8B (Teal circle): ~+2% * Qwen2.5-72B (Red circle): ~0% * GPT-4o (Green pentagon): ~-1% * QWQ-32B (Pink diamond): ~-1% * Qwen2.5-7B (Purple circle): ~-7% (Lowest) **3. AQUA** * **Trend:** High variance, with two models showing very high positive Δ(%). * **Data Points (Approximate):** * GPT-3.5 (Orange pentagon): ~+21% (Highest, tied) * QWQ-32B (Pink diamond): ~+21% (Highest, tied) * GPT-4o (Green pentagon): ~+11% * Claude3.5 (Blue pentagon): ~+6% * Qwen2.5-7B (Purple circle): ~+3% * LLaMA3.1-70B (Light green circle): ~0% * Qwen2.5-72B (Red circle): ~0% * DeepSeek-V3 (Purple diamond): ~-6% **4. GSM8K** * **Trend:** Most models cluster near the baseline, with one significant outlier. * **Data Points (Approximate):** * QWQ-32B (Pink diamond): ~+20% (Highest) * GPT-3.5 (Orange pentagon): ~+13% * GPT-4o (Green pentagon): ~+3% * Qwen2.5-72B (Red circle): ~+1% * Claude3.5 (Blue pentagon): ~0% * Qwen2.5-7B (Purple circle): ~-2% * LLaMA3.1-8B (Teal circle): ~-2% * LLaMA3.1-70B (Light green circle): ~-9% (Lowest) **5. MATH** * **Trend:** Generally positive performance, with models clustered in the +5% to +15% range. * **Data Points (Approximate):** * Qwen2.5-72B (Red circle): ~+11% (Highest) * Claude3.5 (Blue pentagon): ~+10% * GPT-4o (Green pentagon): ~+9% * GPT-3.5 (Orange pentagon): ~+8% * LLaMA3.1-70B (Light green circle): ~+6% * LLaMA3.1-8B (Teal circle): ~-6% * QWQ-32B (Pink diamond): ~-1% * DeepSeek-V3 (Purple diamond): ~-2% **6. GPQA** * **Trend:** High variance, with two models showing strong positive Δ(%). * **Data Points (Approximate):** * Claude3.5 (Blue pentagon): ~+21% (Highest) * Qwen2.5-7B (Purple circle): ~+18% * LLaMA3.1-8B (Teal circle): ~+12% * LLaMA3.1-70B (Light green circle): ~+9% * Qwen2.5-72B (Red circle): ~+5% * GPT-3.5 (Orange pentagon): ~+3% * GPT-4o (Green pentagon): ~-3% * QWQ-32B (Pink diamond): ~-4% **7. HumanEval** * **Trend:** Only two data points are visible, both showing negative Δ(%). * **Data Points (Approximate):** * QWQ-32B (Pink diamond): ~+12% * DeepSeek-V3 (Purple diamond): ~-4% ### Key Observations 1. **Model-Specific Strengths:** No single model dominates across all datasets. For example, DeepSeek-V3 excels in HotpotQA (~+21%) but performs poorly in AQUA (~-6%). Claude3.5 shows strong, consistent performance in CS-QA and GPQA. 2. **Dataset Difficulty:** The MATH dataset shows the most consistently positive Δ(%) values across models, suggesting models generally perform better relative to the baseline on this task. In contrast, CS-QA and GSM8K show more models at or below the baseline. 3. **Reasoning Model Performance:** The "Reasoning LLM" category (diamonds) shows extreme variance. QWQ-32B is the top performer in AQUA and GSM8K but is among the lowest in HotpotQA and GPQA. 4. **Open vs. Closed Models:** There is no clear, consistent performance gap between "Open LLM" (circles) and "Close LLM" (pentagons) across all tasks. Their relative performance is highly dataset-dependent. ### Interpretation This chart demonstrates the **task-specific nature of LLM capabilities**. The significant variance in Δ(%) for a given model across different datasets indicates that benchmark performance is not monolithic; a model's architecture, training data, and fine-tuning create specialized strengths and weaknesses. The data suggests that: * **Evaluation is Multidimensional:** Choosing an LLM for a specific application requires benchmarking on tasks relevant to that domain (e.g., a model strong in mathematical reasoning (MATH, GSM8K) may not be the best for complex question answering (HotpotQA, GPQA)). * **The "Best" Model is Contextual:** The absence of a universally superior model implies that the field is still evolving, with different approaches (open vs. closed, general vs. reasoning-specialized) yielding different trade-offs. * **Baseline Matters:** The Δ(%) metric highlights relative improvement or regression against a fixed baseline, which is crucial for understanding progress but doesn't convey absolute performance scores. **Notable Anomaly:** The HumanEval column contains only two data points (QWQ-32B and DeepSeek-V3), while all other datasets have eight or nine. This suggests either missing data for other models on this coding benchmark or a focused comparison for this specific task. </details> Figure 14: Performance Gains for Reflexion-v2. <details> <summary>x15.png Details</summary>  ### Visual Description ## Scatter Plot: Model Performance Delta (Δ%) Across Datasets ### Overview The image is a scatter plot comparing the performance change (Δ%) of various large language models (LLMs) across seven different benchmark datasets. The plot uses distinct markers and colors to represent different models and model types (Open, Closed, Reasoning). A baseline at Δ=0 is indicated. ### Components/Axes * **X-Axis (Categorical):** Labeled "Dataset". Seven datasets are listed from left to right, separated by vertical dashed gray lines: 1. HotpotQA 2. CS-QA 3. AQUA 4. GSM8K 5. MATH 6. GPQA 7. HumanEval * **Y-Axis (Numerical):** Labeled "Δ(%)". The scale ranges from -30 to 30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Baseline:** A horizontal dashed green line at y=0, labeled "Baseline (Δ=0)" in the legend. * **Legend (Top-Right Corner):** Contains two sections. * **Models (by color):** * LLaMA3.1-8B (Teal circle) * LLaMA3.1-70B (Light green circle) * Qwen2.5-7B (Light purple circle) * Qwen2.5-72B (Red circle) * Claude3.5 (Blue pentagon) * GPT-3.5 (Orange pentagon) * GPT-4o (Green pentagon) * QWQ-32B (Purple diamond) * DeepSeek-V3 (Pink diamond) * **Model Types (by shape):** * Open LLM (Circle) * Close LLM (Pentagon) * Reasoning LLM (Diamond) ### Detailed Analysis The plot shows the Δ(%) for each model on each dataset. Values are approximate based on visual positioning relative to the y-axis. **1. HotpotQA:** * LLaMA3.1-8B (Teal circle): ~ +5% * LLaMA3.1-70B (Light green circle): ~ +9% * Qwen2.5-7B (Light purple circle): ~ -2% * Qwen2.5-72B (Red circle): ~ +4% * Claude3.5 (Blue pentagon): ~ -1% * GPT-3.5 (Orange pentagon): ~ 0% * GPT-4o (Green pentagon): ~ +2% * QWQ-32B (Purple diamond): ~ 0% * DeepSeek-V3 (Pink diamond): ~ +26% (Significant outlier) **2. CS-QA:** * LLaMA3.1-8B (Teal circle): ~ -1% * LLaMA3.1-70B (Light green circle): ~ -1% * Qwen2.5-7B (Light purple circle): ~ 0% * Qwen2.5-72B (Red circle): ~ +4% * Claude3.5 (Blue pentagon): ~ +1% * GPT-3.5 (Orange pentagon): ~ -3% * GPT-4o (Green pentagon): ~ +4% * QWQ-32B (Purple diamond): ~ +1% * DeepSeek-V3 (Pink diamond): ~ +2% **3. AQUA:** * LLaMA3.1-8B (Teal circle): ~ +3% * LLaMA3.1-70B (Light green circle): ~ +2% * Qwen2.5-7B (Light purple circle): ~ -2% * Qwen2.5-72B (Red circle): ~ +1% * Claude3.5 (Blue pentagon): ~ +7% * GPT-3.5 (Orange pentagon): ~ +7% * GPT-4o (Green pentagon): ~ +16% * QWQ-32B (Purple diamond): ~ 0% * DeepSeek-V3 (Pink diamond): ~ +20% **4. GSM8K:** * LLaMA3.1-8B (Teal circle): ~ -2% * LLaMA3.1-70B (Light green circle): ~ -1% * Qwen2.5-7B (Light purple circle): ~ -4% * Qwen2.5-72B (Red circle): ~ -4% * Claude3.5 (Blue pentagon): ~ -1% * GPT-3.5 (Orange pentagon): ~ +11% * GPT-4o (Green pentagon): ~ +5% * QWQ-32B (Purple diamond): ~ 0% * DeepSeek-V3 (Pink diamond): ~ +23% **5. MATH:** * LLaMA3.1-8B (Teal circle): ~ 0% * LLaMA3.1-70B (Light green circle): ~ +10% * Qwen2.5-7B (Light purple circle): ~ +5% * Qwen2.5-72B (Red circle): ~ +7% * Claude3.5 (Blue pentagon): ~ +11% * GPT-3.5 (Orange pentagon): ~ +3% * GPT-4o (Green pentagon): ~ +12% * QWQ-32B (Purple diamond): ~ +9% * DeepSeek-V3 (Pink diamond): ~ +2% **6. GPQA:** * LLaMA3.1-8B (Teal circle): ~ +25% * LLaMA3.1-70B (Light green circle): ~ +20% * Qwen2.5-7B (Light purple circle): ~ +27% * Qwen2.5-72B (Red circle): ~ +30% (Highest value in plot) * Claude3.5 (Blue pentagon): ~ +18% * GPT-3.5 (Orange pentagon): ~ +1% * GPT-4o (Green pentagon): ~ +13% * QWQ-32B (Purple diamond): ~ +21% * DeepSeek-V3 (Pink diamond): ~ +9% **7. HumanEval:** * LLaMA3.1-8B (Teal circle): ~ -6% * LLaMA3.1-70B (Light green circle): ~ +17% * Qwen2.5-7B (Light purple circle): ~ +7% * Qwen2.5-72B (Red circle): ~ +2% * Claude3.5 (Blue pentagon): ~ +1% * GPT-3.5 (Orange pentagon): ~ -1% * GPT-4o (Green pentagon): ~ +17% * QWQ-32B (Purple diamond): ~ +15% * DeepSeek-V3 (Pink diamond): ~ -9% (Notable negative outlier) ### Key Observations 1. **GPQA Shows Largest Gains:** All models show a positive Δ on the GPQA dataset, with most clustering between +10% and +30%. This suggests a significant performance improvement for all evaluated models on this specific benchmark. 2. **DeepSeek-V3 Variability:** The DeepSeek-V3 model (pink diamond) shows extreme variability. It has the highest single Δ on HotpotQA (~+26%) and a strong positive on GSM8K (~+23%), but the most negative Δ on HumanEval (~-9%). 3. **Model Type Trends:** "Reasoning LLMs" (diamonds) are not uniformly superior. While DeepSeek-V3 and QWQ-32B show high peaks, they also show dips. "Close LLMs" (pentagons) like GPT-4o and Claude3.5 show consistently moderate to high positive Δs across most datasets. 4. **Dataset Difficulty Signal:** Datasets like CS-QA and AQUA show Δ values clustered closer to zero for many models, suggesting less performance change or a harder baseline. In contrast, GPQA and GSM8K show wider spreads and higher positive values. 5. **HumanEval Divergence:** Performance on HumanEval is highly split. Some models (LLaMA3.1-70B, GPT-4o) show large gains (~+17%), while others (LLaMA3.1-8B, DeepSeek-V3) show losses. ### Interpretation This chart likely visualizes the performance delta (Δ) of various LLMs compared to a previous version or a baseline model on specific reasoning and knowledge benchmarks. The data suggests: * **Benchmark-Specific Progress:** The large positive Δs on GPQA and GSM8K indicate that recent model iterations or specific model types have made substantial progress on these quantitative and scientific reasoning tasks. * **Inconsistent Gains:** Performance improvement is not uniform. A model excelling on one type of task (e.g., DeepSeek-V3 on HotpotQA) may underperform on another (e.g., HumanEval code generation), highlighting the challenge of creating general-purpose models. * **The "Reasoning LLM" Category is Not Monolithic:** The diamond-shaped markers do not cluster together, indicating that being categorized as a "Reasoning LLM" does not guarantee a specific performance profile. Architectural or training differences within this category lead to divergent outcomes. * **Scale vs. Specialization:** Larger models (e.g., LLaMA3.1-70B) generally show positive Δs, but smaller specialized models or different architectures can achieve higher peaks on specific tasks, as seen with Qwen2.5-72B on GPQA. The chart effectively communicates that LLM advancement is multifaceted, with gains concentrated in certain domains and significant variability between models, even within the same size or type category. </details> Figure 15: Performance Gains for RARR. <details> <summary>x16.png Details</summary>  ### Visual Description \n ## Scatter Plot: LLM Performance Delta Across Datasets ### Overview The image is a scatter plot comparing the performance change (Δ, in percentage) of various Large Language Models (LLMs) across seven different benchmark datasets. Each data point represents a specific model's performance delta relative to a baseline (Δ=0). The models are categorized as "Open LLM," "Close LLM," or "Reasoning LLM" using distinct marker shapes. ### Components/Axes * **Chart Type:** Scatter plot with categorical x-axis. * **X-Axis (Horizontal):** Labeled "Dataset." It lists seven benchmark categories, separated by vertical dashed lines. From left to right: 1. HotpotQA 2. CS-QA 3. AQUA 4. GSM8K 5. MATH 6. GPQA 7. HumanEval * **Y-Axis (Vertical):** Labeled "Δ (%)". It represents the percentage change in performance. The scale runs from -30 to 30, with major tick marks at intervals of 10 (-30, -20, -10, 0, 10, 20, 30). * **Baseline:** A horizontal, light green dashed line at y=0, labeled "Baseline (Δ=0)" in the legend. * **Legend (Right Side):** Positioned to the right of the plot area. It maps model names to specific marker colors and shapes, and defines the marker categories. * **Models & Colors:** * LLaMA3.1-8B: Teal circle * LLaMA3.1-70B: Light yellow circle * Qwen2.5-7B: Light purple circle * Qwen2.5-72B: Salmon/red circle * Claude3.5: Blue pentagon * GPT-3.5: Orange pentagon * GPT-4o: Green pentagon * QWQ-32B: Purple diamond * DeepSeek-V3: Pink diamond * **Marker Categories:** * Open LLM: Circle (○) * Close LLM: Pentagon (⬠) * Reasoning LLM: Diamond (◇) ### Detailed Analysis Data points are grouped vertically above each dataset label. Values are approximate, estimated from the y-axis position. **1. HotpotQA** * **Above Baseline (Δ > 0):** LLaMA3.1-8B (~1%), LLaMA3.1-70B (~1%), Qwen2.5-72B (~1%), GPT-4o (~2%), QWQ-32B (~4%). * **Below Baseline (Δ < 0):** Qwen2.5-7B (~-2%), Claude3.5 (~-7%), GPT-3.5 (~-7%), DeepSeek-V3 (~-4%). **2. CS-QA** * **Above Baseline:** Qwen2.5-7B (~10%), Qwen2.5-72B (~2%), Claude3.5 (~-3% *Note: This point is slightly below 0*), GPT-3.5 (~-1%), GPT-4o (~0%), QWQ-32B (~3%). * **Below Baseline:** LLaMA3.1-8B (~0%), LLaMA3.1-70B (~-1%), DeepSeek-V3 (~-6%). **3. AQUA** * **Above Baseline:** LLaMA3.1-70B (~5%), Qwen2.5-7B (~18%), Qwen2.5-72B (~7%), Claude3.5 (~9%), GPT-3.5 (~10%), GPT-4o (~12%). * **Below Baseline:** LLaMA3.1-8B (~1%), QWQ-32B (~-1%), DeepSeek-V3 (~3% *Note: This point is slightly above 0*). **4. GSM8K** * **Above Baseline:** Qwen2.5-7B (~5%), Qwen2.5-72B (~0%), Claude3.5 (~9%), GPT-3.5 (~8%), GPT-4o (~10%). * **Below Baseline:** LLaMA3.1-8B (~-9%), LLaMA3.1-70B (~-4%), QWQ-32B (~-3%), DeepSeek-V3 (~-1%). **5. MATH** * **Above Baseline:** LLaMA3.1-70B (~8%), Qwen2.5-7B (~11%), Qwen2.5-72B (~14%), Claude3.5 (~4%), GPT-3.5 (~4%), GPT-4o (~4%). * **Below Baseline:** LLaMA3.1-8B (~-3%), QWQ-32B (~-8%), DeepSeek-V3 (~-1%). **6. GPQA** * **Above Baseline:** LLaMA3.1-70B (~11%), Qwen2.5-7B (~16%), Qwen2.5-72B (~6%), Claude3.5 (~5%), GPT-3.5 (~11%), GPT-4o (~3%), QWQ-32B (~5%), DeepSeek-V3 (~5%). * **Below Baseline:** LLaMA3.1-8B (~4% *Note: This point is above 0*). **7. HumanEval** * **Above Baseline:** LLaMA3.1-70B (~4%), Qwen2.5-7B (~-1% *Note: This point is slightly below 0*), Claude3.5 (~0%), GPT-3.5 (~-2%), GPT-4o (~9%), QWQ-32B (~3%), DeepSeek-V3 (~2%). * **Below Baseline:** LLaMA3.1-8B (~-11%), Qwen2.5-72B (~-1%). ### Key Observations 1. **High Variance in AQUA & GPQA:** The AQUA and GPQA datasets show the widest spread of performance deltas, with several models achieving gains above +10% and others falling below the baseline. 2. **Consistently Strong on MATH:** Most models show a positive performance delta on the MATH dataset, with Qwen2.5-72B showing the highest gain (~+14%). 3. **LLaMA3.1-8B Struggles:** The LLaMA3.1-8B model (teal circle) frequently appears below the baseline, most notably on GSM8K (~-9%) and HumanEval (~-11%). 4. **Qwen2.5-7B's Peak:** The Qwen2.5-7B model (light purple circle) achieves the single highest observed delta on the chart, at approximately +18% on the AQUA dataset. 5. **Reasoning LLMs (Diamonds):** The "Reasoning LLM" category (QWQ-32B, DeepSeek-V3) shows mixed results. QWQ-32B has a notable positive spike on HotpotQA, while DeepSeek-V3 is often near or slightly below the baseline, except for a positive showing on GPQA. 6. **Close LLMs (Pentagons):** The "Close LLM" models (Claude3.5, GPT-3.5, GPT-4o) generally cluster together within each dataset, often showing positive deltas, particularly on AQUA and MATH. ### Interpretation This chart visualizes a comparative benchmark study, likely measuring the improvement (or degradation) of various LLMs when using a specific technique, prompting method, or model variant compared to a standard baseline. The "Δ (%)" suggests a relative performance metric. * **Dataset Sensitivity:** Model performance is highly dataset-dependent. A model excelling in one domain (e.g., Qwen2.5-7B on AQUA) may not lead in another (e.g., HumanEval). This underscores the importance of multi-faceted evaluation. * **Model Size vs. Performance:** Larger models (e.g., LLaMA3.1-70B, Qwen2.5-72B) do not universally outperform smaller ones (e.g., Qwen2.5-7B) across all tasks, indicating that architecture, training data, or task alignment play critical roles. * **Specialization:** The strong performance of several models on the MATH dataset suggests the evaluated technique or the models themselves are particularly effective for mathematical reasoning tasks. Conversely, the mixed results on code (HumanEval) and complex reasoning (HotpotQA, GPQA) indicate these remain challenging areas. * **The "Reasoning LLM" Category:** The inclusion of a specific "Reasoning LLM" category implies these models (QWQ-32B, DeepSeek-V3) may have been designed or fine-tuned with a focus on logical inference. Their variable performance suggests that "reasoning" capability is not monolithic and manifests differently across benchmark types. **Language Note:** All text in the image is in English. </details> Figure 16: Performance Gains for RATT. ## Appendix F Contrast Experiments for Diverse LLMs In this section, we compare the performance of various models across multiple datasets using different methods. The HotpotQA, GSM8K, and GPQA datasets are selected to represent commonsense reasoning, mathematical reasoning, and complex reasoning, respectively. Each figure depicts the performance of 9 LLMs on the three datasets under a specific method. For each figure, the first subgraph compares the performance of open-source LLMs, identifying the best-performing one. The second subgraph evaluates the best open-source LLM against closed-source LLMs, and the third subgraph summarizes the performance of the best open-source LLM, the best closed-source model, and reasoning models. The performance of different models across the three datasets using various methods is summarized in Figures 17 to 25. For instance, Figure 17 represents results for Base method, while other figures illustrate performance for methods such as CoT, RCI, Cove, Self-Refine, Reflexion-v1, Reflexion-v2, RARR, and RATT, respectively. Among the evaluated LLMs, GPT-4o and Qwen2.5-72B-Instrcut consistently demonstrate superior performance as the best open-source LLM and closed-source LLM, respectively, across most methods. Based on these results, it is evident that closed-source LLMs generally outperform open-source LLMs. Furthermore, reasoning LLMs (e.g., DeepSeek-V3) exhibit the best overall performance, excelling particularly in tasks requiring complex reasoning capabilities, as demonstrated by their consistent dominance across all datasets and methods. <details> <summary>x17.png Details</summary>  ### Visual Description ## [Grouped Bar Charts]: LLM Performance Comparison Across Datasets ### Overview The image displays three side-by-side grouped bar charts comparing the performance of various Large Language Models (LLMs) on three benchmark datasets: HotpotQA, GSM8k, and GPQA. The charts are titled "Comparison of Open-source LLMs," "Comparison of Closed-source LLMs," and "Instruction-based vs Reasoning LLMs." Each chart uses a consistent y-axis labeled "Scores" ranging from 0 to 100. ### Components/Axes **Common Elements:** * **Y-Axis:** Labeled "Scores" with major tick marks at 0, 20, 40, 60, 80, and 100. * **X-Axis:** Labeled "Datasets" with three categories: "HotpotQA", "GSM8k", and "GPQA". * **Chart Titles:** Located at the top of each respective chart. * **Legends:** Positioned in the top-right corner of each chart area. **Chart 1: Comparison of Open-source LLMs** * **Legend (Top-Right):** * Teal bar: `LLaMA3.1-8B` * Light yellow bar: `LLaMA3.1-70B` * Light purple bar: `Qwen2.5-7B` * Red bar: `Qwen2.5-72B` **Chart 2: Comparison of Closed-source LLMs** * **Legend (Top-Right):** * Red bar: `Qwen2.5-72B` * Blue bar: `Claude3.5` * Orange bar: `GPT-3.5` * Green bar: `GPT-4o` **Chart 3: Instruction-based vs Reasoning LLMs** * **Legend (Top-Right):** * Red bar: `Qwen2.5-72B` * Green bar: `GPT-4o` * Light pink bar: `QWO-32B` * Purple bar: `DeepSeek-V3` ### Detailed Analysis **Chart 1: Comparison of Open-source LLMs** * **HotpotQA:** All models score between ~75 and ~85. `LLaMA3.1-70B` (light yellow) appears highest (~85), followed closely by `Qwen2.5-72B` (red, ~83), `LLaMA3.1-8B` (teal, ~78), and `Qwen2.5-7B` (light purple, ~75). * **GSM8k:** Scores are generally higher. `LLaMA3.1-70B` (light yellow) is the highest (~92), followed by `Qwen2.5-72B` (red, ~88), `Qwen2.5-7B` (light purple, ~85), and `LLaMA3.1-8B` (teal, ~82). * **GPQA:** All models show a significant performance drop. `LLaMA3.1-70B` (light yellow) is highest (~15), followed by `Qwen2.5-72B` (red, ~10), `LLaMA3.1-8B` (teal, ~8), and `Qwen2.5-7B` (light purple, ~5). **Chart 2: Comparison of Closed-source LLMs** * **HotpotQA:** `GPT-4o` (green) leads (~88), followed by `Claude3.5` (blue, ~85), `Qwen2.5-72B` (red, ~83), and `GPT-3.5` (orange, ~82). * **GSM8k:** `GPT-4o` (green) is highest (~95), followed by `Claude3.5` (blue, ~92), `Qwen2.5-72B` (red, ~88), and `GPT-3.5` (orange, ~80). * **GPQA:** Performance is low across the board. `Claude3.5` (blue) is highest (~32), followed by `GPT-3.5` (orange, ~28), `GPT-4o` (green, ~22), and `Qwen2.5-72B` (red, ~10). **Chart 3: Instruction-based vs Reasoning LLMs** * **HotpotQA:** `GPT-4o` (green) and `DeepSeek-V3` (purple) are tied for highest (~88), followed by `Qwen2.5-72B` (red, ~83), and `QWO-32B` (light pink, ~62). * **GSM8k:** `GPT-4o` (green) is highest (~95), followed by `Qwen2.5-72B` (red, ~88), `DeepSeek-V3` (purple, ~65), and `QWO-32B` (light pink, ~62). * **GPQA:** `DeepSeek-V3` (purple) is highest (~32), followed by `GPT-4o` (green, ~22), `QWO-32B` (light pink, ~10), and `Qwen2.5-72B` (red, ~10). ### Key Observations 1. **Dataset Difficulty:** All models, regardless of type (open-source, closed-source, instruction/reasoning), perform significantly worse on the GPQA dataset compared to HotpotQA and GSM8k. This suggests GPQA is a more challenging benchmark. 2. **Model Scaling:** Within the open-source models, the 70B/72B parameter models (`LLaMA3.1-70B`, `Qwen2.5-72B`) consistently outperform their smaller counterparts (`LLaMA3.1-8B`, `Qwen2.5-7B`) across all datasets. 3. **Top Performers:** `GPT-4o` (green) is a top performer in the closed-source and instruction/reasoning charts, especially on GSM8k. `LLaMA3.1-70B` (light yellow) is the top performer among the open-source models shown. 4. **Qwen2.5-72B as a Common Reference:** The `Qwen2.5-72B` model (red bar) appears in all three charts, allowing for a cross-comparison. Its performance is competitive with top open-source models and some closed-source models, but it is outperformed by `GPT-4o` and `Claude3.5` on most tasks. 5. **Specialization:** `DeepSeek-V3` (purple) shows a notable strength on the GPQA dataset relative to other models in its chart, suggesting potential specialization in that type of reasoning task. ### Interpretation This set of charts provides a snapshot of the competitive landscape among LLMs circa the data's collection (likely pre-May 2025). The data demonstrates several key trends: * **The Scaling Law in Action:** The clear performance advantage of larger models (70B/72B vs. 8B/7B) empirically supports the principle that model scale is a primary driver of capability on standard benchmarks. * **The "GPQA Gap":** The universal drop in scores on GPQA indicates this benchmark tests a dimension of reasoning or knowledge that is not as well-captured by the training or optimization of these models compared to HotpotQA (multi-hop reasoning) and GSM8k (mathematical reasoning). It highlights an area for future model improvement. * **The Closed-Source Edge:** While open-source models like `LLaMA3.1-70B` are highly capable, the top closed-source models (`GPT-4o`, `Claude3.5`) generally maintain a performance lead, particularly on the GSM8k math benchmark. This suggests potential advantages in training data, methodology, or compute resources. * **Task-Specific Strengths:** The variation in model rankings across datasets (e.g., `DeepSeek-V3` on GPQA, `GPT-4o` on GSM8k) implies that no single model is universally superior. Model selection should be guided by the specific task or domain, as different architectures and training approaches yield different strengths. **Language Note:** All text in the image is in English. </details> Figure 17: Performance Comparison of Base <details> <summary>x18.png Details</summary>  ### Visual Description ## Bar Charts: LLM Performance Comparison Across Datasets ### Overview The image displays three side-by-side grouped bar charts comparing the performance scores of various Large Language Models (LLMs) on three distinct datasets: HotpotQA, GSM8k, and GPQA. The charts are categorized by model type: open-source, closed-source, and instruction-based vs. reasoning-based. All charts share a common y-axis labeled "Scores" ranging from 0 to 100. ### Components/Axes * **Chart 1 (Left):** "Comparison of Open-source LLMs" * **X-axis (Datasets):** HotpotQA, GSM8k, GPQA. * **Y-axis:** Scores (0-100). * **Legend (Top-Right):** * Teal bar: LLaMA3.1-8B * Yellow bar: LLaMA3.1-70B * Light Purple bar: Qwen2.5-7B * Salmon/Red bar: Qwen2.5-72B * **Chart 2 (Center):** "Comparison of Closed-source LLMs" * **X-axis (Datasets):** HotpotQA, GSM8k, GPQA. * **Y-axis:** Scores (0-100). * **Legend (Top-Left):** * Salmon/Red bar: Qwen2.5-72B *(Note: This model also appears in the open-source chart)* * Blue bar: Claude3.5 * Orange bar: GPT-3.5 * Green bar: GPT-4o * **Chart 3 (Right):** "Instruction-based vs. Reasoning LLMs" * **X-axis (Datasets):** HotpotQA, GSM8k, GPQA. * **Y-axis:** Scores (0-100). * **Legend (Top-Right):** * Salmon/Red bar: Qwen2.5-72B * Light Green bar: GPT-4o * Pink bar: QWO-32B * Purple bar: DeepSeek-V3 ### Detailed Analysis **Chart 1: Open-source LLMs** * **HotpotQA:** LLaMA3.1-70B (Yellow) scores highest (~85), followed by Qwen2.5-72B (Salmon, ~75), LLaMA3.1-8B (Teal, ~70), and Qwen2.5-7B (Purple, ~68). * **GSM8k:** All models show significantly higher performance. Qwen2.5-72B (Salmon) leads (~95), closely followed by LLaMA3.1-70B (Yellow, ~92), Qwen2.5-7B (Purple, ~88), and LLaMA3.1-8B (Teal, ~78). * **GPQA:** Performance drops drastically for all models. LLaMA3.1-70B (Yellow) is highest (~22), followed by Qwen2.5-72B (Salmon, ~15), LLaMA3.1-8B (Teal, ~12), and Qwen2.5-7B (Purple, ~10). **Chart 2: Closed-source LLMs** * **HotpotQA:** Claude3.5 (Blue) and GPT-4o (Green) are nearly tied at the top (~90), with Qwen2.5-72B (Salmon, ~88) and GPT-3.5 (Orange, ~85) slightly behind. * **GSM8k:** All models perform exceptionally well, clustered near the top. GPT-4o (Green) appears marginally highest (~98), with the others (Claude3.5, GPT-3.5, Qwen2.5-72B) all above ~95. * **GPQA:** Scores are uniformly low. GPT-3.5 (Orange) is highest (~18), followed by Claude3.5 (Blue, ~17), GPT-4o (Green, ~15), and Qwen2.5-72B (Salmon, ~14). **Chart 3: Instruction-based vs. Reasoning LLMs** * **HotpotQA:** GPT-4o (Light Green) and Qwen2.5-72B (Salmon) are top (~90), followed by DeepSeek-V3 (Purple, ~88) and QWO-32B (Pink, ~65). * **GSM8k:** GPT-4o (Light Green) and Qwen2.5-72B (Salmon) again lead (~95), with DeepSeek-V3 (Purple, ~80) and QWO-32B (Pink, ~78) following. * **GPQA:** DeepSeek-V3 (Purple) shows a notable relative performance (~25), significantly higher than the others: Qwen2.5-72B (Salmon, ~15), GPT-4o (Light Green, ~12), and QWO-32B (Pink, ~10). ### Key Observations 1. **Dataset Difficulty:** GPQA is consistently the most challenging dataset for all model types, with scores rarely exceeding 25. GSM8k is the easiest, with top models scoring near 100. 2. **Model Scaling:** Within the open-source chart, the 70B/72B parameter models (LLaMA3.1-70B, Qwen2.5-72B) consistently outperform their smaller 7B/8B counterparts. 3. **Closed-source Dominance:** The closed-source models (Chart 2) generally achieve higher scores on HotpotQA and GSM8k compared to the open-source models (Chart 1). 4. **Notable Outlier:** In Chart 3, DeepSeek-V3 (Purple) demonstrates a uniquely strong performance on the difficult GPQA dataset compared to the other models in its group. 5. **Model Consistency:** Qwen2.5-72B (Salmon) appears in all three charts, serving as a common reference point. Its performance is strong but not always the absolute top in the closed-source category. ### Interpretation The data suggests a clear hierarchy of task difficulty, with GPQA representing a significant challenge frontier for current LLMs. The performance gap between open-source and closed-source models, while present, is less pronounced on the easiest task (GSM8k) and the hardest task (GPQA), indicating that scaling and architecture advantages of closed models are most evident on mid-difficulty reasoning tasks like HotpotQA. The inclusion of Qwen2.5-72B across all charts allows for a cross-category comparison. It performs competitively with top closed-source models, suggesting it is a high-performing open-source option. The standout performance of DeepSeek-V3 on GPQA in the third chart hints at potential architectural or training differences that may confer an advantage on that specific type of complex reasoning task, warranting further investigation into its design. Overall, the charts illustrate that model performance is highly dependent on both the model's scale/type and the specific nature of the evaluation dataset. </details> Figure 18: Performance Comparison of CoT <details> <summary>x19.png Details</summary>  ### Visual Description ## Comparative Analysis of Large Language Model (LLM) Performance ### Overview The image presents a composite of three bar charts comparing the performance of various Large Language Models (LLMs) across three benchmark datasets: HotpotQA, GSM8k, and GPQA. The charts are organized to compare different model categories: open-source, closed-source, and a final comparison between instruction-tuned and reasoning-focused models. ### Components/Axes * **Chart Structure:** Three separate bar charts arranged horizontally. * **Common X-Axis (All Charts):** Labeled "Datasets". The three categories are: 1. HotpotQA 2. GSM8k 3. GPQA * **Common Y-Axis (All Charts):** Labeled "Scores". The scale runs from 0 to 100 in increments of 20. * **Legends:** Each chart has its own legend positioned in the top-right corner of its respective plot area. ### Detailed Analysis #### Chart 1: Comparison of Open-source LLMs * **Legend (Top-Right):** * Teal Bar: LLaMA3.1-8B * Yellow Bar: LLaMA3.1-70B * Purple Bar: Qwen2.5-7B * Red Bar: Qwen2.5-72B * **Data Points & Trends:** * **HotpotQA:** Scores range from ~60 (LLaMA3.1-8B) to ~85 (Qwen2.5-72B). The trend is generally upward with model scale (8B < 70B < 72B), with Qwen2.5-7B (~65) performing slightly better than LLaMA3.1-8B. * **GSM8k:** This is the highest-performing category for all models. Scores are tightly clustered between ~70 (LLaMA3.1-8B) and ~95 (Qwen2.5-72B). All models show strong performance here. * **GPQA:** This is the lowest-performing category. Scores are significantly lower, ranging from ~15 (Qwen2.5-7B) to ~25 (LLaMA3.1-70B). The trend is less clear, with the largest model (Qwen2.5-72B) scoring ~15, similar to the smallest 7B model. #### Chart 2: Comparison of Closed-source LLMs * **Legend (Top-Right):** * Red Bar: Qwen2.5-72B (Note: This model appears in both open-source and closed-source charts, suggesting it may be available under different licensing or access models). * Blue Bar: Claude3.5 * Orange Bar: QWO-32B * Green Bar: GPT-4o * **Data Points & Trends:** * **HotpotQA:** All models perform strongly, with scores between ~85 (Qwen2.5-72B) and ~90 (GPT-4o). Performance is very consistent across models. * **GSM8k:** Again, the highest scores. All models are clustered near the top of the scale, between ~90 (Qwen2.5-72B) and ~95 (GPT-4o, Claude3.5). * **GPQA:** Scores drop dramatically for all models, ranging from ~15 (Qwen2.5-72B) to ~25 (QWO-32B). This dataset is clearly the most challenging. #### Chart 3: Instruction-based vs Reasoning LLMs * **Legend (Top-Right):** * Red Bar: Qwen2.5-72B * Green Bar: GPT-4o * Pink Bar: QWO-32B * Purple Bar: DeepSeek-V3 * **Data Points & Trends:** * **HotpotQA:** Scores are high, from ~80 (QWO-32B) to ~90 (GPT-4o, DeepSeek-V3). * **GSM8k:** Scores are very high and tightly grouped, from ~85 (QWO-32B) to ~95 (DeepSeek-V3). * **GPQA:** This chart shows the most significant variation. Qwen2.5-72B and QWO-32B score low (~15). GPT-4o scores moderately (~20). **DeepSeek-V3 is a clear outlier**, scoring approximately 30, which is notably higher than any other model on this dataset across all three charts. ### Key Observations 1. **Dataset Difficulty:** GPQA is consistently the most challenging benchmark for all models, with scores rarely exceeding 30. GSM8k is the easiest, with top models scoring near 95. 2. **Model Scale vs. Performance:** In the open-source chart, larger models (70B, 72B) generally outperform smaller ones (7B, 8B), but the advantage is not uniform across all tasks (e.g., GPQA). 3. **Closed-source Consistency:** Closed-source models (Claude3.5, GPT-4o) show very high and consistent performance on HotpotQA and GSM8k. 4. **Notable Outlier:** DeepSeek-V3 in the third chart demonstrates superior performance on the difficult GPQA benchmark compared to all other models shown. 5. **Model Overlap:** Qwen2.5-72B appears in all three charts, serving as a common reference point. Its performance is strong on GSM8k but weak on GPQA. ### Interpretation This visualization suggests that while modern LLMs have achieved near-human performance on certain reasoning tasks (GSM8k), complex, multi-step reasoning or knowledge-intensive tasks (GPQA) remain a significant challenge. The data implies that simply increasing model scale (from 7B to 72B parameters) improves performance on some benchmarks more than others. The standout performance of DeepSeek-V3 on GPQA indicates that specific architectural or training innovations can yield disproportionate gains on the hardest tasks, potentially more so than raw scale alone. The comparison between instruction-based and reasoning-focused models (Chart 3) highlights that model specialization or training methodology is a critical factor for performance on specific types of problems. </details> Figure 19: Performance Comparison of RCI <details> <summary>x20.png Details</summary>  ### Visual Description ## Bar Charts: LLM Performance Comparison Across Datasets ### Overview The image displays three horizontally arranged bar charts comparing the performance of various Large Language Models (LLMs) on three distinct evaluation datasets. The charts are titled "Comparison of Open-source LLMs," "Comparison of Closed-source LLMs," and "Instruction-based vs Reasoning-based LLMs." Each chart shares a common structure with the same three datasets on the x-axis and a "Scores" metric on the y-axis. ### Components/Axes * **Common Elements (All Charts):** * **X-axis Label:** "Datasets" * **X-axis Categories (from left to right):** "HotpotQA", "GSM8k", "GPQA" * **Y-axis Label:** "Scores" * **Y-axis Scale:** 0 to 100, with major tick marks at 0, 20, 40, 60, 80, 100. * **Chart Layout:** Three separate bar charts arranged side-by-side. Each chart has its own title and legend. * **Chart 1: Comparison of Open-source LLMs (Left)** * **Legend (Top-Right):** * Teal bar: `LLaMA3.1-8B` * Yellow bar: `LLaMA3.1-70B` * Purple bar: `Qwen2.5-7B` * Red bar: `Qwen2.5-72B` * **Chart 2: Comparison of Closed-source LLMs (Center)** * **Legend (Top-Right):** * Red bar: `Qwen2.5-72B` * Blue bar: `Claude3.5` * Orange bar: `GPT-3.5` * Green bar: `GPT-4o` * **Chart 3: Instruction-based vs Reasoning-based LLMs (Right)** * **Legend (Top-Right):** * Red bar: `Qwen2.5-72B` * Green bar: `GPT-4o` * Pink bar: `QWO-32B` * Purple bar: `DeepSeek-V3` ### Detailed Analysis **Chart 1: Open-source LLMs** * **HotpotQA:** `LLaMA3.1-70B` (yellow) and `Qwen2.5-72B` (red) are the top performers, both scoring approximately 90. `Qwen2.5-7B` (purple) scores ~75, and `LLaMA3.1-8B` (teal) scores ~65. * **GSM8k:** All models perform strongly. `Qwen2.5-7B` (purple) and `Qwen2.5-72B` (red) are highest at ~95. `LLaMA3.1-70B` (yellow) is ~90, and `LLaMA3.1-8B` (teal) is ~80. * **GPQA:** This is the most challenging dataset for these models. `LLaMA3.1-70B` (yellow) leads with a score of ~40. `Qwen2.5-7B` (purple) is ~35, `Qwen2.5-72B` (red) is ~30, and `LLaMA3.1-8B` (teal) is significantly lower at ~15. **Chart 2: Closed-source LLMs** * **HotpotQA:** Performance is tightly clustered. `Qwen2.5-72B` (red) and `GPT-4o` (green) are at ~90. `Claude3.5` (blue) is ~88, and `GPT-3.5` (orange) is ~85. * **GSM8k:** All models achieve very high scores. `Qwen2.5-72B` (red), `Claude3.5` (blue), and `GPT-4o` (green) are all at or near ~95. `GPT-3.5` (orange) is slightly lower at ~90. * **GPQA:** Scores are lower and more varied. `GPT-3.5` (orange) leads at ~50. `Claude3.5` (blue) and `GPT-4o` (green) are both ~45. `Qwen2.5-72B` (red) is the lowest at ~30. **Chart 3: Instruction-based vs Reasoning-based LLMs** * **HotpotQA:** `GPT-4o` (green) is the highest at ~92. `Qwen2.5-72B` (red) and `DeepSeek-V3` (purple) are both ~90. `QWO-32B` (pink) is notably lower at ~75. * **GSM8k:** All four models achieve near-perfect scores of ~95. * **GPQA:** `DeepSeek-V3` (purple) is the clear leader with a score of ~55. `GPT-4o` (green) is next at ~45. `Qwen2.5-72B` (red) is ~30, and `QWO-32B` (pink) is the lowest at ~25. ### Key Observations 1. **Dataset Difficulty:** GPQA is consistently the most difficult dataset across all model categories, yielding the lowest scores. GSM8k appears to be the easiest, with most models scoring above 80. 2. **Model Scale Impact (Chart 1):** For open-source models, the larger 70B/72B parameter models (`LLaMA3.1-70B`, `Qwen2.5-72B`) significantly outperform their smaller 7B/8B counterparts, especially on the challenging GPQA dataset. 3. **Performance Clustering:** Closed-source models (Chart 2) show very similar performance on HotpotQA and GSM8k, with differentiation only appearing on the harder GPQA task. 4. **Standout Performer:** `DeepSeek-V3` (Chart 3, purple) demonstrates a notable advantage on the GPQA dataset compared to other models in its comparison group, suggesting potential strength in complex reasoning tasks. 5. **Consistency:** `Qwen2.5-72B` (red) appears in all three charts as a benchmark, allowing for cross-category comparison. It is a top performer among open-source models and competitive with closed-source models on easier tasks but falls behind on GPQA. ### Interpretation The data suggests a clear hierarchy of task difficulty for current LLMs, with GPQA representing a frontier challenge. The significant performance gap between model sizes in the open-source chart underscores the importance of scale for capability, particularly for complex reasoning. The tight clustering of top closed-source models indicates a competitive plateau on standard benchmarks like HotpotQA and GSM8k. The most insightful finding is the divergent performance on GPQA. While most models struggle, `DeepSeek-V3`'s relatively high score hints at a possible architectural or training difference that confers an advantage on this specific type of reasoning task. This chart effectively isolates "reasoning" as a differentiating capability, moving beyond general instruction-following performance measured by the other datasets. The charts collectively argue that evaluating LLMs requires a suite of diverse benchmarks to reveal specific strengths and weaknesses. </details> Figure 20: Performance Comparison of CoVe <details> <summary>x21.png Details</summary>  ### Visual Description \n ## Comparative Analysis of Large Language Model (LLM) Performance Across Datasets ### Overview The image is a composite of three bar charts comparing the performance of various Large Language Models (LLMs) on three distinct benchmark datasets: HotpotQA, GSM8k, and GPQA. The charts are organized by model type: open-source, closed-source, and a comparison of instruction-based versus reasoning-based models. All charts share a common y-axis labeled "Scores" (0-100) and x-axis labeled "Datasets." ### Components/Axes * **Common Elements:** * **Y-Axis:** Labeled "Scores," with major tick marks at 0, 20, 40, 60, 80, and 100. * **X-Axis:** Labeled "Datasets," with three categorical groups: "HotpotQA," "GSM8k," and "GPQA." * **Legend:** Each chart has a legend in the top-right corner, mapping colors to specific model names. * **Chart 1 (Left): "Comparison of Open-source LLMs"** * **Legend (Top-Right):** * Teal: `LLaMA3.1-8B` * Yellow: `LLaMA3.1-70B` * Light Purple: `Qwen2.5-7B` * Salmon: `Qwen2.5-72B` * **Chart 2 (Center): "Comparison of Closed-source LLMs"** * **Legend (Top-Right):** * Salmon: `Qwen2.5-72B` * Blue: `Claude3.5` * Orange: `GPT-3.5` * Green: `GPT-4o` * **Chart 3 (Right): "Instruction-based vs Reasoning-based"** * **Legend (Top-Right):** * Salmon: `Qwen2.5-72B` * Green: `GPT-4o` * Pink: `QWO-32B` * Purple: `DeepSeek-V3` ### Detailed Analysis #### Chart 1: Comparison of Open-source LLMs * **HotpotQA:** Performance increases with model size. `LLaMA3.1-8B` scores ~60, `LLaMA3.1-70B` ~85, `Qwen2.5-7B` ~75, and `Qwen2.5-72B` ~90. * **GSM8k:** All models perform well. `LLaMA3.1-8B` ~78, `LLaMA3.1-70B` ~92, `Qwen2.5-7B` ~88, `Qwen2.5-72B` ~95 (highest in this chart). * **GPQA:** This is the most challenging dataset for these models. `LLaMA3.1-8B` scores ~20, `LLaMA3.1-70B` ~40, `Qwen2.5-7B` ~32, `Qwen2.5-72B` ~30. Notably, the 70B LLaMA model outperforms the 72B Qwen model here. #### Chart 2: Comparison of Closed-source LLMs * **HotpotQA:** All models score very high and similarly, clustered between ~90 and ~95. * **GSM8k:** Performance remains high and consistent across models, all scoring between ~92 and ~96. * **GPQA:** A significant performance drop is observed for all models. `Qwen2.5-72B` scores ~30, `Claude3.5` ~60, `GPT-3.5` ~50, and `GPT-4o` ~52. `Claude3.5` shows the strongest performance on this difficult dataset. #### Chart 3: Instruction-based vs Reasoning-based * **HotpotQA:** `Qwen2.5-72B` and `GPT-4o` score ~90-92. `QWO-32B` and `DeepSeek-V3` score slightly lower, around ~85. * **GSM8k:** `Qwen2.5-72B` and `GPT-4o` again lead with scores ~95. `QWO-32B` scores ~85, and `DeepSeek-V3` ~88. * **GPQA:** Performance is low across the board. `Qwen2.5-72B` ~30, `GPT-4o` ~52, `QWO-32B` ~28, `DeepSeek-V3` ~52. `GPT-4o` and `DeepSeek-V3` show a notable advantage over the other two models on this dataset. ### Key Observations 1. **Dataset Difficulty:** GPQA is consistently the most challenging benchmark, causing a dramatic performance drop for all models compared to HotpotQA and GSM8k. 2. **Model Scaling:** In the open-source chart, larger models (70B/72B) generally outperform smaller ones (7B/8B), with the notable exception on GPQA where `LLaMA3.1-70B` beats `Qwen2.5-72B`. 3. **Closed-source Dominance:** Closed-source models (Chart 2) show less variance and maintain higher scores on the easier datasets (HotpotQA, GSM8k) compared to the open-source models. 4. **Performance Clustering:** On HotpotQA and GSM8k, top-tier models from all categories cluster in the 85-95 score range, suggesting these tasks may be approaching saturation for advanced LLMs. 5. **GPQA as a Discriminator:** The GPQA dataset effectively differentiates model capabilities, with `Claude3.5`, `GPT-4o`, and `DeepSeek-V3` showing a clear lead over others. ### Interpretation The data suggests a clear hierarchy of task difficulty for current LLMs, with GPQA representing a frontier challenge likely requiring deeper reasoning or specialized knowledge. The strong performance of closed-source models, particularly on the harder GPQA task, indicates potential advantages in training data, architecture, or post-training refinement. The comparison between instruction-based and reasoning-based models (Chart 3) is less clear-cut from the labels alone, but the data shows that model performance is highly dataset-dependent. A model's strength on one benchmark (e.g., GSM8k) does not guarantee proportional strength on another (e.g., GPQA). The outlier performance of `LLaMA3.1-70B` on GPQA compared to the larger `Qwen2.5-72B` suggests that raw parameter count is not the sole determinant of capability; training methodology and data quality are critical factors. Overall, the charts demonstrate that while many models excel on standard benchmarks, the development of robust models that perform well across diverse and challenging tasks like GPQA remains an active area of competition and research. </details> Figure 21: Performance Comparison of Self-Refine <details> <summary>x22.png Details</summary>  ### Visual Description ## [Multi-Panel Bar Chart]: LLM Performance Comparison Across Datasets ### Overview The image displays three horizontally arranged bar charts comparing the performance of various Large Language Models (LLMs) on three benchmark datasets: HotpotQA, GSM8k, and GPQA. The charts are segmented by model type: open-source, closed-source, and instruction-based vs. reasoning models. All charts share a common y-axis labeled "Scores" ranging from 0 to 80, and an x-axis labeled "Datasets". ### Components/Axes * **Overall Layout:** Three distinct bar charts arranged side-by-side. * **Common Y-Axis:** Labeled "Scores", with major tick marks at 0, 20, 40, 60, and 80. * **Common X-Axis:** Labeled "Datasets", with three categorical tick marks: "HotpotQA", "GSM8k", and "GPQA". * **Chart 1 (Left):** Title: "Comparison of Open-source LLMs". Legend (top-right corner): LLaMA3.1-8B (teal), LLaMA3.1-70B (light yellow), Qwen2.5-7B (light purple), Qwen2.5-72B (salmon). * **Chart 2 (Center):** Title: "Comparison of Closed-source LLMs". Legend (top-right corner): Qwen2.5-72B (salmon), Claude3.5 (blue), GPT-3.5 (orange), GPT-4o (green). * **Chart 3 (Right):** Title: "Instruction-based vs. Reasoning LLMs". Legend (top-right corner): Qwen2.5-72B (salmon), GPT-4o (green), QWQ-32B (pink), DeepSeek-V3 (purple). ### Detailed Analysis **Chart 1: Comparison of Open-source LLMs** * **HotpotQA:** LLaMA3.1-8B (~70), LLaMA3.1-70B (~68), Qwen2.5-7B (~60), Qwen2.5-72B (~70). * **GSM8k:** LLaMA3.1-8B (~58), LLaMA3.1-70B (~62), Qwen2.5-7B (~60), Qwen2.5-72B (~72). * **GPQA:** LLaMA3.1-8B (~5), LLaMA3.1-70B (~18), Qwen2.5-7B (~8), Qwen2.5-72B (~12). * **Trend:** Performance is relatively high and clustered on HotpotQA and GSM8k, but drops dramatically for all models on the GPQA dataset. Qwen2.5-72B is the top performer on GSM8k. **Chart 2: Comparison of Closed-source LLMs** * **HotpotQA:** Qwen2.5-72B (~70), Claude3.5 (~80), GPT-3.5 (~72), GPT-4o (~72). * **GSM8k:** Qwen2.5-72B (~72), Claude3.5 (~78), GPT-3.5 (~72), GPT-4o (~80). * **GPQA:** Qwen2.5-72B (~12), Claude3.5 (~42), GPT-3.5 (~22), GPT-4o (~16). * **Trend:** Claude3.5 and GPT-4o show strong, leading performance on HotpotQA and GSM8k. Claude3.5 is a significant outlier on GPQA, achieving a score (~42) more than double that of the next closest model (GPT-3.5 at ~22). **Chart 3: Instruction-based vs. Reasoning LLMs** * **HotpotQA:** Qwen2.5-72B (~70), GPT-4o (~72), QWQ-32B (~55), DeepSeek-V3 (~75). * **GSM8k:** Qwen2.5-72B (~72), GPT-4o (~80), QWQ-32B (~52), DeepSeek-V3 (~72). * **GPQA:** Qwen2.5-72B (~12), GPT-4o (~16), QWQ-32B (~22), DeepSeek-V3 (~30). * **Trend:** GPT-4o leads on GSM8k. DeepSeek-V3 shows the strongest performance on HotpotQA and GPQA among this group. QWQ-32B underperforms on HotpotQA and GSM8k but shows relative strength on GPQA compared to its other scores. ### Key Observations 1. **Dataset Difficulty:** GPQA is universally the most challenging dataset, with all models scoring below 45, and most below 20. 2. **Model Standouts:** Claude3.5 demonstrates exceptional performance on the difficult GPQA dataset. GPT-4o is consistently a top performer across all datasets and model groupings. 3. **Open vs. Closed:** The highest-performing open-source model (Qwen2.5-72B) generally matches or slightly trails the top closed-source models on HotpotQA and GSM8k but falls far behind on GPQA. 4. **Cross-Chart Reference:** Qwen2.5-72B and GPT-4o appear in multiple charts, providing a direct performance bridge between the different model categories. ### Interpretation The data suggests a clear hierarchy in LLM capability based on model architecture and training. Closed-source models, particularly Claude3.5 and GPT-4o, exhibit superior performance, especially on the complex reasoning tasks represented by GPQA. The dramatic performance drop for all models on GPQA indicates this benchmark tests a capability frontier that current models struggle with. The standout performance of Claude3.5 on GPQA suggests it may have a unique architectural or training advantage for that specific type of reasoning. The comparison in the third chart implies that models explicitly designed for reasoning (like DeepSeek-V3) may have an edge on certain tasks (GPQA, HotpotQA) over general instruction-tuned models, though this advantage is not universal across all benchmarks (e.g., GSM8k). The charts collectively highlight that while open-source models are competitive on some tasks, a significant performance gap remains on the most challenging benchmarks. </details> Figure 22: Performance Comparison of Reflexion-v1 <details> <summary>x23.png Details</summary>  ### Visual Description \n ## [Multi-Panel Bar Chart]: Performance Comparison of Large Language Models (LLMs) ### Overview The image displays three adjacent bar charts comparing the performance scores of various Large Language Models (LLMs) across three benchmark datasets: HotpotQA, GSM8k, and GPQA. The charts are segmented by model type: open-source, closed-source, and a comparison of instruction-based versus reasoning-focused models. All charts share the same y-axis scale ("Scores" from 0 to 100) and x-axis categories (the three datasets). ### Components/Axes * **Chart 1 (Left):** Title: "Comparison of Open-source LLMs". * **X-axis Label:** "Datasets" * **X-axis Categories:** "HotpotQA", "GSM8k", "GPQA" * **Y-axis Label:** "Scores" * **Y-axis Scale:** 0, 20, 40, 60, 80, 100 * **Legend (Top-Right):** Four models, each with a distinct color: * LLaMA3.1-8B (Teal) * LLaMA3.1-70B (Light Yellow) * Qwen2.5-7B (Light Purple) * Qwen2.5-72B (Salmon/Red) * **Chart 2 (Center):** Title: "Comparison of Closed-source LLMs". * **X-axis Label:** "Datasets" * **X-axis Categories:** "HotpotQA", "GSM8k", "GPQA" * **Y-axis Label:** "Scores" (implied from left chart) * **Legend (Top-Right):** Four models: * Qwen2.5-72B (Salmon/Red) * Claude3.5 (Blue) * GPT-3.5 (Orange) * GPT-4o (Green) * **Chart 3 (Right):** Title: "Instruction-based vs Reasoning LLMs". * **X-axis Label:** "Datasets" * **X-axis Categories:** "HotpotQA", "GSM8k", "GPQA" * **Y-axis Label:** "Scores" (implied from left chart) * **Legend (Top-Right):** Four models: * Qwen2.5-72B (Salmon/Red) * GPT-4o (Green) * QWQ-32B (Pink) * DeepSeek-V3 (Purple) ### Detailed Analysis **Chart 1: Open-source LLMs** * **HotpotQA:** All four models score very similarly, clustered tightly around approximately 85-90. * **GSM8k:** Performance remains high and consistent across models, again in the ~85-90 range. * **GPQA:** A significant performance drop is observed for all models. Scores range from approximately 15 (Qwen2.5-72B) to 25 (Qwen2.5-7B). This dataset appears substantially more challenging for these open-source models. **Chart 2: Closed-source LLMs** * **HotpotQA:** GPT-4o (Green) leads with a score near 95. Claude3.5 (Blue) and Qwen2.5-72B (Salmon) are close behind (~90). GPT-3.5 (Orange) scores slightly lower (~85). * **GSM8k:** GPT-4o again leads (~95). Claude3.5 and Qwen2.5-72B are very close (~90-92). GPT-3.5 is slightly lower (~88). * **GPQA:** A dramatic drop in scores occurs. GPT-4o maintains the highest score (~55). Claude3.5 scores ~30. GPT-3.5 scores ~20. Qwen2.5-72B (Salmon) scores the lowest, approximately 15. **Chart 3: Instruction-based vs Reasoning LLMs** * **HotpotQA:** GPT-4o (Green) leads (~95). DeepSeek-V3 (Purple) is very close (~92). Qwen2.5-72B (Salmon) and QWQ-32B (Pink) score ~85-88. * **GSM8k:** GPT-4o leads (~95). DeepSeek-V3 is again very close (~93). Qwen2.5-72B and QWQ-32B score ~88-90. * **GPQA:** DeepSeek-V3 (Purple) shows the strongest performance among this group, scoring approximately 30. QWQ-32B (Pink) scores ~25. GPT-4o (Green) scores ~20. Qwen2.5-72B (Salmon) scores the lowest, ~15. ### Key Observations 1. **Dataset Difficulty:** GPQA is consistently the most challenging benchmark for all model categories, causing a universal and significant drop in scores compared to HotpotQA and GSM8k. 2. **Model Leadership:** GPT-4o (Green) is the top performer in the closed-source and instruction/reasoning comparisons on the two easier datasets (HotpotQA, GSM8k). 3. **Open-source vs. Closed-source Gap:** On the harder GPQA dataset, the top closed-source model (GPT-4o, ~55) significantly outperforms the top open-source model (Qwen2.5-7B, ~25). 4. **Specialized Performance:** In the third chart, DeepSeek-V3 (Purple), a reasoning-focused model, achieves the highest score on the challenging GPQA dataset (~30), outperforming the general-purpose GPT-4o (~20) on that specific task. 5. **Scale Matters (Open-source):** Within the open-source chart, the larger 70B/72B parameter models (LLaMA3.1-70B, Qwen2.5-72B) do not consistently outperform their smaller 8B/7B counterparts across all datasets, suggesting task-specific optimization may be as important as scale. ### Interpretation This comparative analysis reveals several key insights into the current LLM landscape: * **Benchmark Sensitivity:** Model performance is highly dependent on the evaluation benchmark. Models that excel on knowledge-intensive (HotpotQA) or mathematical (GSM8k) tasks may struggle on more complex reasoning or specialized knowledge tasks (GPQA). * **The State of the Art:** GPT-4o represents a high-water mark for general performance across common benchmarks. However, its advantage narrows or disappears on the most difficult tasks, where specialized models like DeepSeek-V3 can show superior capability. * **Open-source Progress and Limits:** Open-source models have achieved near parity with closed-source models on certain standard benchmarks (HotpotQA, GSM8k). However, a substantial performance gap remains on the most challenging evaluations (GPQA), indicating that the most advanced reasoning or knowledge synthesis capabilities may still be concentrated in proprietary systems. * **Strategic Model Selection:** The data suggests that choosing an LLM requires careful consideration of the target task. For general use, a model like GPT-4o is strong. For specialized reasoning on hard problems, a model like DeepSeek-V3 might be preferable. For cost-effective deployment on standard tasks, a capable open-source model like Qwen2.5-7B could be sufficient. </details> Figure 23: Performance Comparison of Reflexion-v2 <details> <summary>x24.png Details</summary>  ### Visual Description ## Bar Charts: LLM Performance Comparison Across Datasets ### Overview The image displays three side-by-side bar charts comparing the performance scores of various Large Language Models (LLMs) on three distinct evaluation datasets: HotpotQA, GSM8k, and GPQA. The charts are categorized by model type: open-source, closed-source, and a comparison of instruction-tuned versus reasoning-focused models. ### Components/Axes * **Chart Titles (Top):** * Left: "Comparison of Open-source LLMs" * Center: "Comparison of Closed-source LLMs" * Right: "Instruction-based vs Reasoning LLMs" * **Y-Axis (All Charts):** Labeled "Scores". The scale runs from 0 to 100 with major tick marks at 0, 20, 40, 60, 80, and 100. * **X-Axis (All Charts):** Labeled "Datasets". The three categorical datasets are, from left to right: "HotpotQA", "GSM8k", and "GPQA". * **Legends (Top-Right of each chart):** * **Left Chart (Open-source):** * Teal bar: `LLaMA3.1-8B` * Yellow bar: `LLaMA3.1-70B` * Light purple bar: `Qwen2.5-7B` * Salmon/Red bar: `Qwen2.5-72B` * **Center Chart (Closed-source):** * Salmon/Red bar: `Qwen2.5-72B` * Blue bar: `Claude3.5` * Yellow bar: `GPT-3.5` * Green bar: `GPT-4o` * **Right Chart (Instruction vs Reasoning):** * Salmon/Red bar: `Qwen2.5-72B` * Green bar: `GPT-4o` * Pink bar: `QWQ-32B` * Purple bar: `DeepSeek-V3` ### Detailed Analysis **1. Left Chart: Comparison of Open-source LLMs** * **HotpotQA:** `LLaMA3.1-70B` (yellow) scores highest (~90), followed by `Qwen2.5-72B` (salmon, ~88), `LLaMA3.1-8B` (teal, ~82), and `Qwen2.5-7B` (purple, ~72). * **GSM8k:** `LLaMA3.1-70B` (yellow) again leads (~90), with `Qwen2.5-72B` (salmon, ~86) and `LLaMA3.1-8B` (teal, ~80) close behind. `Qwen2.5-7B` (purple) scores ~78. * **GPQA:** All models show a dramatic performance drop. `Qwen2.5-72B` (salmon) scores highest (~40), followed by `LLaMA3.1-70B` (yellow, ~38), `Qwen2.5-7B` (purple, ~36), and `LLaMA3.1-8B` (teal, ~35). * **Trend:** Performance is relatively high and stable for HotpotQA and GSM8k but collapses for GPQA across all open-source models. Larger models (70B/72B) consistently outperform their smaller counterparts (8B/7B). **2. Center Chart: Comparison of Closed-source LLMs** * **HotpotQA:** `GPT-4o` (green) scores highest (~92), followed by `Claude3.5` (blue, ~88), `Qwen2.5-72B` (salmon, ~88), and `GPT-3.5` (yellow, ~86). * **GSM8k:** `GPT-4o` (green) leads (~94), with `Claude3.5` (blue, ~92) and `Qwen2.5-72B` (salmon, ~90) close. `GPT-3.5` (yellow) scores ~88. * **GPQA:** A significant drop occurs again. `Claude3.5` (blue) scores highest (~50), followed by `GPT-4o` (green, ~36), `Qwen2.5-72B` (salmon, ~40), and `GPT-3.5` (yellow, ~28). * **Trend:** Similar to open-source models, performance is strong on HotpotQA/GSM8k but weak on GPQA. `GPT-4o` and `Claude3.5` are the top performers overall. `GPT-3.5` shows the most significant relative decline on GPQA. **3. Right Chart: Instruction-based vs Reasoning LLMs** * **HotpotQA:** `GPT-4o` (green) and `DeepSeek-V3` (purple) are nearly tied for highest (~92), with `Qwen2.5-72B` (salmon, ~88) and `QWQ-32B` (pink, ~86) slightly behind. * **GSM8k:** `GPT-4o` (green) leads (~94), followed by `DeepSeek-V3` (purple, ~92), `Qwen2.5-72B` (salmon, ~90), and `QWQ-32B` (pink, ~88). * **GPQA:** `DeepSeek-V3` (purple) scores highest (~40), followed by `Qwen2.5-72B` (salmon, ~40), `GPT-4o` (green, ~36), and `QWQ-32B` (pink, ~32). * **Trend:** The pattern of high performance on the first two datasets and low performance on GPQA persists. `GPT-4o` and `DeepSeek-V3` are the strongest models in this comparison. The reasoning-focused model `QWQ-32B` (pink) generally scores lower than the others, especially on GPQA. ### Key Observations 1. **Dataset Difficulty:** GPQA is universally the most challenging dataset, causing a performance drop of 40-60 points for every model compared to HotpotQA and GSM8k. 2. **Model Scaling:** Within model families (e.g., LLaMA, Qwen), larger parameter models (70B/72B) consistently outperform smaller ones (8B/7B). 3. **Top Performers:** `GPT-4o` and `Claude3.5` are the top-performing closed-source models. Among open-source models, `Qwen2.5-72B` and `LLaMA3.1-70B` are the strongest. 4. **Instruction vs. Reasoning:** The dedicated reasoning model `QWQ-32B` does not outperform general instruction-tuned models like `GPT-4o` or `DeepSeek-V3` on these benchmarks, and it scores the lowest on the difficult GPQA task. ### Interpretation The data suggests a clear hierarchy of task difficulty for current LLMs. HotpotQA (likely a multi-hop reasoning QA task) and GSM8k (grade-school math) appear to be tasks where state-of-the-art models have achieved high proficiency. In contrast, GPQA (likely a more complex, specialized, or adversarial dataset) represents a significant frontier where all models, regardless of size or training paradigm (open/closed, instruction/reasoning), struggle. The consistent performance gap between model sizes underscores the continued importance of scale. The strong showing of `DeepSeek-V3` and `Qwen2.5-72B` indicates that the performance gap between leading open-source and closed-source models is narrow on these specific benchmarks. However, the catastrophic drop on GPQA for all models implies that current evaluation metrics may not fully capture robustness or generalization to highly complex problems. The charts collectively highlight that while LLMs have mastered certain benchmark tasks, achieving reliable performance on more demanding, real-world-like challenges remains an open problem. </details> Figure 24: Performance Comparison of RARR <details> <summary>x25.png Details</summary>  ### Visual Description ## Bar Charts: LLM Performance Comparison Across Datasets ### Overview The image displays three side-by-side bar charts comparing the performance of various Large Language Models (LLMs) on three benchmark datasets: HotpotQA, GSM8k, and GPQA. The charts are categorized by model type: open-source, closed-source, and instruction-based vs. reasoning-based. The y-axis for all charts represents "Scores" on a scale from 0 to 100. ### Components/Axes * **Common Elements:** * **Y-axis:** Label: "Scores". Scale: 0 to 100, with major ticks at 0, 20, 40, 60, 80, 100. * **X-axis:** Label: "Datasets". Categories: "HotpotQA", "GSM8k", "GPQA". * **Chart Titles:** Located above each chart. * **Legends:** Positioned in the top-right corner of each chart's plot area. * **Chart 1 (Left): "Comparison of Open-source LLMs"** * **Legend (Top-Right):** * Teal bar: `LLaMA3.1-8B` * Yellow bar: `LLaMA3.1-70B` * Light Purple bar: `Qwen2.5-7B` * Salmon bar: `Qwen2.5-72B` * **Chart 2 (Middle): "Comparison of Closed-source LLMs"** * **Legend (Top-Right):** * Salmon bar: `Qwen2.5-72B` * Blue bar: `Claude3.5` * Orange bar: `GPT-3.5` * Green bar: `GPT-4o` * **Chart 3 (Right): "Instruction-based vs. Reasoning LLMs"** * **Legend (Top-Right):** * Salmon bar: `Qwen2.5-72B` * Green bar: `GPT-4o` * Pink bar: `QWQ-32B` * Purple bar: `DeepSeek-V3` ### Detailed Analysis **Chart 1: Comparison of Open-source LLMs** * **HotpotQA:** `Qwen2.5-72B` (salmon) leads with a score of ~85. `LLaMA3.1-70B` (yellow) is next at ~82, followed by `LLaMA3.1-8B` (teal) at ~78, and `Qwen2.5-7B` (light purple) at ~72. * **GSM8k:** All models score significantly higher. `Qwen2.5-7B` (light purple) achieves the highest score of ~95. `LLaMA3.1-70B` (yellow) is close at ~92, `Qwen2.5-72B` (salmon) at ~90, and `LLaMA3.1-8B` (teal) at ~72. * **GPQA:** Performance drops drastically for all models. `LLaMA3.1-70B` (yellow) and `Qwen2.5-72B` (salmon) tie for the lead at ~25. `LLaMA3.1-8B` (teal) and `Qwen2.5-7B` (light purple) are lower at ~15. **Chart 2: Comparison of Closed-source LLMs** * **HotpotQA:** `GPT-4o` (green) leads at ~90. `Qwen2.5-72B` (salmon) is at ~85, `Claude3.5` (blue) at ~82, and `GPT-3.5` (orange) at ~78. * **GSM8k:** `GPT-4o` (green) achieves the highest score across all charts at ~98. `Claude3.5` (blue) is at ~95, `Qwen2.5-72B` (salmon) at ~90, and `GPT-3.5` (orange) at ~85. * **GPQA:** `Claude3.5` (blue) and `GPT-3.5` (orange) tie for the lead at ~38. `Qwen2.5-72B` (salmon) and `GPT-4o` (green) are lower at ~25. **Chart 3: Instruction-based vs. Reasoning LLMs** * **HotpotQA:** `GPT-4o` (green) leads at ~90. `Qwen2.5-72B` (salmon) and `DeepSeek-V3` (purple) are tied at ~85. `QWQ-32B` (pink) is lower at ~65. * **GSM8k:** `GPT-4o` (green) again leads at ~98. `DeepSeek-V3` (purple) is very close at ~95, `Qwen2.5-72B` (salmon) at ~90, and `QWQ-32B` (pink) at ~72. * **GPQA:** `DeepSeek-V3` (purple) leads this category at ~35. `Qwen2.5-72B` (salmon) and `GPT-4o` (green) are at ~25, while `QWQ-32B` (pink) is lowest at ~15. ### Key Observations 1. **Dataset Difficulty:** GPQA is consistently the most challenging dataset, with all models scoring below 40. GSM8k is the easiest, with several models scoring above 90. 2. **Model Scaling:** In the open-source chart, the 70B/72B parameter models (`LLaMA3.1-70B`, `Qwen2.5-72B`) generally outperform their smaller 7B/8B counterparts, especially on HotpotQA and GPQA. 3. **Top Performers:** `GPT-4o` (green) is the top performer on HotpotQA and GSM8k in the closed-source and instruction/reasoning charts. `Claude3.5` (blue) shows strong, consistent performance, particularly on GPQA. 4. **Specialization:** `QWQ-32B` (pink), labeled as a reasoning model, shows a notable performance drop on HotpotQA compared to GSM8k, suggesting potential specialization. 5. **Open vs. Closed:** The top open-source model (`Qwen2.5-72B`) is competitive with, but does not surpass, the top closed-source models (`GPT-4o`, `Claude3.5`) on any dataset. ### Interpretation The data illustrates a clear hierarchy in LLM capabilities across different types of reasoning tasks. The consistent struggle on GPQA suggests it tests a form of reasoning (likely complex, multi-step, or specialized knowledge) that remains a significant challenge for current models, regardless of size or training paradigm. The strong performance of `GPT-4o` and `Claude3.5` on GSM8k (mathematical reasoning) and HotpotQA (multi-hop factual reasoning) indicates these closed-source models have robust general reasoning abilities. The fact that the open-source `Qwen2.5-72B` is competitive but not superior suggests a potential "performance ceiling" that may require different architectural or training innovations to break, not just scaling. The third chart hints at a potential trade-off: models optimized for instruction-following (like `Qwen2.5-72B`) may not excel at the specific type of reasoning tested by `QWQ-32B`, and vice-versa. `DeepSeek-V3`'s strong showing on GPQA is an outlier among the reasoning-focused models in that chart, suggesting its training may have uniquely prepared it for that specific challenge. Overall, the charts depict a landscape where model specialization and task difficulty are critical factors in performance evaluation. </details> Figure 25: Performance Comparison of RATT ## Appendix G Prompt and Result Example ### G.1 Base Prompt Example. Base’s prompt example is as follows: Base’s prompt example Zero-shot Prompt: Your final answer in the form boxed{answer}, at the end of your response. A: Critique Prompt: Review your previous answer and find problems with your answer. Improve Prompt: Based on the problems you found, improve your answer. Please reiterate your answer, with your final answer a single numerical number, In the form boxed{answer}. Result Example on GSM8K. Base’s result example on GSM8K dataset is as follows: Base’s result example on GSM8K dataset $\blacktriangleright$ qustion: Q: Josh decides to try flipping a house. He buys a house for $80,000 and then puts in $50,000 in repairs. This increased the value of the house by 150%. How much profit did he make?Please solve the question above, then store the final answer in boxed{answer}. output: To determine Josh’s profit from flipping the house, let’s break down the costs and the increased value step by step. 1. **Initial Purchase Cost**: Josh buys the house for **$80,000**. 2. **Repair Costs**: He spends **$50,000** on repairs. 3. **Total Investment**: Total cost = Purchase cost + Repair costs Total cost = $80,000 + $50,000 = **$130,000**. 4. **Increased Value**: The repairs increase the value of the house by **150%**. To find the new value: New value = Original value + (150% of original value) New value = $80,000 + (1.5 * $80,000) New value = $80,000 + $120,000 = **$200,000**. 5. **Profit Calculation**: Profit = New value - Total investment Profit = $200,000 - $130,000 = **$70,000**. Thus, Josh’s profit is: boxed{70000}. final_answer: 70000 correct answer: 70000 correct: true ### G.2 CoT Prompt Example. CoT’s prompt example is as follows: CoT’s prompt example prompt_based Prompt: Please understand and solve the question above step by step, then store the final answer in boxed{answer}. Zero-shot-cot Prompt: Let’s think step by step. In the form boxed{answer}, at the end of your response. Critique Prompt: Review your previous answer and find problems with your answer. Improve Prompt: Based on the problems you found, improve your answer. Please reiterate your answer, with your final answer a single numerical number, In the form boxed{answer}. Result Example on MATH. CoT’s result example on MATH dataset is as follows: CoT’s result example on MATH dataset $\blacktriangleright$ qustion: Q: What is the sum of the values of $x$ that satisfy the equation xˆ2-5x+5=9? Please understand and solve the question above step by step, then store the final answer in boxed{answer}. output: To solve the equation ( xˆ2 - 5x + 5 = 9 ), follow these steps: 1. **Move all terms to one side to set the equation to zero:** [ xˆ2 - 5x + 5 - 9 = 0 ] Simplify: [ xˆ2 - 5x - 4 = 0 ] 2. **Identify the quadratic equation:** The equation is in the form ( axˆ2 + bx + c = 0 ), where: - ( a = 1 ) - ( b = -5 ) - ( c = -4 ) 3. **Find the sum of the roots of the quadratic equation:** For a quadratic equation ( axˆ2 + bx + c = 0 ), the sum of the roots is given by: [text{Sum of roots} = -frac{b}{a} ] Substitute the values of ( a ) and ( b ):[ text{Sum of roots} = -frac{-5}{1} = 5] Therefore, the sum of the values of ( x ) that satisfy the equation is: [boxed{5}] final_answer: 5 correct answerr: 3 correct: false error: Final_answer and answer do not match ### G.3 RCI Prompt Example. RCI’s prompt example is as follows: RCI’s prompt example Initial Prompt: Let’s think step by step. In the form boxed{answer}, at the end of your response. Critique Prompt: Review your previous answer and find problems with your answer. Improve Prompt: Based on the problems you found, improve your answer. Please reiterate your answer, with your final answer a single numerical number, In the form boxed{answer}. Result Example on MATH. RCI’s result example on MATH dataset is as follows: RCI’s result example on MATH dataset $\blacktriangleright$ Question: What is the degree of the polynomial $(4+5x^3+100+2π x^4+√{10}x^4+9)$ ? Correct Answer: 4 Round 0: The degree of a polynomial is defined as the highest exponent of the variable, which is the variable with the highest degree. We have a polynomial in the variable $x$ , which includes the terms $4$ , $5x^3$ , $100$ , $2π x^4$ , $√{10}x^4$ , and $9$ . The term $5x^3$ has an exponent of $3$ , the terms $2π x^4$ and $√{10}x^4$ have an exponent of $4$ , and all of the other terms have an exponent of $0$ . Therefore, the degree of the polynomial is boxed4. Round 1 - Critique: No, I do not see any problems with my answer. I think I have already provided a clear and complete answer. Final Answer: The final answer is boxed4. The degree of a polynomial is defined as the highest exponent of the variable, which is the variable with the highest degree. In this case, $x^4$ is the term with the highest power of $x$ . Thus, the degree of the polynomial is $4$ . Round 1 - Output: I think I have already provided a clear and complete answer. To confirm this answer, we can analyze the terms of the polynomial. The polynomial contains the terms $4$ , $5x^3$ , $100$ , $2π x^4$ , $√{10}x^4$ , and $9$ . The term $5x^3$ has an exponent of $3$ , the terms $2π x^4$ and $√{10}x^4$ have an exponent of $4$ , and all of the other terms have an exponent of $0$ . In this case, the variable is $x$ , and the term with the highest degree is $x^4$ . Therefore, the degree of the polynomial is {boxed{4}.} Final Answer: 4 Correct: True ### G.4 CoVe Prompt Example. CoVe’s prompt example is as follows: CoVe’s prompt example Baseline prompt: Please generate a response to the following question. Answer directly and concisely. Question: question Verification prompt: The following is a baseline response to a question. Please generate a set of verification questions to check the accuracy of each fact in the response. List each question on a new line, prefixed with a number and a dot (e.g., 1., 2.). Baseline Response: Baseline response Execute prompt: Please answer the following question independently. Ensure your answer is not influenced by any previous responses. Final prompt: The following is a baseline response and its verification results. Please generate a final response by correcting any errors in the baseline response based on the verification results. Initial Question: question. Baseline Response: Baseline response. Verification Results: Your final answer should be in the form boxed{answer}, at the end of your response. Result Example on MATH. CoVe’s result example on MATH dataset is as follows: CoVe’s result example on MATH dataset $\blacktriangleright$ Question: Berengere and her American foreign-exchange student Emily are at a bakery in Paris that accepts both euros and American dollars. They want to buy a cake, but neither of them has enough money. If the cake costs 6 euros and Emily has an American five-dollar bill, how many euros does Berengere need to contribute to the cost of the cake if 1 euro = 1.25 USD? Baseline Response: Emily’s $5 is equivalent to 4 euros (since $5 / 1.25 = 4 euros). The cake costs 6 euros, so Berengere needs to contribute **2 euros** to cover the remaining cost. 5 Verifications with 1 verification shown : Is the cost of the cake confirmed to be 6 euros? To confirm whether the cost of the cake is 6 euros, you would need to refer to the specific context or source where the price was mentioned. Without additional information or context, I cannot independently verify the cost of the cake. If you have a specific reference or details, please provide them for a more accurate answer. Final Verified Response: Based on the verification results, the baseline response correctly calculates Berengere’s contribution under the assumption that the exchange rate is 1 euro = 1.25 USD, the cake costs 6 euros, and there are no additional fees or factors affecting the total cost. Here’s the corrected and verified response:Emilyś $5 is equivalent to 4 euros (since $5 / 1.25 = 4 euros). The cake costs 6 euros, so Berengere needs to contribute **2 euros** to cover the remaining cost. boxed{2 text{euros}}. Predicted Answer: 2 text{euros} Correct Answer: 2 text{euros} Correct: True ### G.5 Self-Refine Prompt Example. Self-Refine’s prompt example is as follows: 9-Shot init Prompts with 2 Shots Shown Below $\blacktriangleright$ # Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give to Denny? # solution using Python: def solution(): """Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give to Denny?""" jason_lollipops_initial = 20 jason_lollipops_after = 12 denny_lollipops = jason_lollipops_initial - jason_lollipops_after result = denny_lollipops return result $\blacktriangleright$ # Q: There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done, there will be 21 trees. How many trees did the grove workers plant today? # solution using Python: def solution(): """There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done, there will be 21 trees. How many trees did the grove workers plant today?""" trees_initial = 15 trees_after = 21 trees_added = trees_after - trees_initial result = trees_added return result 4-Shot feedback Prompts with 1 Shot Shown Below def solution(): """Twenty dozen cups cost $1200 less than the total cost of half a dozen plates sold at $6000 each. Calculate the total cost of buying each cup.""" plates = 6 plate_cost = 6000 cups = 12 * 20 cup_cost = plate_cost result = cup_cost return result # There is an error in the code above because of lack of understanding of the question. What is the error? To find the error, go through semantically complete blocks of the code, and check if everything looks good. # Let us go through the error and check step-by-step plates = 6 plate_cost = 6000 # looks good # Let’s check the other parts cups = 12 * 20 cup_cost = plate_cost # wrong! The cost of a cup is not the same as the cost of a plate. The cost of a cup is $1200 less than the total cost of half a dozen plates sold at $6000 each. So we need to calculate the cost of a cup first (total cost of half a dozen plates sold at $6000 each - $1200) and use that. Okay! Here is the rewrite: def solution(): """Twenty dozen cups cost $1200 less than the total cost of half a dozen plates sold at $6000 each. Calculate the total cost of buying each cup.""" plates = 6 plate_cost = 6000 cups = 12 * 20 total_cup_cost = (plate_cost * plates) - 1200 cost_per_cup = total_cup_cost / cups return cost_per_cup ### END ### Result Example on MATH. Self-Refine’s result example on MATH dataset is as follows: Self-Refine’s result example on MATH dataset $\blacktriangleright$ question: What is the degree of the polynomial $(4+5x^3+100+2/pix^4+sqrt{10}x^4+9)$ ? $\blacktriangleright$ 8 attempt examples with 1 shown below: solution_curr: def solution():#Parking Lot Cars cars_initial = 3 cars_arrived = 2 total_cars = cars_initial + cars_arrived return total_cars This code is correct and matches the solution I described. It correctly calculates the total number of cars in the parking lot by adding the initial number of cars and the number of cars that arrived.If you’d like me to review the other problems or provide feedback on the code, please let me know! solution_fixed: def solution():#Parking Lot Cars cars_initial = 3 cars_arrived = 2 total_cars = cars_initial + cars_arrived return total_cars Upon reviewing the code, I notice that it is actually correct and matches the solution you described. It correctly calculates the total number of cars in the parking lot by adding the initial number of cars and the number of cars that arrived.Therefore, there is no error in the code. It is a simple and straightforward calculation that produces the correct result.f you’d like me to review any other code or provide feedback, please let me know! feedback: Let’s go through the code step by step to identify the error.The code is calculating the total number of cars in a parking lot by adding the initial number of cars and the number of cars that arrived.Here’s the code:“‘python answer: 5 correct_answer: 4 final_answer: 5 correct: False ### G.6 Reflexion Prompt Example. Reflexion’s prompt example is as follows: Reflexion’s prompt example $\blacktriangleright$ (reflect_prompt) You are an advanced reasoning agent that can improve based on self refection. You will be given a previous reasoning trial in which you were given access to an Docstore API environment and a question to answer. You were unsuccessful in answering the question either because you guessed the wrong answer with Finish[<answer>], or you used up your set number of reasoning steps. In a few sentences, Diagnose a possible reason for failure and devise a new, concise, high level plan that aims to mitigate the same failure. Use complete sentences. Here is an example:{examples} Previous trial: Question: {question} {scratchpad} Reflection: $\blacktriangleright$ (react_agent_prompt) Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. (2) Lookup[keyword], which returns the next sentence containing keyword in the last passage successfully found by Search. (3) Finish[answer], which returns the answer and finishes the task. You may take as many steps as necessary. Here are some examples: examples (END OF EXAMPLES) Question: {question} {scratchpad} $\blacktriangleright$ (react_reflect_agent_prompt) Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search. (2) Lookup[keyword], which returns the next sentence containing keyword in the last passage successfully found by Search. (3) Finish[answer], which returns the answer and finishes the task. You may take as many steps as necessary. Here are some examples: examples (END OF EXAMPLES) {reflections} Question: {question} {scratchpad} $\blacktriangleright$ (reflecion_header) You have attempted to answer following question before and failed. The following reflection(s) give a plan to avoid failing to answer the question in the same way you did previously. Use them to improve your strategy of correctly answering the given question. Result Example on AQUA. Reflexion’s result example on AQUA dataset is as follows: Reflexion’s result example on AQUA dataset $\blacktriangleright$ Question: A travel company wants to charter a plane to the Bahamas. Chartering the plane costs $5,000. So far, 12 people have signed up for the trip. If the company charges $200 per ticket, how many more passengers must sign up for the trip before the company can make any profit on the charter? $\blacktriangleright$ options: A:7 B:9 C:13 D:27 E:45 Trial 1: Scratchpad: Thought 1: To determine how many more passengers need to sign up for the trip before the company can make a profit, we need to calculate the total revenue required to cover the charter cost and then find out how many additional tickets need to be sold. Action 1: The cost of chartering the plane is $5,000. Each ticket is sold for $200. To break even, the company needs to sell enough tickets to cover the $5,000 cost. Observation 1: Invalid Action. Valid Actions are Lookup[topic] Search[topic] and Finish[answer]. Thought 2: To find out how many tickets need to be sold to cover the $5,000 cost, we divide the total cost by the price per ticket. Then, we subtract the number of tickets already sold to find out how many more are needed. Action 2: The company needs to sell enough tickets to cover the $5,000 cost. Each ticket is $200, so the number of tickets needed to break even is $5,000 / $200 = 25 tickets. Since 12 tickets have already been sold, we need to find out how many more are needed to reach 25 tickets. Observation 2: Invalid Action. Valid Actions are Lookup[topic] Search[topic] and Finish[answer]. Thought 3: To find out how many more passengers need to sign up, we calculate the total number of tickets needed to break even and subtract the number of tickets already sold. The company needs to sell 25 tickets to cover the $5,000 cost. Since 12 tickets have already been sold, the company needs to sell 25 - 12 = 13 more tickets to break even. Therefore, to make a profit, they need to sell at least 13 more tickets. Action 3: Finish[13] Observation 3: Answer is CORRECT $\blacktriangleright$ Correctness: True correct_answer: c final_answer: c correct: True ### G.7 RARR Prompt Example. RARR’s prompt example is as follows: RARR’s prompt example $\blacktriangleright$ (Prompts for generating hallucinations.) EVIDENCE_HALLUCINATION = Generate a paragraph that answers the question. $\blacktriangleright$ Question: What is New York-Style pizza? Text: New York-style pizza has slices that are large and wide with a thin crust that is foldable yet crispy. It is traditionally topped with tomato sauce and mozzarella cheese, with any extra toppings placed on top of the cheese. $\blacktriangleright$ Question: When did the first McDonald’s open? Text: The McDonald’s brothers opened their first McDonald’s restaurant in 1940 in San Bernardino, California. Originally, a carhop drive-in system was used to serve customers. The initial menu items were centered around barbecue and the first name the brothers called their business was "McDonald’s Famous Barbecue." $\blacktriangleright$ Question: {query} $\triangleright$ (6 Prompts for RARR with 1 Prompt Shown Below.) QGEN_PROMPT I will check things you said and ask questions. $\blacktriangleright$ You said: Your nose switches back and forth between nostrils. When you sleep, you switch about every 45 minutes. This is to prevent a buildup of mucus. It’s called the nasal cycle. To verify it, 1. I googled: Does your nose switch between nostrils? 2. I googled: How often does your nostrils switch? 3. I googled: Why does your nostril switch? 4. I googled: What is nasal cycle? $\blacktriangleright$ You said: The Stanford Prison Experiment was conducted in the basement of Encina Hall, Stanford’s psychology building. To verify it, 1. I googled: Where was Stanford Prison Experiment was conducted? $\blacktriangleright$ You said: The Havel-Hakimi algorithm is an algorithm for converting the adjacency matrix of a graph into its adjacency list. It is named after Vaclav Havel and Samih Hakimi. To verify it, 1. I googled: What does Havel-Hakimi algorithm do? 2. I googled: Who are Havel-Hakimi algorithm named after? $\blacktriangleright$ You said: "Time of My Life" is a song by American singer-songwriter Bill Medley from the soundtrack of the 1987 film Dirty Dancing. The song was produced by Michael Lloyd. To verify it, 1. I googled: Who sings the song "Time of My Life"? 2. I googled: Which film is the song "Time of My Life" from? 3. I googled: Who produced the song "Time of My Life"? $\blacktriangleright$ You said: Kelvin Hopins was suspended from the Labor Party due to his membership in the Conservative Party. To verify it, 1. I googled: Why was Kelvin Hopins suspended from Labor Party? $\blacktriangleright$ You said: Social work is a profession that is based in the philosophical tradition of humanism. It is an intellectual discipline that has its roots in the 1800s. To verify it, 1. I googled: What philosophical tradition is social work based on? 2. I googled: What year does social work have its root in? $\blacktriangleright$ You said: {claim} To verify it, Result Example on gsm8k. RARR’s result example on gsm8k dataset is as follows: RARR’s result example on gsm8k $\blacktriangleright$ question: A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take? text: Let me help solve this word problem step by step:1. Blue fiber needed: 2 bolts2. White fiber needed: Half of blue = 2 / 2 = 1 bolt 3. Total bolts = Blue + White = 2 + 1 = 3 bolts Therefore, it takes 3 bolts in total to make the robe. revisions: original_text: Let me help solve this word problem step by step:1. Blue fiber needed: 2 bolts 2. White fiber needed: Half of blue = 2 / 2 = 1 bolt 3. Total bolts = Blue + White = 2 + 1 = 3 bolts Therefore, it takes 3 bolts in total to make the robe. revised_text: Default revision text final_answer: 3 correct_answer: 3 correct: True ### G.8 RATT Prompt Example. RATT’s prompt example is as follows: RATT’s prompt example $\blacktriangleright$ (Query_prompt) Question: question.Content: previous_answer.Instruction: I want to verify the content correctness of the given question, especially the last sentences.Please summarize the content with the corresponding question.This summarization will be used as a query to search with Bing search engine. The query should be short but need to be specific to promise Bing can find related knowledge or pages.You can also use search syntax to make the query short and clear enough for the search engine to find relevant language data.Try to make the query as relevant as possible to the last few sentences in the content. **IMPORTANT** Just output the query directly. DO NOT add additional explanations or introducement in the answer unless you are asked to. $\blacktriangleright$ (Filter_prompt) Text: content. Question: question. Please read the following text and extract only the sections that are relevant to the given question. Organize the extracted information coherently, maintaining the structure of multiple paragraphs with subtitles, and split the paragraphs with Question: question, Text to Filter: content, Instruction: Extract only the relevant information related to the question. Keep the structure clear with multiple paragraphs and subtitles. Provide the filtered information directly without additional explanations or commentary. $\blacktriangleright$ (Draft_prompt) Question: question.IMPORTANT: Try to answer this question/instruction with step-by-step thoughts and make the answer more structural.Use ‘\n\n’ to split the answer into several paragraphs.Just respond to the instruction directly. DO NOT add additional explanations or introducement in the answer unless you are asked to.If you have got the final answer, in the form \boxed{answer}, at the end of your response. $\blacktriangleright$ (Revise_prompt) Existing Text in Wiki Web: content. Question: question. Answer: answer. I want to revise the answer according to retrieved related text of the question in WIKI pages. You need to check whether the answer is correct. If you find some errors in the answer, revise the answer to make it better. If you find some necessary details are ignored, add it to make the answer more plausible according to the related text. If you find that a part of the answer is correct and does not require any additional details, maintain that part of the answer unchanged. Directly output the original content of that part without any modifications. **IMPORTANT** Try to keep the structure (multiple paragraphs with its subtitles) in the revised answer and make it more structural for understanding. Split the paragraphs with ‘\n\n‘ characters. Just output the revised answer directly. DO NOT add additional explanations or announcements in the revised answer unless you are asked to. If you have got the final answer, in the form \boxed{answer}, at the end of your response. $\blacktriangleright$ (Refine_prompt) Agent_drafts: agent_drafts.Referencing the answers provided by all agents, synthesize a more detailed and comprehensive response by integrating all relevant details from these answers. Ensure logical coherence and provide ONLY THE MERGED ANSWER AS THE OUTPUT, omitting any discussion of the comparison process or analytical thoughts.If you have got the final answer, in the form \boxed{answer}, at the end of your response. $\blacktriangleright$ (Refine_prompt) Final_prompt: Based on the original answer and an additional supplementary answer, generate a response that is richer in detail and logically coherent. Review the original answer:1. If any part of the answer is correct and requires no further details, retain that portion unchanged and output it directly as it is.2. For parts that may be improved or lack necessary details, enhance them by integrating information from the supplementary answer to make the response more comprehensive and accurate.3. If you identify any errors within the answers, correct these errors while ensuring that the revised content remains logically coherent. Original Answer: previous_answer. Supplementary Answer: final_draft_raw. **IMPORTANT** Ensure the revised answer maintains a structured format (multiple paragraphs with subtitles) for better clarity. Separate the paragraphs with ‘\n\n’ characters. Output only the enhanced answer directly, without any extra explanations or announcements unless specifically requested.If you have got the final answer, in the form \boxed{answer}, at the end of your response. $\blacktriangleright$ (Conclude_prompt (optional)) Final_draft:final_answers.Referencing the answers provided by each step, synthesize a more detailed and comprehensive response by integrating all relevant details from these answers. Ensure logical coherence and provide ONLY THE MERGED ANSWER AS THE OUTPUT, omitting any discussion of the comparison process or analytical thoughts.If you have got the final answer, in the form \boxed{answer}, at the end of your response. Result Example on MATH. RATT’s result example on MATH dataset is as follows: RATT’s result example on MATH dataset $\blacktriangleright$ question: What is the value of $(2x + 5)ˆ2$ when $x = 3$?" draft_cot: To find the value of ((2x + 5)ˆ2) when (x = 3), follow these steps: First, substitute (x = 3) into the expression: [(2(3) + 5)ˆ2] Next, simplify inside the parentheses: [(6 + 5)ˆ2 = (11)ˆ2] Finally, calculate the square: [11ˆ2 = 121 The value of ((2x + 5)ˆ2) when (x = 3) is (boxed{121}). final_answer: 121 correct_answer: 121 is_correct: True ### G.9 CRITIC Prompt Example. CRITIC’s prompt example is as follows: 8-Shot CoT Prompts with 2 Shots Shown Below $\blacktriangleright$ Question: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers’ market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers’ market? # Python code, return answer total_eggs = 16 eaten_eggs = 3 baked_eggs = 4 sold_eggs = total_eggs - eaten_eggs - baked_eggs dollars_per_egg = 2 answer = sold_eggs * dollars_per_egg $\blacktriangleright$ Question: A robe takes 2 bolts of blue fiber and half that much white fiber.How many bolts in total does it take? # Python code, return answer bolts_of_blue_fiber = 2 bolts_of_white_fiber = num_of_blue_fiber / 2 answer = bolts_of_blue_fiber + bolts_of_white_fiber 4 Critique Prompts with 1 Shot Shown Below: $\blacktriangleright$ Question: A farmer has twice as many pigs as cows, and 4 more cows than goats. If the farmer has 56 animals total, how many pigs does he have? “‘python num_of_pigs = 2 * num_of_cows num_of_cows = number_of_goats + 4 num_of_animals = 56 number_of_goats = (num_of_animals - 4) / 3 num_of_cows = (number_of_goats + 4) * 2 answer = num_of_cows “‘ $\blacktriangleright$ Execution: NameError("name ’num_of_cows’ is not defined") $\blacktriangleright$ Output: answer = None $\triangleright$ What’s the problem with the above code? 1. The above code causes the "NameError" because it use the variable ‘num_of_cows‘ before it is defined. 2. The variable names in the code are a little bit confusing, becase both ‘num_of_pigs‘ and "number_of_goats" are used. Let’s analysis the problem, we know that the total number of animals are 56, but we don’t konw the number of pigs, cows or goats, and we can’t get any of them directly, so we can build equations to solve for the number of pigs. $\triangleright$ Here’s a better solution: “‘python # let’s define the number of goats is x # then the number of cows is 4+x # then the number of pigs is 2(4+x) # so, total animals is x + (4+x) + 2(4+x) = 56 # Now we can solve for "x": # x + 4 + x + 2x + 8 = 56 # 4x + 12 = 56 # 4x = 44 # x = 11 # So the farmer has 11 goats. num_of_goats = 11 num_of_cows = num_of_goats + 4 # calculate the answer given the number of goats num_of_pigs = 2 * num_of_cows $\blacktriangleright$ answer = num_of_pigs “‘ 6 Shot CoT Prompts with 2 Shots Shown Below (Question Answering): $\blacktriangleright$ Q: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into? A: Let’s think step by step. The eastern sector of Colorado orogeny extends into the High Plains. High Plains rise in elevation from around 1,800 to 7,000 ft. So the answer is: 1,800 to 7,000 ft. $\blacktriangleright$ Q: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who? A: Let’s think step by step. Milhouse was named after U.S. president Richard Nixon. So the answer is: Richard Nixon. 6 Critique Prompts with 1 Shot Shown Below (Question Answering): $\blacktriangleright$ Question: Are John H. Auer and Jerome Robbins both directors? Proposed Answer: Let’s think step by step. John H. Auer is a director, producer, and screenwriter. Jerome Robbins is a director, producer, choreographer, and dancer. So the answer is: Jerome Robbins. $\triangleright$ What’s the problem with the above answer? $\triangleright$ 1. Plausibility: This is a yes/no question, the proposed answer is "Jerome Robbins", which does not clearly answer whether they are both directors. The answer should be a clear "yes" or "no". $\triangleright$ 2. Truthfulness: Let’s search the question in google: $\blacktriangleright$ Search Query: Are John H. Auer and Jerome Robbins both directors? $\blacktriangleright$ Evidence: [A history of Jerome Robbins at PNB - Pacific Northwest Ballet] Robbins retained the title of associate artistic director until approximately 1963, … Ballets: USA, from the late 1950s to the late 1960s. The evidence suggests Jerome Robbins is a director. We then need to verify if John H. Auer is a director. $\blacktriangleright$ Search Query: Is John H. Auer a director? site: wikipedia.org $\blacktriangleright$ Evidence: [John H. Auer - Wikipedia] Auer was a Hungarian-born child actor who, on coming to the Americas in 1928, became a movie director and producer, initially in Mexico but, from the early 1930s, in Hollywood. The evidence suggests that John H. Auer is an actor, director and producer. Therefore, both John H. Auer and Jerome Robbins are directors. $\blacktriangleright$ Question: Are John H. Auer and Jerome Robbins both directors? Here’s the most possible answer: Let’s think step by step. John H. Auer is an actor, director and producer. Jerome Robbins is a director. Therefore, Both John H. Auer and Jerome Robbins are directors. So the answer is: Yes. Result Example on MATH. CRITIC’s Mathematical Program Synthesis task result example on MATH dataset is as follows: CRITIC’s Mathematical Program Synthesis task result example on MATH dataset $\blacktriangleright$ Question: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers’ market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers’ market? Correct Answer: 18 Round 0: \n total_eggs = 16\n eaten_eggs = 3\n baked_eggs = 4\n sold_eggs = total_eggs - eaten_eggs - baked_eggs\n dollars_per_egg = 2\n answer = sold_eggs * dollars_per_egg\n print(answer) Round 1 - Critique(Through python Interpreter): \n total_eggs = 16\n eaten_eggs = 3\n baked_eggs = 4\n sold_eggs = total_eggs - eaten_eggs - baked_eggs\n dollars_per_egg = 2\n answer = sold_eggs * dollars_per_egg\n print(answer) Round 1 - Output: 18 Final Answer: 18 Correct: True Result Example on HotPotQA. CRITIC’s Question Answering task result example on HotPotQA dataset is as follows: CRITIC’s Question Answering task result example on HotPotQA dataset $\blacktriangleright$ Question: Were Scott Derrickson and Ed Wood of the same nationality? Correct Answer: yes Round 0: Let’s think step by step. Scott Derrickson is an American film director, screenwriter, and producer. Ed Wood was an American filmmaker, actor, and writer. So Scott Derrickson and Ed Wood are of the same nationality. So the answer is: Yes. Round 1 - Critique(Through Google search): supporting_facts: "title": ["Scott Derrickson", "Ed Wood"] Round 1 - Output: Yes.This answer is plausible and truthful, and the evidence supports the claim that both Scott Derrickson and Ed Wood are American Final Answer: Yes Correct: True ## Appendix H All Performance Results ### H.1 All Performance Results for Intrinsic Correction Table 9: Performance of Intrinsic Correction Methods on Different LLMs ### H.2 All Performance Results for External Correction Table 10: Performance of External Correction Methods on Different LLMs | LLaMA3.1-8B CoT Reflexion-v2 | Base 72.14 (-3.66) 89.02 (+13.22) | 75.8 73.85 (-2.31) 78.66 (+2.5) | 76.16 44.55 (-9.33) 59.77 (+5.89) | 53.88 78.28 (-3.27) 83.9 (+2.35) | 81.55 77.65 (+3.28) 78.14 (+3.77) | 74.37 13.11 (+3.37) 22.14 (+12.4) | 9.74 51.83 (-21.61) - | 73.44 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | RARR | 81.31 (+5.51) | 74.6 (-1.56) | 56.54 (+2.66) | 79.31 (-2.24) | 84.43 (+10.06) | 34.58 (+24.84) | 67.58 (-5.86) | | | RATT | 76.82 (+1.02) | 76.72 (+0.56) | 54.94 (+1.06) | 72.34 (-9.21) | 81.16 (+6.79) | 13.83 (+4.09) | 62.33 (-11.11) | | | CRITIC | 69.33 (-6.47) | - | - | 69.12 (-12.43) | - | - | - | | | LLaMA3.1-70B | Base | 81.28 | 81.88 | 62.65 | 90.63 | 78.21 | 15.62 | 62.18 | | CoT | 86.13 (+4.85) | 82.84 (+0.96) | 62.93 (+0.28) | 95.92 (+5.29) | 60.11 (-18.1) | 23.74 (+8.12) | 46.39 (-15.79) | | | Reflexion-v2 | 88.78 (+7.5) | 86.17 (+4.29) | 62.45 (-0.2) | 82.11 (-8.52) | 84.82 (+6.61) | 24.62 (+9) | - | | | RARR | 90.49 (+9.21) | 79.93 (-1.95) | 67.29 (+4.64) | 88.5 (-2.13) | 87.8 (+9.59) | 35.79 (+20.17) | 78.96 (+16.78) | | | RATT | 82.37 (+1.09) | 82.08 (+0.2) | 68.22 (+5.57) | 86.15 (-4.48) | 85.79 (+7.58) | 26.09 (+10.47) | 66.05 (+3.87) | | | CRITIC | 85.57 (+4.29) | - | - | 95.24 (+4.61) | - | - | - | | | Qwen2.5-7B | Base | 74.05 | 74.75 | 47.5 | 90.23 | 74.28 | 7.53 | 79.11 | | CoT | 72.07 (-1.98) | 56.19 (-18.56) | 50.75 (+3.25) | 87.01 (-3.22) | 68.61 (-5.67) | 12.17 (+4.64) | 63.24 (-15.87) | | | Reflexion-v2 | 87.21 (+13.16) | 67.95 (-6.8) | 54.35 (+6.85) | 89.31 (-0.92) | 85.63 (+11.35) | 26.18 (+18.65) | - | | | RARR | 71.67 (-2.38) | 75.52 (+0.77) | 45.61 (-1.89) | 85.22 (-5.01) | 79.44 (+5.16) | 34.44 (+26.91) | 85.71 (+6.6) | | | RATT | 72.4 (-1.65) | 84.55 (+9.8) | 65.17 (+17.67) | 94.92 (+4.69) | 84.61 (+10.33) | 14.61 (+7.08) | 80.69 (+1.58) | | | CRITIC | 69.89 (-4.16) | - | - | 74.42 (-15.81) | - | - | - | | | Qwen2.5-72B | Base | 83.63 | 81.92 | 57.58 | 91.11 | 66.91 | 9.85 | 86.13 | | CoT | 89.87 (+6.24) | 88.44 (+6.52) | 57.34 (-0.24) | 94.14 (+3.03) | 63.71 (-3.2) | 16.19 (+6.34) | 62.45 (-23.68) | | | Reflection-v2 | 83.69 (+0.06) | 82.33 (+0.41) | 57.46 (-0.12) | 92.64 (+1.53) | 77.46 (+10.55) | 14.72 (+4.87) | - | | | RARR | 87.77 (+4.14) | 86.16 (+4.24) | 57.23 (-0.35) | 86.4 (-4.71) | 73.66 (+6.75) | 40.13 (+30.28) | 87.63 (+1.5) | | | RATT | 84.68 (+1.05) | 84.12 (+2.2) | 64.41 (+6.83) | 90.64 (-0.47) | 77.94 (+11.03) | 25.74 (+15.89) | 84.72 (-1.41) | | | CRITIC | 84.78 (+1.15) | - | - | 79.35 (-11.76) | - | - | - | | | Claude3.5 | Base | 88.29 | 80.25 | 81.26 | 95.81 | 83.51 | 32.34 | 84.69 | | CoT | 92.3 (+4.01) | 82.48 (+2.23) | 64.51 (-16.75) | 97.55 (+1.74) | 85.48 (+1.97) | 16.67 (-15.67) | 80.53 (-4.16) | | | Reflexion-v2 | 93.62 (+5.33) | 91.77 (+11.52) | 84.83 (+3.57) | 94.1 (-1.71) | 92.56 (+9.05) | 53.21 (+20.87) | - | | | RARR | 87.44 (-0.85) | 80.79 (+0.54) | 83.52 (+2.26) | 94.36 (-1.45) | 94.18 (+10.67) | 50.34 (+18.0) | 86.27 (+1.58) | | | RATT | 82.13 (-6.16) | 77.62 (-2.63) | 88.51 (+7.25) | 96.39 (+0.58) | 97.31 (+13.8) | 38.28 (+5.94) | 85.29 (+0.6) | | | CRITIC | 95.16 (+6.87) | - | - | 94.85 (-0.96) | - | - | - | | | GPT-3.5 | Base | 82.94 | 77.92 | 55.15 | 79.14 | 70.44 | 27.29 | 80.29 | | CoT | 89.34 (+6.4) | 81.47 (+3.55) | 65.56 (+10.41) | 97.41 (+18.27) | 74.91 (+4.47) | 17.84 (-9.45) | 76.77 (-3.52) | | | Reflexion-v2 | 90.31 (+7.37) | 82.72 (+4.8) | 76.22 (+21.07) | 92.62 (+13.48) | 79.84 (+9.4) | 30.51 (+3.22) | - | | | RARR | 82.37 (-0.57) | 74.04 (-3.88) | 62.04 (+6.89) | 89.93 (+10.79) | 73.36 (+2.92) | 28.42 (+1.13) | 78.58 (-1.71) | | | RATT | 75.92 (-7.02) | 78.39 (+0.47) | 64.99 (+9.84) | 88.27 (+9.13) | 73.66 (+3.22) | 37.87 (+10.58) | 80.56 (+0.27) | | | CRITIC | 82.49 (-0.45) | - | - | 82.72 (+3.58) | - | - | - | | | GPT-4o | Base | 89.16 | 80.65 | 65.82 | 91.15 | 69.54 | 22.49 | 77.04 | | CoT | 91.86 (+2.7) | 81.68 (+1.03) | 61.45 (-4.37) | 97.81 (+6.66) | 73.46 (+3.92) | 13.75 (-8.74) | 64.58 (-12.46) | | | Reflexion-v2 | 91.17 (+2.01) | 84.23 (+3.58) | 77.01 (+11.19) | 94.13 (+2.98) | 77.65 (+8.11) | 19.47 (-3.02) | - | | | RARR | 90.89 (+1.73) | 84.87 (+4.22) | 82.13 (+16.31) | 95.34 (+4.19) | 81.61 (+12.07) | 35.69 (+13.2) | 94.09 (+17.05) | | | RATT | 90.84 (+1.68) | 80.49 (-0.16) | 77.23 (+11.41) | 98.73 (+7.58) | 73.48 (+3.94) | 24.95 (+2.46) | 86.04 (+9.0) | | | CRITIC | 91.08 (+1.92) | - | - | 97.44 (+6.29) | - | - | - | | | QWQ-32B | Base | 62.43 | 82.78 | 52.42 | 63.41 | 73.78 | 10.85 | 19.86 | | CoT | 65.86 (+3.43) | 75.23 (-7.55) | 62.43 (+10.01) | 81.41 (+18.0) | 75.62 (+1.84) | 11.31 (+0.46) | 14.19 (-5.67) | | | Reflexion-v2 | 83.18 (+20.75) | 81.71 (-1.07) | 73.55 (+21.13) | 83.75 (+20.34) | 73.56 (-0.22) | 23.12 (+12.27) | - | | | RARR | 88.28 (+25.85) | 85.03 (+2.25) | 72.67 (+20.25) | 86.33 (+22.92) | 82.97 (+9.19) | 31.66 (+20.81) | 34.92 (+15.06) | | | RATT | 66.19 (+3.76) | 76.78 (-6.0) | 50.88 (-1.54) | 72.85 (+9.44) | 66.12 (-7.66) | 15.62 (+4.77) | 22.71 (+2.85) | | | CRITIC | 79.22 (+16.79) | - | - | 90.83 (+27.42) | - | - | - | | | DeepSeek-V3 | Base | 89.29 | 83.35 | 74.79 | 95.12 | 85.02 | 31.35 | 91.67 | | CoT | 90.08 (+0.79) | 80.08 (-3.27) | 72.67 (-2.12) | 98.13 (+3.01) | 73.73 (-11.29) | 23.91 (-7.44) | 80.92 (-10.75) | | | Reflexion-v2 | 84.84 (-4.45) | 84.38 (+1.03) | 68.47 (-6.32) | 96.28 (+1.16) | 82.62 (-2.4) | 27.72 (-3.63) | - | | | RARR | 88.99 (-0.3) | 84.15 (+0.8) | 74.28 (-0.51) | 94.93 (-0.19) | 87.55 (+2.53) | 40.31 (+8.96) | 82.41 (-9.26) | | | RATT | 84.93 (-4.36) | 86.53 (+3.18) | 78.19 (+3.4) | 92.47 (-2.65) | 86.57 (+1.55) | 36.14 (+4.79) | 92.55 (+0.88) | | | CRITIC | 91.72 (+2.43) | - | - | 94.37 (-0.75) | - | - | - | |