2505.23480v1

# Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt > Corresponding Authors. Abstract Reasoning Large Language Models (RLLMs) have demonstrated impressive performance on complex tasks, largely due to the adoption of Long Chain-of-Thought (Long CoT) reasoning. However, they often exhibit overthinking — performing unnecessary reasoning steps even after arriving at the correct answer. Prior work has largely focused on qualitative analyses of overthinking through sample-based observations of long CoTs. In contrast, we present a quantitative analysis of overthinking from the perspective of self-doubt, characterized by excessive token usage devoted to re-verifying already-correct answer. We find that self-doubt significantly contributes to overthinking. In response, we introduce a simple and effective prompting method to reduce the model’s over-reliance on input questions, thereby avoiding self-doubt. Specifically, we first prompt the model to question the validity of the input question, and then respond concisely based on the outcome of that evaluation. Experiments on three mathematical reasoning tasks and four datasets with missing premises demonstrate that our method substantially reduces answer length and yields significant improvements across nearly all datasets upon 4 widely-used RLLMs. Further analysis demonstrates that our method effectively minimizes the number of reasoning steps and reduces self-doubt. Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt Keqin Peng 1, Liang Ding 2 thanks: Corresponding Authors., Yuanxin Ouyang 1, Meng Fang 3, Dacheng Tao 4 1 Beihang University 2 The University of Sydney 3 University of Liverpool 4 Nanyang Technological University keqin.peng@buaa.edu.cn, liangding.liam@gmail.com 1 Introduction <details> <summary>2505.23480v1/x1.png Details</summary>  ### Visual Description # Technical Document Extraction: Pie Chart Analysis ## Image Description The image contains three pie charts arranged horizontally, each representing different datasets. The charts are labeled as follows: 1. **GSM8K** 2. **GSM8K-Zero** 3. **Math-500** Each chart uses three color-coded segments (orange, blue, green) to represent categorical data. No explicit legend box is present; color associations are embedded within the chart segments. --- ## Chart 1: GSM8K ### Labels and Data Points - **OT w/o SD**: 72.0% (orange) - **SD**: 8.8% (blue) - **NOT**: 19.2% (green) ### Spatial Grounding - Legend colors are embedded in segment colors: - Orange = OT w/o SD - Blue = SD - Green = NOT ### Trend Verification - **Dominant Category**: OT w/o SD (72.0%) significantly outweighs other categories. - **Secondary Category**: NOT (19.2%) is twice the size of SD (8.8%). --- ## Chart 2: GSM8K-Zero ### Labels and Data Points - **OT w/o SD**: 56.8% (orange) - **SD**: 35.5% (blue) - **NOT**: 7.7% (green) ### Spatial Grounding - Legend colors are embedded in segment colors: - Orange = OT w/o SD - Blue = SD - Green = NOT ### Trend Verification - **Balanced Distribution**: OT w/o SD (56.8%) and SD (35.5%) are closer in magnitude compared to GSM8K. - **Minor Category**: NOT (7.7%) remains the smallest segment. --- ## Chart 3: Math-500 ### Labels and Data Points - **SD**: 59.5% (blue) - **OT w/o SD**: 23.2% (orange) - **NOT**: 17.3% (green) ### Spatial Grounding - Legend colors are embedded in segment colors: - Blue = SD - Orange = OT w/o SD - Green = NOT ### Trend Verification - **Reversed Dominance**: SD (59.5%) is the largest category, surpassing OT w/o SD (23.2%). - **NOT** (17.3%) is the smallest but larger than in GSM8K-Zero. --- ## Cross-Chart Analysis 1. **Color Consistency**: - **Orange** consistently represents "OT w/o SD" across all charts. - **Blue** represents "SD" in GSM8K-Zero and Math-500 but "SD" in GSM8K. - **Green** consistently represents "NOT" across all charts. 2. **Data Trends**: - **GSM8K**: High dominance of "OT w/o SD" (72.0%). - **GSM8K-Zero**: More balanced between "OT w/o SD" (56.8%) and "SD" (35.5%). - **Math-500**: "SD" dominates (59.5%), with "OT w/o SD" reduced to 23.2%. 3. **NOT Category**: - Largest in GSM8K (19.2%). - Smallest in GSM8K-Zero (7.7%). - Intermediate in Math-500 (17.3%). --- ## Notes - No additional languages or textual elements are present. - All percentages sum to 100% for each chart, confirming data integrity. - No axis titles or numerical scales are present, as these are pie charts. </details> Figure 1: The proportion of overthinking and self-doubt across three different datasets with Deepseek-Deepseek-R1-Distill-Qwen-32B. SD: Self-Doubt, OT w/o SD: Overthinking without Self-Doubt, NOT: Non-Overtinking. Recent advances in reasoning models, such as OpenAI’s o1 Jaech et al. (2024) and DeepSeekR1 Guo et al. (2025), have demonstrated significant progress in complex reasoning capabilities, particularly in domains such as mathematical reasoning Wei et al. (2022). A central factor in their success lies in the application of long chain-of-thought (Long CoT), a kind of in-context learning Dong et al. (2024); Peng et al. (2024), which enhances reasoning abilities by a more detailed, iterative process of exploration and reflection Chen et al. (2025). However, while long CoT reasoning significantly boosts accuracy, step-by-step thinking mechanisms also lead to lengthy output responses, resulting in substantial computational overhead and increased reasoning time Chen et al. (2024); Sui et al. (2025). Previous studies have predominantly employed qualitative analyses of the overthinking phenomenon by examining the observed reasoning paths. For example, Chen et al. (2024) found that Reasoning models will generate hundreds of response tokens even for simple questions like "2+3=?"; Fu et al. (2025) identify self-doubt, which repeats check and verify when obtaining the correct information, as one major source of overthinking; Fan et al. (2025) found that an ill-posed question with missing premises will exacerbate overthinking. To conduct a more in-depth analysis of the phenomenon of overthinking, we performed a quantitative analysis of overthinking from the perspective of self-doubt. Specifically, we categorize the generated reasoning paths into three classes: overthinking with Self-Doubt (SD), overthinking without Self-Doubt (OT w/o SD), and non-overthinking (NOT). We then employ a third-party LLM to evaluate these reasoning paths. Experimental results reveal that overthinking is widespread, and self-doubt significantly contributes to overthinking, especially in complex tasks. Inspired by the social comparison theory Festinger (1957), we attribute the self-doubt observed in long CoT to the model’s excessive deference to the user, manifested as an overreliance on user input and a strong desire to please, which makes it overly cautious and reluctant to challenge user inputs. In response, we introduce a simple and effective prompting method to mitigate the overthinking phenomenon in Long CoT. Specifically, we first prompt the model to assess the validity of the input question, thereby engaging its critique abilities and boosting its confidence; We then instruct the model to provide a concise response based on the outcome of this evaluation. Experiments on three mathematical reasoning tasks demonstrate that our method substantially reduces answer length and yields significant improvements across nearly all datasets upon 4 widely-used RLLMs. Further experiments on four datasets with missing premises demonstrate that our method significantly reduces token consumption and more effectively identifies cases where premises are missing. 2 Revisiting Overthinking Recently, overthinking in the Long Chain-of-Thought has received widespread attention due to the huge success of RLLM. Many researchers Chen et al. (2025) have explored the causes and characteristics of overthinking through qualitative analysis of reasoning paths. In this section, we adopt the lens of self-doubt to quantitatively investigate its influence on the overthinking phenomenon. To systematically assess the impact of self-doubt on overthinking, we conduct experiments across diverse datasets using a widely used reasoning model. Specifically, we generate the reasoning path for three mathematical reasoning tasks, GSM8K Cobbe et al. (2021), GSM8K-Zero Chiang and Lee (2024), and Math-500 Lightman et al. , with reasoning model Deepseek-R1-Distill-Qwen-32B Guo et al. (2025). For evaluation, we categorize each reasoning path into one of three groups: - Overthinking with Self-Doubt (SD): The model’s answer includes unnecessarily long reasoning and repeatedly verifies information that is already correct. - Overthinking without Self-Doubt (OT w/o SD): The answer is excessively long but does not involve redundant verification. - Non-Overthinking (NOT): The answer avoids both unnecessary reasoning and redundant verification. We employ Qwen2.5-72B-Instruct Yang et al. (2024) as an automated evaluator. The full evaluation prompt is provided in Appendix A. Figure 1 reveals that the overthinking phenomenon is prevalent in Long CoT reasoning. Specifically, the proportion of overthinking samples (comprising both self-doubt and overthinking, excluding self-doubt) exceeds 80% across all three datasets, even reaching as high as 92% in the GSM8K-Zero dataset. Furthermore, the figure highlights that self-doubt is a major contributor to overthinking, particularly in the complex math-500 task, where it accounts for nearly 60% of cases, underscoring the importance of self-doubt in the overthinking process. | DS Distill 14B | baseline | 591 | 92.4 | 1070 | 80.6 | 3095 | 92.8 | 1585 | 88.6 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Ours | 426 | 92.4 | 392 | 86.2 | 1922 | 92.2 | 913 | 90.3 | | | DS Distill 32B | baseline | 551 | 94.2 | 613 | 72.9 | 3058 | 92.4 | 1407 | 86.5 | | Ours | 562 | 95.2 | 495 | 89.6 | 2141 | 93.6 | 1066 | 92.8 | | | DS Distill 70B | baseline | 555 | 94.2 | 604 | 74.8 | 2847 | 92.4 | 1335 | 87.1 | | Ours | 434 | 95.1 | 438 | 89.5 | 1927 | 91.6 | 933 | 92.1 | | | Qwen3-32B | baseline | 1681 | 95.8 | 1854 | 91.0 | 3970 | 97.2 | 2502 | 94.7 | | Ours | 631 | 94.6 | 542 | 92.0 | 2981 | 96.8 | 1385 | 94.5 | | Table 1: Performance of our method across three well-defined models on four reasoning models. The best results are in red. On average, we reduce reasoning length by -37.1% while improving accuracy by +3.6%. 3 Method Social comparison theory Festinger (1957) holds that when someone relies heavily on others’ opinions or approval to judge themselves, it can lead to self-doubt if those comparisons are unfavorable or inconsistent. Similarly, large language models, due to their heavy dependence on human evaluations, continuously engage in self-doubt and self-verification to produce answers that satisfy users, which can lead to the phenomenon of overthinking. To mitigate this, we propose a simple yet effective prompting strategy to elicit the model’s critical ability, thereby alleviating overthinking. Specifically, we first prompt the model to assess the validity of the input query. Based on the evaluation outcome, we then instruct the model to provide a concise answer or indicate what information is missing. The full templates of prompts are presented in Table 2. | Baseline \hdashline Ours | "role": "user", "content": [Question] "role": "user", "content": [Question] Before reasoning deeply, check whether all necessary information is available. If any key data is missing or ambiguous, explicitly state that first; otherwise, answer it with the minimum number of tokens required. | | --- | --- | Table 2: The prompt template of our method. 4 Experimental Setup Models and Baselines. We perform experiments across different sizes of reasoning models, including DeepSeek-R1-Distill-Qwen-14B, DeepSeek-R1-Distill-Qwen-32B, DeepSeek-R1-Distill-Llama-70B Guo et al. (2025) and Qwen3-32B Yang et al. (2025), which are all widely-used RLLMs. For the baseline, we utilize the default chat template without any additional information. We set the temperature to 0 by default to obtain deterministic output, as proposed in Peng et al. (2023). Datasets. We conduct a systematic study across three mathematical reasoning tasks, including GSM8K Cobbe et al. (2021), GSM8K-Zero Chiang and Lee (2024), and Math-500 Lightman et al. . And GSM8K-Zero, whose answer can be easily generated by LLMs, specifically targets the analysis of over-reasoning and redundancy in LLM-generated outputs. We also evaluate our method in ill-posed questions with missing premises (MiP), which will exacerbate Overthinking Fan et al. (2025), and we follow the setting of Fan et al. (2025) using four MiP datasets: MiP-Formula, MiP-SVAMP, MiP-GSM8K, and MiP-Math. Evaluation Metrics. To evaluate the answer quality in different datasets, we follow the setting of Fan et al. (2025) to adopt both statistic metric and LLM-based metric Zheng et al. (2023), i.e., LLM-as-a-Judge. Specifically, for MiP datasets, we use Abstain Rate, defined as the proportion of answers in which the model successfully identifies a missing premise. For the well-defined questions, we employ Accuracy, the proportion of answers where the model produces a definitive response that aligns with the reference answer. Additionally, we report the Response Length, capturing both the reasoning process and the final answer, to assess tendencies toward overthinking. Evaluation protocols are provided in Appendix A. | DS Distill 14B | baseline | 5740 | 54.0 | 1695 | 76.0 | 1552 | 32.5 | 6106 | 54.9 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Ours | 735 | 96.0 | 413 | 96.7 | 327 | 73.7 | 2406 | 86.3 | | | DS Distill 32B | baseline | 5633 | 66.0 | 1060 | 78.0 | 1249 | 38.1 | 6414 | 67.3 | | Ours | 1060 | 92.0 | 451 | 97.0 | 446 | 79.6 | 2789 | 76.9 | | | DS Distill 70B | baseline | 4761 | 52.0 | 812 | 80.3 | 973 | 28.9 | 6286 | 61.5 | | Ours | 529 | 88.0 | 428 | 96.3 | 415 | 78.2 | 2728 | 68.6 | | | Qwen3-32B | baseline | 5039 | 36.0 | 1771 | 82.0 | 2957 | 43.0 | 6081 | 61.5 | | Ours | 631 | 98.0 | 319 | 97.0 | 530 | 81.9 | 3669 | 82.4 | | Table 3: Performance of our method across four Missing Premise (MiP) datasets on four reasoning models. The best results are in red. We can see that our method largely improves the abstain rate and significantly reduces the response length. 5 Main Results We demonstrate the effectiveness of our method in 3 well-defined mathematical reasoning tasks and four Missing Premise (MiP) datasets across four RLLMs. We summarize the results in Table 1 and Table 3. Based on the results, we can find that: Our method significantly reduces token consumption while maintaining accuracy in well-defined mathematical reasoning tasks. Results on 1 show that our method can consistently significantly reduce token consumption across different datasets and different reasoning models. Specifically, our method can reduce token consumption by 1/3 on average compared to the baseline. Especially in the GSM8K-Zero dataset under Qwen3-32B, our method brings 2/3 token consumption reduction, showing that our method can effectively alleviate the overthinking phenomenon. Furthermore, we can notice that our method can maintain the accuracy on almost all datasets and even bring improvements on some datasets. In particular, on GSM8K-Zero, our method brings +16.7% and +14.7% improvement with DeepSeek-R1-Distill-Qwen-32B and DeepSeek-R1-Distill-Llama-70B, respectively. We attribute these gains primarily to the correction of errors caused by overthinking. Our method brings consistent and significant improvement and token reduction in ill-posed questions with missing premises (MiP). To address the issue of overthinking caused by missing premises, we evaluate our method on the MiP dataset. As shown in Table 3, our method consistently outperforms the baseline across all datasets in terms of both response length and abstain rate. Specifically, it reduces token consumption by at least 50% on nearly all datasets. It’s worth highlighting that our method reduces token consumption by more than 80% on the MiP-Formula dataset, from more than 5,000 tokens to about 1,000. Additionally, our approach significantly improves abstain rates, achieving gains of nearly 40% on MiP-Formula and MiP-GSM8K. These results highlight our method’s effectiveness in reducing overthinking while enhancing the model’s critique capability. 6 Analysis Our method can significantly reduce the number of reasoning steps in Long Chain-of-Thought. Self-doubt often causes the model to generate more steps in its thought process. To validate whether our method can reduce self-doubt, we first compare our method with the baseline from thinking patterns. Specifically, we split the generated text with \n\n to obtain the number of reasoning steps. Figure 2 represents the average thinking steps across three datasets with DeepSeek-R1-Distill-Qwen-32B. From Figure 2, we can see that our method can consistently reduce the thinking steps, demonstrating that our method can effectively alleviate overthinking by avoiding self-doubt. <details> <summary>2505.23480v1/x2.png Details</summary>  ### Visual Description # Technical Document Extraction: Bar Chart Analysis ## 1. Chart Title and Labels - **Title**: "Baseline vs Ours Performance on Different Datasets" - **X-Axis Labels**: - GSM8K - GSM8K-Zero - MATH-500 - **Y-Axis Label**: Numerical scale (0.0 to 80.0, increments of 20.0) - **Legend**: - **Blue**: Baseline - **Red**: Ours ## 2. Data Points and Trends ### Spatial Grounding of Data Series - **Legend Position**: Top-center (x: [0.4, 0.6], y: [0.95, 1.0]) - **Color Consistency Check**: - Blue bars correspond to "Baseline" - Red bars correspond to "Ours" ### Component Isolation #### Main Chart Region | Dataset | Baseline (Blue) | Ours (Red) | Visual Trend Description | |---------------|-----------------|------------|--------------------------| | **GSM8K** | ~12.0 | ~10.0 | Baseline > Ours | | **GSM8K-Zero**| ~13.0 | ~7.0 | Baseline > Ours | | **MATH-500** | ~78.0 | ~50.0 | Baseline > Ours | ### Trend Verification - **Baseline Series**: - Slopes upward across datasets (12.0 → 13.0 → 78.0) - Sharp increase observed between GSM8K-Zero and MATH-500 - **Ours Series**: - Peaks at MATH-500 (50.0) - Declines from GSM8K-Zero to MATH-500 (7.0 → 50.0) ## 3. Axis Markers - **Y-Axis Ticks**: 0.0, 20.0, 40.0, 60.0, 80.0 - **X-Axis Ticks**: Centered under each dataset label ## 4. Structural Analysis - **Header**: Legend (blue/red labels) - **Main Chart**: Three grouped bars per dataset - **Footer**: No additional text or components ## 5. Critical Observations 1. **Performance Gap**: - Ours underperforms Baseline in GSM8K and GSM8K-Zero - Ours achieves 50.0 on MATH-500 vs Baseline's 78.0 2. **Scale Utilization**: - Y-axis max (80.0) only reached by Baseline on MATH-500 - Ours never exceeds 50.0 across all datasets ## 6. Missing Elements - No gridlines or error bars present - No textual annotations on bars or axes ## 7. Language Declaration - All text in English. No non-English content detected. </details> Figure 2: The average thinking steps across three mathematical reasoning tasks with DeepSeek-R1-Distill-Qwen-32B. | | GSM8K | GSM8K-Zero | MATH-500 | | --- | --- | --- | --- | | baseline | 5.9 | 13.5 | 62 | | Ours | 17.2 | 10.1 | 38.2 | | \hdashline $\Delta$ | -11.3 | +3.5 | 23.8 | Table 4: The ratio of self-doubt across three datasets. Our method can effectively reduce the self-doubt in overthinking, especially in complex datasets. To verify whether our method effectively reduces self-doubt in overthinking, we report the performance using the evaluation metric described in Section 2. As shown in Table 4, our method reduces the self-doubt ratio in two out of three datasets, with a particularly notable 23.8% reduction on the complex dataset MATH-500, demonstrating our method can effectively reduce self-doubt. Regarding the increase in self-doubt observed in GSM8K, we suspect it stems from the early verification of the question’s correctness and the short reasoning path. 7 Related Works Chain-of-Thought. Chain-of-Thought (CoT) prompting elicits multi-step reasoning by having LLMs generate intermediate “thought” traces, substantially improving performance on arithmetic, commonsense, and symbolic benchmarks. Early work showed CoT can unlock latent reasoning capabilities in models like GPT-3 and PaLM Wei et al. (2022). Many follow-up works have shown the effectiveness of carefully-designed CoT prompting on various tasks Zhong et al. (2024); Zhang et al. (2025), and recent surveys have catalogued extensions to longer, more detailed thought chains Chen et al. (2025). Overthinking in Long CoT. Despite boosted accuracy, long CoTs often induce overthinking problem, where models continue producing needless steps or repeat verifications after arriving at the correct answer. Chen et al. (2024); Sui et al. (2025) provide qualitative analyses of these redundant patterns, while Fan et al. (2025) demonstrate that missing premises exacerbate overthinking in ill-posed questions. Fu et al. (2025) attribute much of this to self-doubt —models probing their own certainty—and propose certainty-probing probes to trim reasoning length. Prompting and Reinforcement for Efficient Reasoning. To curb overthinking, several works have explored targeted interventions. Guo et al. (2025) use reinforcement-learning to incentivize concise reasoning, Zheng et al. (2023) introduce LLM-as-a-Judge for automated quality assessment, and Fan et al. (2025) design prompts to detect missing information before reasoning. Our method builds on these by adding a lightweight input-validation step that directly mitigates self-doubt and over-reliance on user queries. 8 Conclusion To further explore overthinking in reasoning large language models, we conduct a quantitative analysis of overthinking from the perspective of self-doubt. We find that self-doubt is one of the main causes of overthinking. In response, we propose a simple and effective prompt method to reduce self-doubt by eliciting the critique ability. Our experiments across various datasets and models demonstrate that our method can effectively reduce token consumption and improve performance. Limitations Despite the encouraging results, our study has several important limitations that future work should address: 1) Domain and Task Scope: We evaluate our prompting strategy solely on three mathematical reasoning benchmarks (GSM8K, GSM8K-Zero, Math-500) and four missing-premise (MiP) datasets. It remains unclear whether the same reduction in overthinking and self-doubt would hold for other reasoning domains (e.g., commonsense QA, multi-step planning, or multimodal tasks). 2) Reliance on Automated Evaluation: Our quantitative analysis of overthinking and self-doubt depends on an LLM-based judge (Qwen2.5-72B-Instruct) and a simple paragraph-segmentation heuristic to categorize reasoning paths. This introduces potential biases or misclassifications, since the evaluator itself may misunderstand nuanced reasoning patterns or miss subtle verification steps. Addressing these limitations will be crucial for deploying more efficient and trustworthy chain-of-thought systems in real-world applications. References - Chen et al. (2025) Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567. - Chen et al. (2024) Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187. - Chiang and Lee (2024) Cheng-Han Chiang and Hung-Yi Lee. 2024. Over-reasoning and redundant calculation of large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 161–169. - Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. - Dong et al. (2024) Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A survey on in-context learning. In EMNLP. - Fan et al. (2025) Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. 2025. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514. - Festinger (1957) Leon Festinger. 1957. Social comparison theory. Selective Exposure Theory. - Fu et al. (2025) Yichao Fu, Junda Chen, Yonghao Zhuang, Zheyu Fu, Ion Stoica, and Hao Zhang. 2025. Reasoning without self-doubt: More efficient chain-of-thought through certainty probing. In ICLR 2025 Workshop on Foundation Models in the Wild. - Guo et al. (2025) Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. - Jaech et al. (2024) Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720. - (11) Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations. - Peng et al. (2024) Keqin Peng, Liang Ding, Yancheng Yuan, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2024. Revisiting demonstration selection strategies in in-context learning. In ACL. - Peng et al. (2023) Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. Towards making the most of ChatGPT for machine translation. In Findings of EMNLP 2023. - Sui et al. (2025) Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and 1 others. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419. - Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. NIPS. - Yang et al. (2025) An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. - Yang et al. (2024) An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. - Zhang et al. (2025) Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao. 2025. Intention analysis makes llms a good jailbreak defender. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2947–2968. - Zheng et al. (2023) Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. NIPS, 36:46595–46623. - Zhong et al. (2024) Qihuang Zhong, Kang Wang, Ziyang Xu, Juhua Liu, Liang Ding, and Bo Du. 2024. Achieving> 97% on gsm8k: Deeply understanding the problems makes llms better solvers for math word problems. arXiv preprint arXiv:2404.14963. Appendix A Prompt Template for Evaluation As we need LLM-as-a-judge to evaluate the quality of the generation of the models in various experiments in this study. In this section, we showcase the prompt template we use for each kind of evaluation. To evaluate self-doubt in long chain-of-thought (CoT) responses, we adopt the template shown in Table 5. Since the generated long CoTs are often lengthy and self-doubt typically emerges later in the response, we split the output into two segments with the same reasoning steps using the delimiter \n\n. We then use the second segment to assess the proportion of self-doubt in the dataset. For the evaluation of the models’ answer accuracy and abstain rate, we follow Fan et al. (2025) to adopt the following prompt templates designed for ’paired’ and ’non-paired’ data, respectively. Since RLLMs often output an additional \n\n at the end of response, we also take the last two paragraph segmented by \n\n to avoid passing in an empty string. | Prompt Template for Response Evaluation of MiP Datasets | | --- | | System: You are a helpful assistant that evaluates the quality of a model’s answer. You will be given a question and a model’s answer. You need to evaluate the correctness of the model’s answer. If the answer explicitly says that the condition of the question is insufficient, you should return 0. If the model provides an answer that is a number or formula with variables, you should return 1. Please only return the number, no other text. | | User: Model answer: [model answer short]\n | | Prompt Template for Response Evaluation of well-defined Datasets | | System: You are a helpful assistant that evaluates the quality of a model’s answer. You will be given a question and a model’s answer. You need to evaluate the correctness of the model’s answer. If the model output says that the condition of the question is insufficient, you should return 0. Otherwise, if the model give a clear answer and matches the reference answer, you should return 1. If the model’s answer does not match the reference answer, you should return 2. Please only return the number, no other text. | | User: Model answer: [model answer]\nReference answer: [reference answer]\n | | Prompt Template for Self-Doubt Evaluation | | System: You are a helpful assistant tasked with evaluating the quality of a model’s answer. You will be given a question, a reference answer and the model’s answer. Evaluate the model’s answer according to the following criteria: - Return 0 if the answer includes unnecessarily long reasoning and repeats verification of already correct answer; - Return 1 if the answer is unnecessarily long but does not include redundant verification; - Return 2 if the answer avoids both unnecessary reasoning and redundant verification. Return only the appropriate number. Do not include any explanation or extra text. User: Model answer: [model answer]\nReference answer: [reference answer]\n | Table 5: Prompt Template for Response Evaluation. Appendix B Case Study We present a case on GSM8K-Zero with DeepSeek-R1-Distill-Qwen-32B in Table 6, we can notice that baseline method will repeat reflection and our method can largely reduce the answer length. | Col1 | | --- | | Question: Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How many minutes of babysitting did she do yesterday? | | Baseline: "Okay, so I have this problem here: Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How many minutes of babysitting did she do yesterday? Hmm, wait a second. That seems a bit confusing. Let me read it again to make sure I understand."\n\n…That seems straightforward, but maybe I’m missing something. Perhaps it’s a trick question or there’s more to it….So, maybe it’s just a simple question where the answer is 50 minutes….So, I think the answer is 50 minutes.\n\nBut just to be thorough, let me consider if there’s another angle….Wait, perhaps the question is trying to set up a follow-up question, like how much she earned, but the way it’s phrased is confusing. The question gives the time she worked and asks for the same information, so the answer is 50 minutes. (476 tokens) | | Ours: Okay, so I need to figure out how many minutes Weng did babysitting yesterday. The problem says she earns $12 an hour and that she just did 50 minutes of babysitting. Wait, that seems a bit confusing. Let me read it again….So, is the answer just 50 minutes? That seems too simple, but maybe that’s the case….but the way it’s phrased, it’s just asking for the minutes she worked, which is given as 50. So, I think the answer is 50 minutes. (304 tokens.) | Table 6: A case study on GSM8K-Zero with DeepSeek-R1-Distill-Qwen-32B. We can see that our method reduce the token consumption.