2601.09088

# Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning **Authors**: Shaotian Yan, Kaiyuan Liu, Chen Shen, Bing Wang, Sinan Fan, Jun Zhang, Yue Wu, Zheng Wang, Jieping Ye, Alibaba Cloud Computing Y¿ X ## Abstract In this report, we introduce DASD-4B-Thinking, a lightweight yet highly capable, fully open-source reasoning model. It achieves state-of-the-art performance among open-source models of comparable scale across challenging benchmarks in mathematics, scientific reasoning, and code generation—even outperforming several larger models (e.g., 32B-scale). We begin by critically reexamining a widely adopted distillation paradigm in the community: supervised fine-tuning (SFT) on teacher-generated responses, also known as sequence-level distillation. Although a series of recent works following this scheme have demonstrated remarkable efficiency and strong empirical performance, they are primarily grounded in the SFT perspective. Consequently, these approaches focus predominantly on designing heuristic rules for SFT data filtering, while largely overlooking the core principle of distillation itself—enabling the student model to learn the teacher’s full output distribution so as to inherit its generalization capability. Specifically, we identify three critical limitations in current practice: i) Inadequate representation of the teacher’s sequence-level distribution; ii) Misalignment between the teacher’s output distribution and the student’s learning capacity; and iii) Exposure bias arising from teacher-forced training versus autoregressive inference. In summary, these shortcomings reflect a systemic absence of explicit teacher–student interaction throughout the distillation process, leaving the essence of distillation underexploited. To address these issues, we propose several methodological innovations that collectively form an enhanced sequence-level distillation training pipeline. Remarkably, DASD-4B-Thinking obtains competitive results using only 448K training samples—an order of magnitude fewer than those employed by most existing open-source efforts. To support community research, we publicly release our models and the training dataset. | <details> <summary>x1.png Details</summary>  ### Visual Description Icon/Small Image (26x24) </details> Models & Datasets | <details> <summary>x2.png Details</summary>  ### Visual Description Icon/Small Image (42x24) </details> Models & Datasets | <details> <summary>x3.png Details</summary>  ### Visual Description Icon/Small Image (25x24) </details> Code | | --- | --- | --- | footnotetext: * Core contributors. † Project lead. <details> <summary>x4.png Details</summary>  ### Visual Description ## Bar Chart Comparison: Model Accuracy Across Four Benchmarks ### Overview The image displays a 2x2 grid of four bar charts, each comparing the accuracy performance of various AI models on a specific benchmark. The charts share a common legend and visual style, using grouped bars to represent three distinct model categories. The overall purpose is to benchmark the performance of a model labeled "Ours" against other models, categorized by their training data openness. ### Components/Axes * **Common Legend (Top Center):** Positioned above the charts, it defines three categories: * **Ours:** Represented by orange bars. * **Open-Weights Only:** Represented by gray bars. * **Open-Weights & Open-Data:** Represented by beige/light tan bars. * **Chart Structure:** Each of the four charts has: * **Y-axis:** Labeled "Accuracy". The scale varies per chart (e.g., 65-95 for AIME24, 50-90 for AIME25). * **X-axis:** Lists the names of specific AI models being compared. * **Bars:** Grouped bars for each model, colored according to the legend. The exact accuracy value is printed atop each bar. ### Detailed Analysis **1. Top-Left Chart: AIME24** * **Y-axis Scale:** 65 to 95. * **Models & Data Points (from left to right):** * `DASD-4B-ThinkIng` (Orange - "Ours"): **88.5** * `AIM-Thinking-v1-32B` (Gray - "Open-Weights Only"): **85.3** * `OpenReasoning-Nem...-7B` (Beige - "Open-Weights & Open-Data"): **84.7** * `Qwen3-32B` (Gray): **81.4** * `POLARIS-4B-Preview` (Gray): **81.2** * `GLM-Z1-32B` (Gray): **80.8** * `Qwen3-4B` (Gray): **73.8** * `OpenThoughts-3-7B` (Beige): **69.0** * **Trend:** The "Ours" model (`DASD-4B-ThinkIng`) achieves the highest accuracy. Performance generally decreases moving right, with the "Open-Weights & Open-Data" models (`OpenReasoning-Nem...-7B`, `OpenThoughts-3-7B`) showing a wider spread in results. **2. Top-Right Chart: AIME25** * **Y-axis Scale:** 50 to 90. * **Models & Data Points (from left to right):** * `DASD-4B-ThinkIng` (Orange - "Ours"): **83.3** * `Qwen3-4B-Thinking` (Gray): **81.3** * `POLARIS-4B-Preview` (Gray): **79.4** * `DeepSeek-R1-0528-Qwen3-8B` (Beige): **78.2** * `AIM-Thinking-v1-32B` (Gray): **76.3** * `GLM-Z1-32B` (Gray): **74.4** * `Qwen3-32B` (Gray): **63.6** * `OpenThoughts-3-7B` (Beige): **53.3** * **Trend:** Similar to AIME24, the "Ours" model leads. There is a significant drop-off for the last two models (`Qwen3-32B`, `OpenThoughts-3-7B`), with the latter showing the lowest score across all charts. **3. Bottom-Left Chart: LiveCodeBench v5** * **Y-axis Scale:** 50 to 75. * **Models & Data Points (from left to right):** * `DASD-4B-ThinkIng` (Orange - "Ours"): **69.3** * `DeepSeek-R1-0528-Qwen3-8B` (Beige): **60.5** * `GLM-Z1-32B` (Gray): **59.1** * `Qwen3-32B` (Gray): **57.5** * `AceReason-Nemotron-1.1-7B` (Beige): **57.2** * `Qwen3-4B` (Gray): **54.2** * `OpenThoughts-3-7B` (Beige): **51.7** * **Trend:** The "Ours" model has a substantial lead of nearly 9 percentage points over the next best model. The performance gradient is more gradual here compared to the AIME charts. **4. Bottom-Right Chart: GPQA** * **Y-axis Scale:** 50 to 75. * **Models & Data Points (from left to right):** * `DASD-4B-ThinkIng` (Orange - "Ours"): **68.4** * `Qwen3-4B-Thinking` (Gray): **65.8** * `Doubao-1.5-thinking-pro` (Gray): **62.1** * `OpenReasoning-Nem...-7B` (Beige): **61.4** * `DeepSeek-R1-0528-Qwen3-8B` (Beige): **61.1** * `GLM-Z1-9B` (Gray): **58.5** * `Doubao-1.5-thinking-lite` (Gray): **56.1** * `OpenThoughts-3-7B` (Beige): **53.7** * **Trend:** The "Ours" model again leads, but the margin is smaller (~2.6 points) compared to the second-place `Qwen3-4B-Thinking`. The chart shows a relatively steady decline in accuracy across the listed models. ### Key Observations 1. **Consistent Leader:** The model labeled "Ours" (`DASD-4B-ThinkIng`) is the top performer on all four benchmarks presented. 2. **Category Performance:** Models in the "Open-Weights Only" (gray) category are the most numerous and occupy a wide range of positions, from near the top to the bottom. The "Open-Weights & Open-Data" (beige) models are fewer and often appear in the lower half of the rankings, with `OpenThoughts-3-7B` consistently scoring the lowest in each chart it appears in. 3. **Benchmark Variability:** The absolute accuracy scores and the performance gaps between models vary significantly across benchmarks. For example, the lead of "Ours" is most pronounced in LiveCodeBench v5 and smallest in GPQA. 4. **Model Name Consistency:** Some models appear across multiple benchmarks (e.g., `DASD-4B-ThinkIng`, `GLM-Z1-32B`, `Qwen3-32B`, `OpenThoughts-3-7B`), allowing for cross-benchmark performance tracking. ### Interpretation This set of charts serves as a comparative performance report, likely from the creators of the `DASD-4B-ThinkIng` model. The data suggests that their model achieves state-of-the-art or leading accuracy across diverse reasoning and knowledge benchmarks (AIME math, coding, graduate-level QA) when compared to a selection of other open-weight models. The grouping by data openness ("Open-Weights Only" vs. "Open-Weights & Open-Data") implies an investigation into whether access to training data correlates with benchmark performance. The results here do not show a clear advantage for models with open data; in fact, the top-performing "Ours" model is categorized separately, and the open-data models often trail behind. This could indicate that model architecture, training methodology, or data quality (not just openness) are more critical factors for these specific tasks. The consistent placement of `OpenThoughts-3-7B` at the bottom might suggest it is a smaller or less optimized model within this comparison group. The variability in the performance gap (e.g., large in LiveCodeBench, smaller in GPQA) highlights that model strengths are benchmark-dependent, and no single model dominates by an identical margin across all domains. The charts effectively argue for the competitive performance of the "Ours" model within the open-weights landscape as of the data's collection date. </details> Figure 1: Performance of DASD-4B-Thinking on benchmark datasets. All metrics for the comparison models are taken from their official reports. ## 1 Introduction Recently, DeepSeek (Guo et al., 2025) was the first to demonstrate that distillation from powerful teacher models can substantially empower smaller models with reasoning capabilities. Specifically, they curated the reasoning data generated by DeepSeek-R1 and directly fine-tuned several widely used open-source compact models. Owing to the simplicity of this supervised fine-tuning (SFT) approach combined with the favorable deployment and inference efficiency of small models, this work has significantly reinvigorated the community’s interest in exploring distillation-based methods for reasoning enhancement. A series of open-source projects (e.g., OpenR1 (Hugging Face, 2025), OpenThoughts (Guha et al., 2025), a-m-team (Zhao et al., 2025), NVIDIA AceReason (Chen et al., 2025b), NVIDIA OpenMathReasoning (Moshkov et al., 2025), OmniThought (Cai et al., 2025), Light-R1 (Wen et al., 2025), LIMO (Ye et al., 2025), s1 (Muennighoff et al., 2025), DeepMath (He et al., 2025), MiroMind-M1 (Li et al., 2025b), Syntheic-1 (Mattern et al., 2025), NaturalThoughts (Li et al., 2025c), and Sky-T1 (Team, 2025)) have since replicated the DeepSeek-R1 distillation paradigm, dedicating substantial efforts to its faithful reproduction and extension. These efforts typically involve collecting and open-sourcing large-scale corpora of challenging reasoning questions, paired with responses generated by powerful teacher models. These datasets have undergone rigorous quality filtering, including stringent correctness verification (Lei et al., 2025; Wu et al., 2025), preference-based selection prioritizing higher reasoning difficulty or longer output length (Muennighoff et al., 2025; Li et al., 2025c), and diversity-aware curation (Jung et al., 2025; Li et al., 2025a). Subsequently, through SFT on such publicly released reasoning corpora, researchers have obtained distilled models that exhibit strong reasoning capabilities. This paradigm of SFT on teacher-generated responses (Agarwal et al., 2024), i.e., sequence-level distillation (Kim and Rush, 2016), has achieved state-of-the-art or highly competitive performance across diverse domains, including mathematics (Hugging Face, 2025; Chen et al., 2025b; Wen et al., 2025; He et al., 2025; Li et al., 2025b), scientific reasoning (Guha et al., 2025; Li et al., 2025c; Mattern et al., 2025), code generation (Ahmad et al., 2025; Guha et al., 2025; Zhao et al., 2025; Cai et al., 2025), and instruction-following (Bercovich et al., 2025; Zhao et al., 2025). Another paradigm is logit distillation, a classic approach in knowledge distillation (Hinton et al., 2015) that aligns the logit distributions of student and teacher models to better leverage the rich ``dark knowledge'' encoded in the teacher’s outputs. Notably, recent advancements such as Qwen3 (Yang et al., 2025) and Gemma (Kamath et al., 2025) adopt an on-policy variant: they first generate on-policy sequences using the student model and then align the student’s logit distributions with those of the teacher by minimizing the KL divergence. Recently, Thinking Machines Lab (Lu and Lab, 2025) released an open-source implementation of this paradigm. However, beyond the requirement of accessing token-level logits, these methods face significant challenges when the teacher and student employ different tokenizers, as direct logit alignment becomes infeasible due to misaligned output spaces. In this work, we aim to improve the aforementioned paradigm—namely, the sequence-level distillation— as it is simple and efficient, has already inspired substantial community efforts (including the release of a large number of open-source datasets), imposes no arbitrary constraints on the choice of teacher and student model architectures, and does not require access to token-level logits. Consistent with Kim and Rush (2016), we first argue that SFT on teacher-generated data serves as an effective form of distillation, since such data approximately reflects the teacher model’s sequence-level output distribution and thereby aligns the student model with it. However, existing works in this first paradigm are primarily grounded in the SFT perspective; consequently, they focus predominantly on designing heuristic rules to filter SFT data, while largely overlooking the core principle of distillation itself—enabling the student model to learn the teacher’s full output distribution so as to inherit its generalization capability (Hinton et al., 2015). In other words, they lack an explicit mechanism to enforce teacher–student interaction throughout the distillation process, leaving the essence of distillation underexploited. More concretely, such approaches neglect three critical issues: - From the teacher’s perspective: How to better capture and represent the teacher’s sequence-level distribution? Existing works randomly sample response data under certain quality-based filtering rules. Although such responses can serve as an approximation of the teacher’s sequence-level distribution, this strategy often fails to adequately cover the full support of that distribution. As a result, it may suffer from poor mode coverage or overrepresent low-probability or noisy sequences—making learning particularly challenging for smaller or less capable student models. - From the student’s learning perspective: How to address misleading gradients when training on teacher-generated data, and more broadly, what target sequence-level distribution better supports effective learning? Classical knowledge distillation leverages the teacher’s logit distributions to accurately match the student’s predictive distributions over the entire vocabulary at every decoding step, adjusting probabilities up or down as appropriate. In contrast, SFT primarily increases the likelihood of the ground-truth tokens at each prediction position, which can yield misleading gradients (e.g., for tokens the teacher assigns low probabilities but the student assigns high probabilities, SFT pushes those probabilities even higher, driving the student away from the teacher’s distribution). Therefore, identifying a teacher sequence-level distribution that is better aligned with the student model’s learning is of critical importance. - How to mitigate exposure bias caused by training with teacher forcing while evaluating in a free-running, real-world setting? We emphasize that models distilled on teacher-generated data suffer from pronounced exposure bias: during training, they are exposed to teacher-forced inputs, whereas at inference, they must rely entirely on their own autoregressive predictions. This training–inference mismatch induces a distributional shift, leading to error accumulation and compounding deviations over time. Indeed, we observe that the trained student model’s outputs often diverge from the training distribution in critical aspects—such as response length—and may enter unexpected states that result in incorrect answers. We present a preliminary exploration of approaches to address the above limitations and introduce several key advancements to enhance sequence-level distillation for long chain-of-thought (CoT) reasoning: - Temperature-scheduled Learning: Broadening coverage of the teacher’s modes. A natural approach is to use a higher sampling temperature to better cover the teacher’s output distribution. However, empirical comparison of training convergence across different temperature settings reveals that low-temperature samples exhibit more consistent patterns, making them easier for the student to learn, whereas high-temperature samples—though covering more of the teacher’s modes—introduce greater diversity, hindering learning efficiency. This motivates a temperature-scheduled learning strategy: the student first trains on low-temperature, high-confidence samples to grasp consistent patterns, then gradually incorporates higher-temperature samples to broaden mode coverage. Across the multi-domain settings we evaluated, this two-stage training approach achieves performance gains—particularly in complex reasoning domains such as mathematics and code generation—over single-stage training using either high- or low-temperature sampling. - Divergence-aware Sampling: Finding target sequence-level distribution better supports effective learning. Identifying a target distribution—namely, the sequence-level distribution over full responses, as opposed to the token-level logit distribution at each output position—that facilitates effective learning for the student model is nontrivial. Prior work often relies on heuristic rules to select human-expected target distributions, which introduce substantial manual intervention bias and lack theoretical guarantees. To address this, we propose a systematic distribution decomposition framework that analyzes discrepancies between teacher and student predictive probabilities across response candidates. This analysis reveals four canonical distribution patterns. Crucially, we find that one particular pattern—where the teacher assigns high confidence while the student has low probability—consistently correlates with improved test-set performance. Motivated by this finding, we encourage the student model to prioritize learning from such high-divergence instances. Notably, this distribution type naturally mitigates misleading gradients (e.g., from overconfident but incorrect student predictions), and thereby promoting more robust and efficient learning. - Mixed-policy Distillation: Mitigating exposure bias of distilled model. To mitigate exposure bias, we further introduce a lightweight constructively mixed-policy Mixed-policy refers to data generation involving both student and teacher models. distillation after the initial off-policy SFT phase. Specifically, we randomly select a small subset of training examples, prompt the trained student model to generate full responses, randomly truncate the generated prefixes, and then have the teacher complete the sequence from the truncation point. Only teacher continuations that pass predefined quality filters are retained for the student’s fine-tuning. With just a small amount of data and a few additional training steps, this approach yields further performance gains while encouraging more concise model outputs. Building on these innovations, we present DASD-4B-Thinking, a lightweight yet highly capable reasoning model, post-trained via our D istribution- A ligned S equence D istillation pipeline. Specifically, we use Qwen3-4B-Instruct-2507 (Yang et al., 2025) as the student model and gpt-oss-120b (Agarwal et al., 2025) as the teacher model, highlighting the broad compatibility of our approach across diverse model families and architectures. Despite substantial differences between the two models in scale, architecture, vocabulary, tokenizer, and pretraining corpora, our pipeline achieves robust distillation performance. We curate a multi-domain dataset spanning mathematics, code generation, scientific reasoning, and complex instruction-following tasks. Figure 2 illustrates the overall training pipeline. We first sample a small amount of cross-domain data at a low temperature and perform one-stage SFT on the student; we then increase the sampling temperature to generate a larger, more diverse training set, resuming training from the checkpoint of the previous stage. Throughout both stages, all synthetic data are generated using our divergence-aware sampling strategy, designed to better align the teacher’s output distribution with the student’s learning capacity. Beyond our core innovations, we also inherit established practices from prior work in this line: our pipeline incorporates rigorous quality control measures, such as filtering truncated outputs and repetitive content, to avoid introducing undesirable patterns into the student model. After these stages, the student exhibits strong reasoning capabilities. To further mitigate exposure bias during autoregressive generation, we introduce a lightweight mixed-policy distillation stage, combining teacher-forced and student-sampled trajectories. This yields the final DASD-4B-Thinking, which balances fidelity to high-quality reasoning paths with robustness against self-generated errors. <details> <summary>x5.png Details</summary>  ### Visual Description \n ## Process Diagram: Machine Learning Training Pipeline with Temperature Scheduling and Distillation ### Overview The image is a technical flowchart illustrating a machine learning training pipeline. It depicts a multi-stage process involving temperature-controlled training, response filtering, divergence-aware sampling, and knowledge distillation from a large teacher model. The diagram uses a combination of rectangular process blocks, cylindrical data stores, and directional arrows to show the flow of data and operations. ### Components/Axes The diagram is organized into two primary vertical sections (left and right) with interconnected flows. **Left Section (Training & Sampling Loop):** 1. **Top Block:** "Temperature-scheduled Learning" * Contains two sub-blocks: "Low-temperature Training" and "High-temperature Training". 2. **Middle Block:** "Response Filtering" * Receives input from the "Temperature-scheduled Learning" block. 3. **Data Stores (Cylinders):** * "Low-temperature Responses" (left cylinder) * "High-temperature Responses" (right cylinder) * Both are connected to the "Response Filtering" block above and the "Divergence-aware Sampling" block below. 4. **Bottom Block:** "Divergence-aware Sampling" * Receives input from the two response data stores. * Sends output to the "Teacher Model" in the right section. **Right Section (Distillation & Inference):** 1. **Top Block:** "Mixed-policy Distillation" * Receives input from the "Temperature-scheduled Learning" block on the left. 2. **Middle Block:** "DASD-4B-Thinking" * Receives input from the "Mixed-policy Distillation" block. 3. **Data Store (Cylinder):** "Questions" * Positioned below "DASD-4B-Thinking". 4. **Bottom Block:** "Teacher Model" * Contains the text "e.g., gpt-oss-120b". * Receives input from the "Questions" data store and the "Divergence-aware Sampling" block from the left section. * Sends output back to the "Divergence-aware Sampling" block, forming a feedback loop. **Flow Arrows:** * Arrows indicate the direction of data or process flow between all components. * A key feedback loop exists from the "Teacher Model" back to "Divergence-aware Sampling". ### Detailed Analysis The diagram outlines a sophisticated, iterative training methodology: 1. **Core Training Loop (Left Side):** The process begins with "Temperature-scheduled Learning," which involves two parallel training regimes: one at low temperature (likely for more deterministic, confident outputs) and one at high temperature (for more random, exploratory outputs). The outputs from these training phases are filtered ("Response Filtering") and stored separately as "Low-temperature Responses" and "High-temperature Responses." These stored responses are then processed by "Divergence-aware Sampling," which presumably selects data points that are diverse or informative. 2. **Knowledge Distillation Pathway (Right Side):** The "Temperature-scheduled Learning" block also feeds into "Mixed-policy Distillation." This suggests that the knowledge or policies learned under different temperature regimes are combined and distilled. The output of this distillation is a model or component named "DASD-4B-Thinking." 3. **Teacher-Student Interaction:** The "Teacher Model" (exemplified as "gpt-oss-120b") is central to the pipeline. It receives "Questions" (from a dedicated store) and data selected by "Divergence-aware Sampling." The teacher model's outputs are fed back into the "Divergence-aware Sampling" block, creating a closed loop where the teacher's responses inform the selection of new data for training or evaluation. ### Key Observations * **Explicit Temperature Control:** The pipeline explicitly separates and manages training under low and high temperature conditions, indicating a focus on controlling the entropy or randomness of the model's outputs during learning. * **Diversity Emphasis:** The component named "Divergence-aware Sampling" highlights a core objective: to actively seek out and utilize diverse data points or model responses, likely to improve robustness and generalization. * **Large Teacher Model:** The specified example "gpt-oss-120b" suggests the teacher model is a very large-scale language model (120 billion parameters), used to guide the training of a potentially smaller or differently structured student model ("DASD-4B-Thinking"). * **Iterative Refinement:** The feedback loop from the "Teacher Model" to "Divergence-aware Sampling" indicates an iterative process where the teacher's knowledge continuously refines the data selection for subsequent training cycles. * **Policy Combination:** The "Mixed-policy Distillation" block implies that strategies or behaviors learned under different conditions (temperatures) are integrated into a single, cohesive policy. ### Interpretation This diagram represents a advanced framework for training a language model (likely the "DASD-4B-Thinking" model) by leveraging a much larger teacher model. The core innovation appears to be the structured use of **temperature scheduling** to generate a spectrum of training data—from conservative to exploratory—and the **divergence-aware sampling** mechanism to intelligently select the most valuable data points from this spectrum for the teacher model to evaluate. The process aims to create a student model that is not just a compressed version of the teacher, but one that has learned to balance confident reasoning (low-temperature behavior) with creative exploration (high-temperature behavior) in a controlled manner. The "Mixed-policy Distillation" step is crucial for fusing these different behavioral modes into the final model. This approach could lead to a model that is more robust, less prone to repetitive or generic outputs, and better at handling ambiguous or novel tasks by effectively managing its own uncertainty. The entire pipeline emphasizes **data quality and diversity** over sheer data quantity, using the teacher model as a guide to filter and refine the learning process. </details> Figure 2: Overall training pipeline of DASD-4B-Thinking. DASD-4B-Thinking achieves state-of-the-art performance among models of comparable scale across multiple mainstream reasoning benchmarks in mathematics, code generation, and scientific reasoning—even outperforming some larger models (e.g., 32B-scale). Specifically, it attains 88.5 and 83.3 on the highly challenging mathematical competition benchmarks AIME24 and AIME25, respectively; scores 69.3 on LiveCodeBench v5, a widely adopted benchmark for code generation; and achieves 68.4 on GPQA-Diamond, a doctoral-level scientific reasoning benchmark. Notably, thanks to our methodological innovations, these results are obtained using only 448K training samples—an order of magnitude fewer than those employed by most existing open-source efforts. We have open-sourced our models (DASD-4B-Thinking and a Mixture-of-Experts (MoE) version DASD-30B-A3B-Thinking-Preview—both derivative models of the Qwen family), together with the training dataset on Hugging Face and ModelScope. In the following sections, we first elaborate on the motivation, methodology, and roles of the three core components of our approach. Section 6 details the full training pipeline implementation, and Section 7 presents a comprehensive evaluation of the models. ## 2 Preliminaries The current long CoT distillation paradigm of SFT on teacher-generated data can be traced back to the sequence-level distillation (Kim and Rush, 2016), with the original aim of allowing the student model to mimic the teacher’s distribution at the sequence level and thereby acquiring capabilities comparable to the teacher. Given an input $\boldsymbol{x}$ , it trains a student model $p_S$ to minimize the sequence level divergence from the teacher model $p_T$ : $$ minD_KL(p_T(\boldsymbol{y}∈Y|\boldsymbol{x})||p_S(\boldsymbol{y}∈Y|\boldsymbol{x})), \tag{1} $$ where $Y$ is the set of all possible responses that the teacher model can generate for the input prompt $\boldsymbol{x}$ . This formulation closely parallels classical logit-based distillation (Hinton et al., 2015); the key difference is that logit distillation typically matches the teacher’s conditional next-token distribution at each position, whereas sequence-level distillation aligns with the teacher’s distribution over entire output sequences. By modeling the teacher’s distribution, logit distillation conveys the teacher’s implicit ``dark knowledge'', enabling the student to inherit its generalization. Analogously, sequence-level distillation pursues the same objective at the level of complete sequences. Expanding the KL divergence in Equation 1 yields the following sequence-level distillation objective: $$ L_SEQ=∑_\boldsymbol{y∈Y}p_T(\boldsymbol{y}|\boldsymbol{x})[\log p_T(\boldsymbol{y}|\boldsymbol{x})-\log p_S(\boldsymbol{y}|\boldsymbol{x})], \tag{2} $$ the value of $p_T(\boldsymbol{y}|\boldsymbol{x})\log{p_T(\boldsymbol{y}|\boldsymbol{x})}$ depends only on the teacher and is therefore constant with respect to the student’s parameters. It does not affect the gradients and can thus be dropped. Consequently, the sequence-level distillation objective simplifies to: $$ L_SEQ=-∑_\boldsymbol{y∈Y}p_T(\boldsymbol{y}|\boldsymbol{x})\log p_S(\boldsymbol{y}|\boldsymbol{x}). \tag{3} $$ However, the exponential size of $Y$ makes exact computation of $L_SEQ$ intractable. A practical approximation is to use a sampled response $\hat{\boldsymbol{y}}$ and replace $p_T(·\mid\boldsymbol{x})$ with a point mass at $\hat{\boldsymbol{y}}$ : $$ p_T(\boldsymbol{y}\mid\boldsymbol{x})≈1\{\boldsymbol{y}=\hat{\boldsymbol{y}}\}. \tag{4} $$ Kim and Rush (2016) adopt beam search to produce $\hat{\boldsymbol{y}}$ , which approximates the mode of $p_T(·|\boldsymbol{x})$ (i.e., the response with the highest probability). In contrast, much of the recent literature on long CoT distillation employs randomly sampled responses as $\hat{\boldsymbol{y}}$ . After this approximation, $L_SEQ$ can be further simplified to: $$ L_SEQ∼-∑_\boldsymbol{y∈Y}1\{\boldsymbol{y}=\hat{\boldsymbol{y}}\}\log p_S(\boldsymbol{y}|\boldsymbol{x})=-\log p_S(\hat{\boldsymbol{y}}|\boldsymbol{x}), \tag{5} $$ which exactly recovers the standard SFT loss on teacher-generated outputs. This perspective clarifies that the success of SFT on teacher-generated data hinges on effectively transferring the teacher’s sequence-level distribution to the student model. Building on the above analysis, we argue that the current sequence-level long CoT distillation paradigm should place greater emphasis on teacher–student interaction throughout the distillation process. However, most existing methods are primarily grounded in the SFT perspective. Consequently, they prioritize filtering high-quality teacher outputs (i.e., SFT data) while neglecting such interaction, leading to three main limitations: (i) Inadequate coverage of the teacher’s sequence-level distribution; (ii) Misalignment between the teacher’s output distribution and the student’s learning capacity; and (iii) Exposure bias stemming from teacher forcing during training versus autoregressive inference at test time. In the following sections, we detail our rationale and empirical explorations for the three limitations. ## 3 Temperature-scheduled Learning According to Equation 4, SFT on teacher-generated data can implicitly convey the teacher’s distributional information. Consequently, the strategy for selecting samples—used to approximate the teacher’s output distribution—plays a pivotal role in how effectively this knowledge is transferred to the student. However, most existing methods for long CoT distillation overlook this consideration, typically relying on random sampling (RS) from the teacher followed by quality-based filtering prior (Yan et al., 2025; Lei et al., 2025). This approach tends to produce samples that cover only a small subset of the teacher’s modes, thereby under-utilizing the rich latent information embedded in the teacher’s distribution. A natural remedy is to increase the sampling temperature, which flattens the teacher’s distribution and leads to better coverage of its full mode structure (Holtzman et al., 2020; Jang et al., 2017). <details> <summary>x6.png Details</summary>  ### Visual Description \n ## Histogram and Line Chart: Temperature Effects on Probability Distribution and Training Loss ### Overview The image contains two side-by-side subplots, labeled (a) and (b), comparing the effects of "low temperature" and "high temperature" on two different metrics. Subplot (a) is a histogram with an accompanying rug plot showing probability distributions. Subplot (b) is a line graph tracking loss over training steps. Both plots use a consistent color scheme: gray for "low temperature" and orange for "high temperature." ### Components/Axes **Subplot (a) - Left:** * **Chart Type:** Histogram with a rug plot below. * **X-axis Label:** "Probability Distribution of Samples" * **X-axis Scale:** Linear, from 0.0 to 1.0, with major ticks at 0.0, 0.2, 0.4, 0.6, 0.8, 1.0. * **Y-axis Label:** "Density" * **Y-axis Scale:** Linear, with ticks visible but unlabeled. The highest density bar reaches approximately 1.6 on the implied scale. * **Legend:** Located in the top-left corner. Contains two entries: * A gray square labeled "low temperature" * An orange square labeled "high temperature" * **Rug Plot:** Positioned directly below the histogram, sharing the same x-axis. It displays vertical tick marks representing individual data points, colored gray and orange. **Subplot (b) - Right:** * **Chart Type:** Line graph. * **X-axis Label:** "Training Step" * **X-axis Scale:** Linear, from 0 to 2000, with major ticks at 0, 500, 1000, 1500, 2000. * **Y-axis Label:** "Loss" * **Y-axis Scale:** Linear, with ticks visible but unlabeled. The loss ranges from approximately 0.25 to 1.6. * **Legend:** Located in the top-left corner. Contains two entries: * A gray line labeled "low temperature" * An orange line labeled "high temperature" ###### Detailed Analysis **Subplot (a) - Probability Distribution:** * **High Temperature (Orange):** The distribution is roughly unimodal and centered. The peak density occurs at a probability value of approximately 0.55. The distribution spans from about 0.3 to 0.75, with the bulk of the mass between 0.4 and 0.7. The rug plot shows a dense cluster of orange ticks in this central region. * **Low Temperature (Gray):** The distribution is also unimodal but shifted to the right compared to the high-temperature distribution. Its peak density is at a probability value of approximately 0.75. The distribution spans from about 0.5 to 0.9, with the bulk of the mass between 0.65 and 0.85. The rug plot shows gray ticks concentrated in this higher probability range. * **Relationship:** The two distributions overlap significantly between probability values of 0.5 and 0.75. The low-temperature distribution is clearly skewed toward higher probability values. **Subplot (b) - Training Loss:** * **High Temperature (Orange Line):** The loss starts very high, at approximately 1.6 at step 0. It drops sharply within the first ~100 steps to about 0.8. After this initial drop, the loss continues to decrease very gradually, exhibiting minor fluctuations. By step 2000, the loss is approximately 0.65. The overall trend is a steep initial descent followed by a long, slow, noisy plateau. * **Low Temperature (Gray Line):** The loss starts lower than the high-temperature line, at approximately 0.4 at step 0. It also drops sharply in the initial steps, reaching about 0.3 by step 100. It then continues a slow, steady decline with very minor fluctuations. By step 2000, the loss is approximately 0.25. The line is consistently below the high-temperature line throughout the entire training process. * **Relationship:** There is a clear and consistent gap between the two lines. The low-temperature condition results in significantly lower loss at all training steps shown. Both curves follow a similar shape: rapid initial improvement followed by diminishing returns. ### Key Observations 1. **Distinct Distributions:** The "low temperature" setting produces model outputs (or samples) with a probability distribution centered at a higher value (~0.75) compared to the "high temperature" setting (~0.55). 2. **Loss Performance:** The "low temperature" setting achieves and maintains a substantially lower training loss throughout the 2000 steps shown. The gap between the two conditions is large and persistent. 3. **Convergence Behavior:** Both settings show classic learning curves with rapid early improvement. However, the "high temperature" loss appears to plateau at a higher value and exhibits slightly more noise or variance in its trajectory after the initial drop. 4. **Visual Confirmation:** The color coding in the legends (gray=low, orange=high) is consistently applied across both subplots, allowing for direct comparison of the same experimental conditions on different metrics. ### Interpretation This data suggests a fundamental trade-off controlled by the "temperature" parameter, likely in a machine learning or probabilistic modeling context. * **Temperature and Uncertainty:** In such contexts, "temperature" often controls the randomness or entropy of a model's output distribution. A **high temperature** increases randomness, flattening the probability distribution. This is reflected in subplot (a), where the orange distribution is broader and centered at a lower probability value, indicating less confident, more exploratory predictions. Conversely, a **low temperature** makes the model more deterministic and confident, sharpening the distribution around higher probability values, as seen in the gray histogram. * **Impact on Optimization:** Subplot (b) demonstrates that this increase in confidence (low temperature) correlates with better optimization, as measured by lower training loss. The model converges to a better solution on the training data. The high-temperature model's higher loss suggests its increased randomness hinders its ability to fit the training data as precisely. * **The Trade-off:** The charts together illustrate a potential tension between **exploration** (high temperature, broader distributions, higher loss) and **exploitation/confidence** (low temperature, peaked distributions, lower loss). The choice of temperature would depend on the goal: for tasks requiring diverse, creative outputs, a higher temperature might be preferred despite higher loss; for tasks requiring precise, accurate fits to known data, a lower temperature is superior. The data does not show validation or test performance, so it's unclear which setting generalizes better. </details> Figure 3: Comparison of probability distribution and training loss with data sampled from gpt-oss-120b under different temperatures. We randomly sampled 50K mathematical reasoning responses at both low (T=0.6) and high (T=1.0) temperatures. To characterize the overall likelihood of a response, we compute the geometric mean of its token-level probabilities. (a) Probability distributions of sampled responses: the upper panel displays the density of probability distribution, while the lower panel shows the probability intervals covered by the sampled responses. (b) SFT training loss curves for the student model trained on responses sampled at these temperatures. As shown in Figure 3 (a), we visualize the probability distribution of teacher-generated responses sampled at different temperatures. At lower temperatures (T=0.6), the resulting distribution becomes sharper and more peaked, concentrating most probability mass in a narrow range of high-likelihood responses. In contrast, higher-temperature (T=1.0) sampling yields a flatter and broader density, substantially increasing the covered probability range and markedly enhancing data diversity. However, we observe that high-temperature sampling introduces many rare teacher modes or potentially noisy samples. When the student model has limited capacity and exhibits a substantial architectural or behavioral gap from the teacher, it struggles to effectively learn from such heterogeneous data. Figure 3 (b) compares the SFT training loss using datasets sampled at different temperatures. Specifically, we randomly sampled 50K math responses from the gpt-oss-120b teacher model at low and high temperatures, and fine-tuned the Qwen3-4B-Instruct-2507 student model on each dataset separately. The low-temperature dataset enables rapid convergence to a lower loss with a smooth downward trajectory, whereas the high-temperature dataset makes learning difficult: the loss stays higher. Despite being more challenging to learn from, training on data sampled at temperature T=1.0 consistently outperforms that sampled at T=0.6, as evidenced in Table 1. Training on T=1.0 samples yields a +1.4 absolute improvement on AIME24 and an even larger +4.2 gain on the more representative and challenging AIME25. This indicates that—even under more difficult optimization dynamics and slower convergence—broader coverage of the teacher’s output modes can lead to substantially greater gains for the student model. It also underscores the crucial impact of the sampling strategy in determining the efficacy of sequence-level distillation. To further evaluate the impact of high-temperature sampling, we scaled the dataset size to 100K samples (doubling the 50K baseline). However, this increase in data volume yields only marginal improvements: as shown in Table 1, the 100K T=1.0 setup achieves no gain on AIME24 relative to 50K, and only a +2.8 improvement on AIME25. This suggests that the student’s capacity to absorb diverse teacher behaviors becomes a bottleneck; adding more high-temperature samples does not translate into proportional performance gains. Table 1: Performance comparison with different temperature settings. | Settings of training data | AIME24 | AIME25 | | --- | --- | --- | | Teacher: gpt-oss-120b Student: Qwen3-4B-Instruct-2507 | | | | 50K Math + RS ( $T=0.6$ ) | 81.7 | 71.9 | | 50K Math + RS ( $T=1.0$ ) | 83.1 | 76.1 | | 100K Math + RS ( $T=1.0$ ) | 83.1 | 78.9 | | 50K Math + RS ( $T=1.0$ ) w/ cold start ( $T=0.6$ ) | 85.2 | 81.3 | | Teacher: Qwen3-Next-80B-A3B-Thinking Student: Qwen3-4B-Instruct-2507 | | | | 25K Math + RS ( $T=0.6$ ) | 79.0 | 71.3 | | 25K Math + RS ( $T=1.0$ ) | 82.9 | 70.2 | | 25K Math + RS ( $T=1.0$ ) w/ cold start ( $T=0.6$ ) | 83.1 | 73.1 | Based on these observations, we propose a temperature-scheduled learning pipeline for sequence-level distillation, an approach inspired by classic logit-based distillation (Caron et al., 2021; Zhou et al., 2021), which we extend to the SFT setting. We begin by sampling from the teacher at a low temperature, yielding a concentrated set of high-probability, easier-to-learn modes. We then switch to a higher temperature to collect more diverse samples that capture rarer teacher modes and richer latent information, albeit at the cost of increased learning difficulty. Accordingly, we use the low-temperature data to cold-start the student and then continue training with the high-temperature data. As an analogy, this can be intuitively viewed as an easy-to-hard curriculum temperature schedule (Li et al., 2023b), or equivalently, a ``reverse'' version of temperature annealing, a strategy that increases temperature over training, metaphorically inverting the cooling process of conventional annealing. As shown in Table 1, cold-starting with 50K samples at temperature 0.6 followed by continued training on another 50K samples at temperature 1.0 yields significant performance gains over all static-temperature baselines. This demonstrates that our strategy successfully reconciles two objectives: (i) facilitating stable early-stage learning, and (ii) broadening coverage of the teacher’s output distribution, thereby transferring more valuable latent knowledge from the teacher to the student. <details> <summary>x7.png Details</summary>  ### Visual Description ## Histogram and Line Chart: Temperature Effects on Sample Probability and Training Loss ### Overview The image contains two adjacent subplots, labeled (a) and (b), presenting a comparative analysis of two experimental conditions: "low temperature" and "high temperature." Subplot (a) is a histogram with an accompanying rug plot, visualizing the probability distribution of samples. Subplot (b) is a line graph, tracking the training loss over 1000 steps. The overall figure appears to be from a technical document, likely a research paper or report, as indicated by the page number "Seite 6" (German for "Page 6") in the bottom-right corner. ### Components/Axes **Subplot (a): Histogram** * **Type:** Histogram with a rug plot below. * **X-Axis:** Labeled "Probability Distribution of Samples". Scale ranges from 0.0 to 1.0, with major tick marks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **Y-Axis:** Labeled "Density". Scale is not numerically marked, but the height of the bars represents density. * **Legend:** Positioned in the top-left corner. Contains two entries: * A gray rectangle labeled "low temperature". * An orange rectangle labeled "high temperature". * **Data Series:** * **Low Temperature (Gray):** A distribution skewed towards higher probabilities. The highest density bar is located approximately between 0.85 and 0.95. The distribution appears to start around 0.5 and extends to 1.0. * **High Temperature (Orange):** A distribution centered around a lower probability compared to the low temperature series. The peak density is approximately between 0.7 and 0.8. The distribution spans roughly from 0.4 to 0.95. * **Rug Plot:** Located directly below the histogram, sharing the same x-axis. It displays individual data points as vertical lines. The orange lines (high temperature) are densely clustered between approximately 0.5 and 0.9. The gray lines (low temperature) are clustered more towards the right, between approximately 0.7 and 1.0. **Subplot (b): Line Chart** * **Type:** Line chart. * **X-Axis:** Labeled "Training Step". Scale ranges from 0 to 1000, with major tick marks at 0, 500, and 1000. * **Y-Axis:** Labeled "Loss". Scale ranges from 0.0 to 1.0, with major tick marks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **Legend:** Positioned in the top-right corner. Contains two entries: * A gray line labeled "low temperature". * An orange line labeled "high temperature". * **Data Series:** * **Low Temperature (Gray Line):** The line starts at a very high loss (approximately 0.8-0.9) at step 0. It shows a steep initial decline, followed by a more gradual, stepwise decrease. By step 1000, the loss has decreased to approximately 0.15. * **High Temperature (Orange Line):** The line starts at a lower initial loss (approximately 0.4-0.5) at step 0. It exhibits a more gradual decline with noticeable fluctuations and plateaus throughout the training. By step 1000, the loss is approximately 0.25-0.3. ### Detailed Analysis **Subplot (a) - Distribution Analysis:** * **Trend Verification:** The gray histogram (low temperature) slopes upward to the right, indicating a concentration of samples with high predicted probabilities. The orange histogram (high temperature) has a central peak, indicating samples are concentrated around a medium-high probability. * **Key Data Points (Approximate):** * **Low Temperature Peak:** Highest density at probability ~0.9. * **High Temperature Peak:** Highest density at probability ~0.75. * **Range:** Low temperature samples are primarily found between 0.7 and 1.0. High temperature samples are primarily found between 0.5 and 0.9. **Subplot (b) - Training Loss Analysis:** * **Trend Verification:** The gray line (low temperature) has a strong downward slope, especially at the beginning, indicating rapid convergence. The orange line (high temperature) has a gentler downward slope with more noise, indicating slower and less stable convergence. * **Key Data Points (Approximate):** * **Start (Step 0):** Low Temp Loss ~0.85; High Temp Loss ~0.45. * **Middle (Step 500):** Low Temp Loss ~0.2; High Temp Loss ~0.35. * **End (Step 1000):** Low Temp Loss ~0.15; High Temp Loss ~0.28. ### Key Observations 1. **Inverse Relationship in Starting Points:** In the histogram (a), the low temperature condition produces samples with higher confidence (probability). In the loss chart (b), the low temperature condition starts with a much higher error (loss). 2. **Convergence Behavior:** The low temperature training (gray line) converges faster and to a lower final loss value compared to the high temperature training. 3. **Distribution Shape:** The probability distribution for the low temperature condition is more skewed and concentrated at the high end, while the high temperature distribution is more symmetric and centered. 4. **Fluctuations:** The high temperature loss curve shows more step-like plateaus and fluctuations during training, suggesting more exploration or instability in the optimization process. ### Interpretation The data suggests a trade-off or a specific dynamic between the "temperature" parameter and model behavior. In machine learning, "temperature" often controls the randomness of predictions. A **low temperature** makes the model more deterministic and confident. This is reflected in histogram (a), where the model assigns very high probabilities to its samples. However, this confidence does not initially align with the ground truth, as shown by the very high starting loss in chart (b). The model then learns rapidly from its confident mistakes. Conversely, a **high temperature** makes the model's predictions more random and less confident. This results in a more spread-out, medium-confidence probability distribution in (a) and a lower initial loss in (b), as the model's predictions are less extreme and thus less wrong at the start. However, this condition leads to slower and less effective learning, as the model's updates are noisier. The relationship between the two charts is complementary. They show that the condition leading to high model confidence (low temperature) also leads to high initial error but fast learning. The condition leading to lower confidence (high temperature) leads to lower initial error but slower learning. This could imply that a certain level of model confidence, even if initially misplaced, is beneficial for rapid convergence during training. The outlier is the starting point of the loss curves, which is inversely related to the final confidence of the samples. </details> Figure 4: Comparison of probability distribution and training loss with data sampled from Qwen3-Next-80B-A3B-Thinking under different temperatures. We further validate this approach across a broad range of domains and diverse teacher–student model pairs. In the lower part of Table 1, we use Qwen3-Next-80B-A3B-Thinking as the teacher and randomly sample a small-scale of 25K math reasoning traces at temperatures 0.6 and 1.0, respectively. Continuing training—starting from a cold initialization with T=0.6 samples—and incorporating additional T=1.0 samples yields performance gains of +4.1 on AIME24 and +1.8 on AIME25, which also surpasses training on T=1.0 data alone. As shown in Figure 4, the shift in response probability distributions between T=0.6 and T=1.0 is markedly smaller for Qwen3-Next-80B-A3B-Thinking than for gpt-oss-120b. When training on data from both temperatures, the loss on T=1.0 samples remains higher, but the gap relative to T=0.6 is much narrower than for gpt-oss-120b. Nevertheless, temperature-scheduled learning still improves performance, indicating that samples drawn at different temperatures are complementary. In practice, optimal temperature combinations can be selected based on the model's evaluation performance and the response probability distributions observed across temperature settings. In Table 2, we further evaluate our approach under multi-domain mixed training. Using a small-scale mixture of math, code, and science reasoning data, we show that temperature-scheduled learning remains effective: it delivers substantial gains on AIME25, LiveCodeBench v6, and GPQA Diamond over training with data generated solely at T=0.6 or T=1.0. With the total training data held constant, it also outperforms the T=1.0-only baseline on AIME25 and GPQA Diamond, while achieving comparable performance on LiveCodeBench v6 (potentially attributable to the relatively small proportion of code data in the mixture, where further scaling could still yield noticeable improvements). Table 2: Performance comparison with different settings of training data across domains. | Settings of training data | AIME25 | LCB v6 | GPQA-D | | --- | --- | --- | --- | | Teacher: gpt-oss-120b Student: Qwen3-4B-Instruct-2507 | | | | | 25K Math + 10K Code + 10K Science + RS ( $T=0.6$ ) | 74.6 | 44.1 | 65.5 | | 25K Math + 10K Code + 10K Science + RS ( $T=1.0$ ) | 75.2 | 47.3 | 65.4 | | 50K Math + 20K Code + 20K Science + RS ( $T=1.0$ ) | 75.8 | 51.3 | 65.4 | | 25K Math + 10K Code + 10K Science + RS ( $T=1.0$ ) w/ cold start ( $T=0.6$ ) | 77.5 | 51.0 | 66.4 | ## 4 Divergence-aware Sampling Despite employing temperature-scheduled learning to broaden coverage of the teacher’s modes, the student still struggles to align with the teacher’s sequence-level distribution. Classical logit distillation leverages teacher logit distribution to precisely calibrate the student’s token-level probabilities, increasing or decreasing them as needed (Gu et al., 2024; Agarwal et al., 2024). By contrast, SFT on teacher-generated data typically amplifies the probabilities of all target tokens relative to the student’s current predictions. This can induce misleading gradients: for tokens assigned low probabilities by the teacher but high probabilities by the student, SFT erroneously pushes the student’s probabilities even higher, thereby driving them away from the teacher’s distribution. This discrepancy motivates a core question: How can we identify a teacher-derived sequence-level distribution that is better aligned with the student model’s learning capacity? <details> <summary>x8.png Details</summary>  ### Visual Description \n ## Line Chart with Shaded Regions: Sentence Probability Comparison ### Overview The image displays a line chart overlaid with vertical shaded regions, plotting probability values against a sequence of sentence indices. It compares the probability scores of sentences from a "Teacher" model, a "Student" model, and a "Distilled Student" model across 180 sentence indices. The shaded background categorizes each sentence index by its type (Teacher, Boosted, Shared, or Student). ### Components/Axes * **Chart Type:** Line chart with categorical background shading. * **Y-Axis:** * **Label:** "Probability" * **Scale:** Linear, ranging from 0.0 to 1.0, with major tick marks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **X-Axis:** * **Label:** "Sentence Index" * **Scale:** Linear, ranging from 0 to 180, with major tick marks every 20 units (0, 20, 40, ..., 180). * **Legend (Top Center):** * **Shaded Region Categories (from left to right):** 1. `Teacher Sentence` (Light Cyan/Teal fill) 2. `Boosted Sentence` (Light Purple fill) 3. `Shared Sentence` (Light Orange/Peach fill) 4. `Student Sentence` (Light Blue fill) * **Line Categories (from left to right):** 1. `Student Sentence Probability` (Solid Orange line) 2. `Distilled Student Sentence Probability` (Solid Gray line) 3. `Teacher Sentence Probability` (Solid Blue line) ### Detailed Analysis The chart visualizes three probability series across 180 data points (sentence indices). The background is segmented into vertical bands of four colors, indicating the type of sentence at each index. **Line Trends & Approximate Values:** 1. **Teacher Sentence Probability (Blue Line):** * **Trend:** This line generally maintains the highest probability values, frequently oscillating between ~0.6 and 1.0. It shows high volatility but rarely drops below 0.4. * **Key Points:** Peaks near 1.0 are common (e.g., around indices 10, 30, 80, 140). Notable troughs occur near index 45 (~0.4) and index 100 (~0.3). 2. **Student Sentence Probability (Orange Line):** * **Trend:** This line exhibits the most extreme volatility, with sharp, frequent drops to very low probabilities (near 0.0) and recoveries to high values (near 1.0). Its average appears lower than the Teacher line. * **Key Points:** Deep troughs near 0.0 are visible at indices ~45, ~100, ~125, and ~150. It often peaks near 1.0, especially in regions shaded as "Teacher Sentence" or "Shared Sentence." 3. **Distilled Student Sentence Probability (Gray Line):** * **Trend:** This line's behavior is intermediate. It is less volatile than the Student line but more volatile than the Teacher line. It often tracks between the other two lines. * **Key Points:** It frequently dips to values between 0.2 and 0.4 (e.g., indices ~45, ~100) but avoids the near-zero drops seen in the Student line. Its peaks often align with peaks in the Teacher line. **Shaded Region Distribution:** The background shading is not uniform. "Teacher Sentence" (cyan) and "Shared Sentence" (orange) regions appear most frequently and in longer contiguous blocks. "Boosted Sentence" (purple) and "Student Sentence" (blue) regions are shorter and more interspersed. ### Key Observations 1. **Correlation with Sentence Type:** The `Teacher Sentence Probability` (blue line) is consistently high within "Teacher Sentence" (cyan) and "Shared Sentence" (orange) shaded regions. The `Student Sentence Probability` (orange line) shows its most severe drops (near 0.0) within "Student Sentence" (blue) and "Boosted Sentence" (purple) regions. 2. **Volatility Hierarchy:** Visual inspection confirms a clear order of volatility: Student Sentence Probability (most volatile) > Distilled Student Sentence Probability > Teacher Sentence Probability (least volatile). 3. **Synchronized Dips:** All three probability lines often dip simultaneously at specific indices (e.g., ~45, ~100, ~125), suggesting these sentences are challenging for all models. 4. **Distillation Effect:** The `Distilled Student` line (gray) consistently acts as a smoothed, intermediate version of the `Student` line (orange), rarely reaching its extreme lows, which suggests the distillation process stabilizes the student's confidence. ### Interpretation This chart likely illustrates the process and outcomes of **knowledge distillation** in natural language processing, where a smaller "Student" model is trained to mimic a larger "Teacher" model. * **What the data suggests:** The high and relatively stable probability of the Teacher model indicates it is confident in its predictions across most sentences. The Student model's extreme volatility suggests it is highly uncertain or "confused" on many sentences, particularly those not directly from the teacher ("Student" and "Boosted" types). The Distilled Student model shows improved stability over the base Student, demonstrating the effectiveness of the distillation training. * **Relationship between elements:** The shaded regions define the *source* or *type* of each training sentence. The lines show the *confidence* (probability) each model assigns to its own prediction for that sentence. The strong alignment of high Teacher probability with "Teacher" and "Shared" regions is expected. The poor Student performance on "Student" and "Boosted" regions highlights areas where the student struggles without direct teacher guidance. * **Notable patterns:** The synchronized dips are critical—they identify a subset of sentences that are inherently difficult, as even the powerful Teacher model's confidence drops (though less severely). The "Shared Sentence" regions are particularly interesting, as they show where both models' training data overlaps, and here, all models tend to have higher probabilities. The chart effectively argues that distillation (gray line) successfully transfers knowledge, pulling the student's confidence profile closer to the teacher's, especially by preventing catastrophic drops in probability. </details> Figure 5: Joint comparison of the three models’ predicted probabilities. An example of output probabilities: the x-axis indexes sentences, and the y-axis shows predicted probabilities. Foreground lines plot the probabilities of the three models, while background colors indicate the inferred source of each sentence. By comparing probability differences, every sentence is categorized into one of four source types. To identify an effective sequence-level target distribution from the student’s perspective, we introduce a distribution decomposition and analysis framework (Liu et al., 2025): each sequence-level response is decomposed into consecutive sentences and the corresponding sentence-level generation probabilities are computed for both the teacher and the student; by quantifying the probability discrepancy on each shared sentence, we categorize distinct behavioral patterns; finally, we systematically analyze these patterns (i.e., components) and establish their empirical relationship to effective student learning. Concretely, following the experimental setup in Section 3, we first sample responses from the distilled model (i.e., the trained student model) on test-set prompts and segment each response into sentences. This sentence-level analysis ensures the broad applicability of our method across heterogeneous model families —unlike approaches such as on-policy distillation, which typically requires all models to share the same tokenizer and vocabulary (a constraint imposed by its reliance on token-level supervision). Then, we feed these samples to the teacher model, the pre-distillation student model (hereafter, the ``student model''), and the post-distillation student model (hereafter, the ``distilled model''). For each sentence of the response data, we compute its probability under each of the three models as the geometric mean of per-token probabilities in this sentence. As illustrated in Figure 5, we observe that the sequence-level distribution admits a natural decomposition into four well-defined distribution types (each corresponding to a distinct sentence category). Let $p_T$ , $p_S$ and $p_D$ denote the predicted probabilities of the teacher, student, and distilled models for the same sentence, respectively. Based on the relative magnitude discrepancies of these probabilities, we define the following distribution (or sentence) types: - Student-originated sentences (hereafter referred to as Student Sentence) and teacher-originated sentences (hereafter referred to as Teacher Sentence): When there is a large discrepancy between $p_S$ and $p_T$ , the distilled model still outputs the sentence, suggesting the sentence is more consistent with the model assigning the higher likelihood. For example, if $p_T\gg p_S$ and distilled model nevertheless produces the sentence, it is more likely teacher-originated. Moreover, when $p_T\gg p_S$ , the student can relatively freely increase its probability under SFT without concern about misleading gradients. Intuitively, this type of pattern is more likely to apply under our current distillation setup. Note that a Teacher Sentence does not imply that the action is entirely absent from the student model, but rather that it is primarily originated from the teacher. The same applies to a Student Sentence. - Pre-existing sentences in both pre-distillation student model and teacher model, not enhanced by distillation (hereafter referred to as Shared Sentence): The output probabilities for these sentences are similar across all three models. This indicates that these sentences are already well-supported by both the pre-distillation student and the teacher, and that distillation does not materially change their probabilities or increase inter-model distribution discrepancies. - Pre-existing sentences boosted through distillation (hereafter referred to as Boosted Sentence): Similar to the second type, $p_T$ and $p_S$ remain close, but $p_D$ differs significantly (and $p_D$ is typically higher in practice, since trajectories are sampled from the distilled model). These sentences also exist in both the teacher and the student before distillation, but their probabilities are amplified by training on distilled data. Having decoupled the output distributions, we next investigate which distribution types are most conducive to the student model’s learning (i.e., those that best support effective knowledge acquisition). To this end, we assess effective learning by analyzing the correlation between the four distribution types and test-set answer correctness. Specifically, for each sentence position, we compute the probability that the distilled model assigns to each distribution type. Since solutions often contain multiple sentences (correct answers typically contain fewer sentences than incorrect ones), analyzing at the sentence-position-level, rather than at the full solution level, allows us to focus more directly on the distribution types themselves and mitigate confounding effects arising from sentence position. For example, to estimate the probability of the Teacher Sentence at the third sentence position, we calculate the fraction of third sentences that are categorized as Teacher Sentence, across all correct and incorrect model outputs. Notably, the number of sentences per answer varies, limiting data availability at later positions. To ensure statistical reliability, we therefore focus primarily on earlier sentence positions, where sufficient samples exist. We also replicate this analysis on the open-source model DeepSeek-Distill-Qwen3-8B (Guo et al., 2025) to ensure generalizability. As shown in Figure 6, across models, Teacher Sentences tend to receive higher probabilities in correct answers, evidenced by the light-green solid line (—–) persistently lying above the light-green dashed line (- - -). This is as expected: since the teacher model performs better on the test set, aligning the student’s outputs with teacher-preferred responses enhances learning efficacy and, consequently, the likelihood of generating correct answers. In contrast, we find that Shared Sentence and Student Sentence occur with low probability and exert a relatively minor influence. For Boosted Sentence, we observe a potential negative correlation between Boosted Sentences and test-set accuracy. We conjecture that this possibly stem from suboptimal misleading gradients. More importantly, the distillation pipeline only admits the teacher and student models prior to training, rendering it impossible to directly identify Boosted Sentences. We therefore focus primarily on Teacher Sentences in the remainder of this work. <details> <summary>x9.png Details</summary>  ### Visual Description \n ## Line Charts: Comparative Probability Analysis of Sentence Types Across Models ### Overview The image displays four line charts arranged horizontally, comparing the probability trends of four different sentence types ("Teacher Sentence," "Boosted Sentence," "Shared Sentence," "Student Sentence") across two different models or conditions labeled "Ours" and "DeepSeek-Distill-Qwen3-8B." Each chart plots "Probability" on the y-axis against "Sentence Index" on the x-axis. The charts include shaded confidence intervals or variance bands and specific numerical annotations at the end of each data series. ### Components/Axes * **Legend (Top Center):** A shared legend for all four charts defines the color coding: * **Green Line:** Teacher Sentence * **Purple Line:** Boosted Sentence * **Orange Line:** Shared Sentence * **Blue Line:** Student Sentence * **Chart 1 (Far Left):** * **Title/Label:** "Ours" (top-right corner) * **Y-axis:** Label "Probability," scale from 0.1 to 0.6. * **X-axis:** Label "Sentence Index," scale from 0 to 1500. * **Chart 2 (Center Left):** * **Title/Label:** "Ours" (top-right corner) * **Y-axis:** Unlabeled, scale from 0.0 to 1.0. * **X-axis:** Label "Sentence Index," scale from 0 to 1500. * **Chart 3 (Center Right):** * **Title/Label:** "DeepSeek-Distill-Qwen3-8B" (top-right corner) * **Y-axis:** Unlabeled, scale from 0.4 to 0.9. * **X-axis:** Label "Sentence Index," scale from 0 to 1500. * **Chart 4 (Far Right):** * **Title/Label:** "DeepSeek-Distill-Qwen3-8B" (top-right corner) * **Y-axis:** Unlabeled, scale from 0.0 to 0.5. * **X-axis:** Label "Sentence Index," scale from 0 to 1500. ### Detailed Analysis **Chart 1 ("Ours"):** * **Teacher Sentence (Green):** Shows a steep initial decline from a probability of ~0.6 at index 0, followed by a gradual, noisy decrease. The line ends with an annotation **"= 45.08"**. A shaded green band indicates variance around the line. * **Boosted Sentence (Purple):** Remains relatively flat and low, hovering just above 0.1 probability throughout. * **Shared Sentence (Orange) & Student Sentence (Blue):** Both remain very low, near the bottom of the chart (probability ~0.05 or lower), with minimal visible fluctuation. **Chart 2 ("Ours"):** * **Boosted Sentence (Purple):** Dominates this chart. It rises quickly from near 0 to a plateau around 0.6 probability and remains stable. Annotated with **"= 62.99"**. * **Shared Sentence (Orange):** Shows a gradual, slight increase from near 0 to approximately 0.15-0.2 probability. Annotated with **"= 28.98"**. * **Student Sentence (Blue):** Remains very low, near zero, with a slight upward trend at the very end. Annotated with **"= -11.07"**. * **Teacher Sentence (Green):** Not visibly plotted in this chart. **Chart 3 ("DeepSeek-Distill-Qwen3-8B"):** * **Teacher Sentence (Green):** Fluctuates significantly between approximately 0.5 and 0.7 probability, showing a slight overall downward trend. Annotated with **"= 61.26"**. * **Boosted Sentence (Purple):** Remains relatively flat in the lower portion of the chart, around 0.4 probability. * **Shared Sentence (Orange) & Student Sentence (Blue):** Both remain very low, near the bottom of the chart's visible range (probability ~0.4 or lower). **Chart 4 ("DeepSeek-Distill-Qwen3-8B"):** * **Boosted Sentence (Purple):** Is the highest line, fluctuating between approximately 0.3 and 0.45 probability. Annotated with **"= -75.71"**. * **Shared Sentence (Orange):** Shows a noisy but generally increasing trend from near 0 to about 0.1 probability. Annotated with **"= 9.61"**. * **Student Sentence (Blue):** Remains very low, near zero, with a slight increase towards the end. Annotated with **"= 4.48"**. * **Teacher Sentence (Green):** Not visibly plotted in this chart. ### Key Observations 1. **Model/Condition Comparison:** The "Ours" model (Charts 1 & 2) shows a clear separation where the Teacher Sentence probability is high in one view and the Boosted Sentence probability is high in another. The "DeepSeek" model (Charts 3 & 4) shows a more mixed picture with the Teacher Sentence having moderate, volatile probability and the Boosted Sentence being the highest in its respective view. 2. **Annotation Anomaly:** Several numerical annotations are negative (e.g., "-11.07", "-75.71"), which is atypical for a metric labeled "Probability." This strongly suggests the annotations may represent a different calculated value (e.g., a score, difference, or log probability) rather than the raw probability shown on the y-axis. 3. **Line Visibility:** The Teacher Sentence line is only prominent in the first and third charts. The Boosted Sentence line is prominent in the second and fourth charts. This may indicate the charts are grouped to highlight different primary comparisons. 4. **Trend Consistency:** The Shared and Student Sentence types consistently show the lowest probabilities across all charts and models. ### Interpretation The data appears to analyze the internal probability dynamics of different sentence types within two language model distillation or training frameworks ("Ours" vs. "DeepSeek-Distill-Qwen3-8B"). The "Sentence Index" likely represents a sequence of tokens or steps in a generation or training process. The contrasting profiles between the "Ours" charts suggest that in this model, the "Teacher" and "Boosted" sentence types dominate under different conditions or metrics. The "DeepSeek" model shows less dominance by a single type, with the Teacher sentence exhibiting high variance, possibly indicating less stability. The most critical finding is the disconnect between the y-axis ("Probability") and the final annotations. The negative values for what should be a non-negative probability metric indicate these annotations are **not** the final y-axis values. They likely represent a derived performance score, a change relative to a baseline, or a different metric altogether (e.g., a reward signal). Without additional context, their exact meaning is uncertain, but their presence is a key feature of the data presentation. The charts collectively suggest that the "Boosted Sentence" type is engineered to achieve high probability in specific contexts, while the "Teacher Sentence" provides a baseline that can be volatile. </details> Figure 6: Position-wise distribution over the four sentence types for our internally trained model (left two panels) and the open-source DeepSeek-Distill-Qwen3-8B (right two panels). The x-axis denotes the sentence position, and the y-axis denotes the predicted probabilities of the four sentence types. Solid lines (—–) indicate probabilities when the answer is correct, while dashed lines(- - -) indicate probabilities when the answer is incorrect. $Δ$ denotes the area difference between the solid and dashed curves, which reflects the influence of each sentence type on answer correctness. Building on the above analysis, a natural idea is to emphasize, during training, patterns that are more indicative of answer correctness. Although the full distribution-decomposition framework requires output probabilities from three models (the teacher, the student, and the distilled model) to identify the most effective distribution post hoc, we show that Teacher Sentences/Student Sentences can be identified prior to training: Teacher Sentences/Student Sentences are those sentences for which the teacher assigns significantly higher/lower output probabilities than the student. Therefore, we propose divergence-aware sampling (DAS), which prioritizes training examples rich in Teacher Sentences and thereby implicitly targets a teacher-derived sequence-level distribution better aligned with the student’s learning capacity (Liu et al., 2025). This sampling distribution naturally mitigates misleading gradients and facilitates more effective knowledge transfer from teacher to student. Notably, our method only requires, for each token in the teacher-generated response, its predicted probability by both the teacher and the student. The teacher-side probabilities are naturally obtained during sampling—and are often exposed even by many closed-source APIs—while the student-side probabilities are readily computed from the local model. In contrast, classical logit-based distillation necessitates the teacher’s full-vocabulary logits (i.e., probabilities over the entire vocabulary) at every position. Even recent on-policy distillation methods—when simplified to operate on token-level probabilities—still require, for every token in the student’s generated outputs, the corresponding probabilities under both models. Critically, the teacher-side probabilities for the student’s outputs are typically unavailable for proprietary models. Table 3: Performance comparison with different settings of training data (RS vs. DAS). | Settings of training data | AIME24 | AIME25 | | --- | --- | --- | | Teacher: gpt-oss-120b Student: Qwen3-4B-Instruct-2507 | | | | 50K Math + RS ( $T=0.6$ ) | 81.7 | 71.9 | | 50K Math + DAS ( $T=0.6$ ) | 83.3 | 74.2 | | 50K Math + RS ( $T=1.0$ ) | 83.1 | 76.1 | | 100K Math + RS ( $T=1.0$ ) | 83.1 | 78.9 | | 50K Math + DAS ( $T=1.0$ ) | 85.0 | 79.2 | | Teacher: Qwen3-Next-80B-A3B-Thinking Student: Qwen3-4B-Instruct-2507 | | | | 25K Math + RS | 79.0 | 71.3 | | 25K Math + DAS | 82.5 | 71.9 | Building on the experimental setup in Section 3, we conduct a controlled comparison between DAS and random sampling (DS) under an identical sampling budget. As shown in Table 3, DAS consistently achieves higher test performance, and in several cases, even surpasses the results obtained by RS after scaling up its data volume. This demonstrates that DAS effectively identifies teacher-generated sequences whose distribution is better aligned with the student’s learning capacity. Further, as shown in the lower part of Table 3 and in Table 4, DAS maintains a clear advantage over random sampling across different teacher models and domains, validating the generalizability of the DAS method. Table 4: Performance comparison with different settings of training data (RS vs. DAS across domains). | Settings of training data | AIME25 | LCB v6 | GPQA-D | | --- | --- | --- | --- | | Teacher: gpt-oss-120b Student: Qwen3-4B-Instruct-2507 | | | | | 25K Math + 10K Code + 10K Science + RS | 74.6 | 44.1 | 65.5 | | 25K Math + 10K Code + 10K Science + DAS | 75.6 | 47.3 | 65.7 | Finally, DAS does not require re-sampling data for every new student model. For instance, as demonstrated in Section 7, data curated to match the learning capacity of the Qwen3-4B-Instruct-2507 student model generalizes effectively to the Qwen3-30B-A3B-Instruct-2507 student model. ## 5 Mixed-policy Distillation <details> <summary>x10.png Details</summary>  ### Visual Description \n ## Line Chart: Cut-off Ratio vs. Token Length ### Overview The image is a line chart with a filled area underneath, plotting the "Cut-off Ratio" against "Token Length." The chart illustrates how the cut-off ratio changes as the token length increases from 0 to over 60,000 tokens. The overall trend shows an initial sharp rise, followed by a period of fluctuation, and then a more volatile, increasing trend at higher token lengths. ### Components/Axes * **X-Axis (Horizontal):** * **Label:** "Token Length" * **Scale:** Linear scale from 0K to beyond 60K. * **Major Tick Marks:** 0K, 20K, 40K, 60K. * **Y-Axis (Vertical):** * **Label:** "Cut-off Ratio" * **Scale:** Linear scale from 0.0 to 0.8. * **Major Tick Marks:** 0.0, 0.2, 0.4, 0.6, 0.8. * **Data Series:** * A single data series represented by a solid orange line. * The area under the line is filled with a semi-transparent orange color. * **Legend:** No explicit legend is present. The single data series is identified by its color and axis labels. * **Spatial Layout:** The chart is contained within a rectangular frame. The axes form the left and bottom borders. The title or caption is not visible within the cropped image. ### Detailed Analysis * **Trend Description:** The line begins at a cut-off ratio of approximately 0.0 at 0K tokens. It rises steeply to about 0.3 by roughly 5K tokens. From ~5K to ~40K tokens, the ratio fluctuates in a band between approximately 0.15 and 0.3, with no strong directional trend. After ~40K tokens, the line exhibits a clear upward trend with significantly increased volatility (larger, more frequent peaks and valleys). The line reaches its highest point, a peak of approximately 0.7, at the far right of the chart, corresponding to a token length slightly beyond 60K. * **Key Data Points (Approximate):** * At Token Length 0K: Cut-off Ratio ≈ 0.0 * At Token Length ~5K: Cut-off Ratio ≈ 0.3 (first local peak) * At Token Length ~20K: Cut-off Ratio ≈ 0.25 * At Token Length ~40K: Cut-off Ratio ≈ 0.2 * At Token Length ~50K: Cut-off Ratio ≈ 0.35 * At Token Length ~60K: Cut-off Ratio ≈ 0.45 * At Token Length >60K (end of chart): Cut-off Ratio ≈ 0.7 (global maximum) ### Key Observations 1. **Two-Phase Behavior:** The data suggests two distinct regimes: a relatively stable, low-volatility phase for token lengths up to ~40K, and a high-volatility, increasing phase beyond ~40K. 2. **Increased Volatility with Length:** The magnitude of the fluctuations (the "spikiness" of the line) increases noticeably as token length grows, particularly after the 40K mark. 3. **Overall Positive Correlation:** Despite the fluctuations, the general direction of the line is upward, indicating a positive correlation between token length and cut-off ratio. 4. **Sharp Initial Increase:** The most rapid change in the ratio occurs at the very beginning, from 0K to ~5K tokens. ### Interpretation This chart likely visualizes a performance or behavioral metric of a system processing sequential data (like text tokens). The "Cut-off Ratio" could represent a truncation rate, an error rate, a resource utilization threshold, or a similarity score decay. The data suggests that the system's behavior is relatively predictable and stable for shorter sequences (under 40K tokens). However, as sequences become very long (approaching and exceeding 60K tokens), the measured outcome becomes both higher on average and much less predictable, as evidenced by the large swings in the ratio. This could indicate that the system encounters edge cases, memory limitations, or compounding errors when handling extremely long contexts, leading to more erratic cutoff behavior. The sharp initial rise might represent a warm-up phase or the point where a baseline process engages. The lack of a plateau suggests that, within the observed range, the cut-off ratio does not stabilize for very long sequences. </details> Figure 7: The ratio between cut-off responses under different token lengths. In the previous stages, we employed off-policy methods to approximate the teacher’s sequence-level distribution through high-quality data generation. Nevertheless, we find that the resulting student model still suffers from exposure bias (Ranzato et al., 2016): during training, the student is conditioned on the teacher’s prefix using teacher forcing, whereas at inference time, it must rely on its own autoregressive predictions, leading to a distribution mismatch. To empirically investigate this phenomenon, we use the student model trained in the previous round (50K DAS sampling, T=0.6; see Table 3) to re-generate the training data within its own context, in order to examine whether the student model is overly reliant on the teacher’s context. During inference, we set the maximum generation token length to 1.5 times the length of the teacher-provided solution, in order to compare the differences between the teacher’s reference response and the student’s self-generated counterpart. Figure 7 plots the cut-off rate of the student’s generated responses across different training-response lengths, where a higher cut-off rate indicates greater divergence between student and teacher behavior. The results reveal that, even on the training data, the student still exhibits substantial deviations from the teacher, and this discrepancy becomes increasingly pronounced as the length of the training response grows. This observation confirms that training with teacher forcing under longer teacher prefixes exacerbates exposure bias. To overcome these limitations, Chen et al. (2025a) present that on-policy data collection is an effective alternative method. Accordingly, we propose a mixed-policy distillation approach that synergistically combines off-policy and on-policy signals. Specifically, we first use the student model from the previous training round to re-generate responses for the training queries, and then identify instances that differ substantially from the teacher’s outputs, e.g., solutions that have been cut off in Figure 7. For these data points, we randomly cut off the solutions generated by the student and prompt the teacher to continue the generation, thereby enabling the teacher to provide targeted guidance on the student’s errors. We present an ablation study of the proposed mixed-policy distillation in Table 5. As described above, we collect 7.7K mixed-policy samples, and train the model with only one epoch across these samples. Our baseline is the model trained on the 50K DAS-generated dataset at temperature T=0.6 (Table 3). In addition, we investigate a masking variant, where student-generated portions are masked out, and only teacher-completed segments are retained for training. Table 5: Ablation of the mixed-policy distillation method. #Num: number of mixed-policy data. | #Num | Mask | AIME24 | AIME25 | | --- | --- | --- | --- | | Baseline | | | | | 50K DAS ( $T=0.6$ ) | — | 83.3 | 74.2 | | Mixed-Policy Variants | | | | | 7.7K | ✓ | 80.8 | 72.3 | | 7.7K | ✗ | 83.3 | 74.8 | To maintain a balanced proportion between mixed-policy and off-policy data during training, we introduce 20K additional off-policy samples for joint training with the mixed-policy data. Our main experimental observations are as follows: (1) The results indicate that our mixed-policy dataset, despite containing only 7.7K samples, is capable of enhancing the model’s performance. (2) The masking variant tends to yield worse performance. This is because masking removes the on-policy segments generated by the student, leaving only the off-policy segments from the teacher for training. The observed performance drop in this setting further demonstrates the importance of incorporating on-policy data during distillation. As shown in Table 7, we also validate the effectiveness of the mixed-policy distillation approach in our final training pipeline. Incorporating even a small amount of mixed-policy data yields measurable gains across strong models and diverse domains. These results motivate continued exploration of this promising direction in the future. ## 6 Overall Training Recipe In this section, we detail the concrete implementation of DASD-4B-Thinking. Our pipeline comprises (i) question collection, (ii) candidate response sampling, (iii) filtering to remove low-quality responses, and (iv) multi-stage training on the curated dataset. Beyond our core innovations, we also integrate well-established practices from prior work in this line to ensure the overall quality of responses. We present each component in turn and describe the associated design choices in detail below. ### 6.1 Question Collection Our goal is to collect challenging questions spanning a diverse set of domains in order to obtain responses that demonstrate meaningful reasoning. To this end, we select four representative domains: mathematical reasoning, code generation, scientific reasoning, and instruction following. To gather training questions, we utilize a variety of publicly available open-source datasets (Chen et al., 2025b; Nvidia, 2024; Ji et al., 2025), which summarize numerous high-quality questions. - Mathematical Reasoning. Our math questions are primarily sourced from the supervised fine-tuning dataset of NVIDIA AceReason (Chen et al., 2025b). We experimented with different question scales during training and, considering the diminishing returns with larger question scales, ultimately sampled about 105K questions. These questions include those from original sources such as NuminaMath-CoT (Li et al., 2024) and the Art of Problem Solving (AoPS) community forums. - Code Generation. Our code questions primarily come from the OpenCodeReasoning dataset (Ahmad et al., 2025), with data sources including TACO (Li et al., 2023a), CodeContests (Li et al., 2022), APPs (Hendrycks et al., 2021), and Codeforces. - Scientific Reasoning. Scientific questions are primarily collected from NVIDIA's OpenScience Reasoning dataset (Nvidia, 2024), which consists entirely of multiple-choice question-answer pairs. We prioritize questions with longer example answers, as they tend to better elicit the model's reasoning abilities. - Instruction Following. Instruction-following questions are sourced from AM-DeepSeek-R1-Distilled-1.4M (Ji et al., 2025), which contain a large number of subjective instructions from various datasets. ### 6.2 Response Sampling We select a representative student-teacher pair, Qwen3-4B-Instruct-2507 and gpt-oss-120b, as our primary model pair, highlighting the broad compatibility of our approach across diverse model families and architectures. During sampling, we leverage the teacher’s high-level reasoning capabilities to generate multiple candidate samples for each question. To enhance coverage of the teacher model's behavior, we follow the methodology outlined in Section 3, sampling multiple responses per question at both low and high temperatures. Furthermore, we apply divergence-aware sampling as described in Section 4 to prioritize examples that better support student learning while preserving the correct gradient directions. <details> <summary>x11.png Details</summary>  ### Visual Description \n ## Pie Charts: Instance and Token Level Distributions at Different Temperatures ### Overview The image displays four pie charts arranged in a 2x2 grid. They compare the distribution of data across four categories ("math", "code", "science", "instruction following") at two different levels of analysis ("Instance Level" and "Token Level") and under two different "temperature" settings (0.6 and 1.0). The charts illustrate how the proportional composition of the dataset changes with the temperature parameter. ### Components/Axes * **Chart Type:** Pie Charts (4 total). * **Titles:** Each chart has a two-line title indicating the analysis level and temperature. * Top-Left: `Instance Level Distribution` / `temperature = 0.6` * Top-Right: `Token Level Distribution` / `temperature = 0.6` * Bottom-Left: `Instance Level Distribution` / `temperature = 1.0` * Bottom-Right: `Token Level Distribution` / `temperature = 1.0` * **Legend/Categories:** The same four categories are used in all charts, represented by consistent colors: * **Orange:** `math` * **Gray:** `code` * **Blue:** `science` * **Beige/Light Yellow:** `instruction following` * **Data Labels:** Each pie slice is labeled with the category name, an absolute numerical value (with 'k' for thousands, 'M' for millions, 'G' for billions), and a percentage in parentheses. ### Detailed Analysis #### Chart 1: Instance Level Distribution (temperature = 0.6) - Top-Left * **math (Orange):** 45.4k instances (43.3%). This is the largest slice, positioned from the top-center extending clockwise. * **code (Gray):** 20.1k instances (19.2%). Positioned in the bottom-right quadrant. * **science (Blue):** 19.8k instances (18.9%). Positioned in the bottom-left quadrant. * **instruction following (Beige):** 19.6k instances (18.7%). Positioned in the top-left quadrant. * **Total Instances (Approximate):** 104.9k. #### Chart 2: Token Level Distribution (temperature = 0.6) - Top-Right * **math (Orange):** 873.5M tokens (60.4%). Dominates the chart, occupying the entire left half and more. * **code (Gray):** 371.2M tokens (25.7%). The second-largest slice, in the top-right quadrant. * **science (Blue):** 147.2M tokens (10.2%). A smaller slice in the bottom-right. * **instruction following (Beige):** 53.7M tokens (3.7%). The smallest slice, a thin wedge at the bottom. * **Total Tokens (Approximate):** 1.445 Billion. #### Chart 3: Instance Level Distribution (temperature = 1.0) - Bottom-Left * **math (Orange):** 103.3k instances (31.3%). Still the largest slice but proportionally smaller than at temp=0.6. Positioned top-center. * **code (Gray):** 91.3k instances (27.7%). Significantly larger share than at temp=0.6. Positioned in the right half. * **science (Blue):** 65.2k instances (19.8%). Slightly larger share. Positioned in the bottom-left. * **instruction following (Beige):** 70k instances (21.2%). Larger share. Positioned in the top-left. * **Total Instances (Approximate):** 329.8k. The total number of instances is notably higher at temperature 1.0. #### Chart 4: Token Level Distribution (temperature = 1.0) - Bottom-Right * **math (Orange):** 2.4G tokens (41.2%). Still the largest category but its dominance is greatly reduced compared to temp=0.6. Occupies the top-left portion. * **code (Gray):** 2.5G tokens (43.7%). Now the largest category by token count, a dramatic increase. Occupies the entire right half. * **science (Blue):** 681M tokens (11.8%). Slightly larger share. Positioned at the bottom. * **instruction following (Beige):** 186.6M tokens (3.2%). Similar small share. A thin wedge between math and science. * **Total Tokens (Approximate):** 5.768 Billion. The total token count is substantially higher at temperature 1.0. ### Key Observations 1. **Temperature Effect on Distribution:** Increasing the temperature from 0.6 to 1.0 causes a major shift in the data distribution. The dominance of "math" decreases significantly, while "code" becomes much more prominent, especially at the token level where it surpasses math. 2. **Instance vs. Token Disparity:** At temperature 0.6, "math" instances (43.3%) account for a vastly disproportionate share of tokens (60.4%), suggesting math instances are, on average, much longer. Conversely, "instruction following" instances (18.7%) account for a tiny fraction of tokens (3.7%), indicating they are very short. 3. **Scale Increase at Higher Temperature:** Both the total number of instances (~3.1x) and total tokens (~4.0x) are substantially higher at temperature 1.0, indicating the generation process produced more and/or longer outputs. 4. **Category Resilience:** The "science" category maintains a relatively stable proportion (18.9% to 19.8% of instances, 10.2% to 11.8% of tokens) across both temperatures. ### Interpretation This data visualizes the impact of the "temperature" parameter (a setting that controls randomness/creativity in AI model outputs) on the composition of a generated dataset. * **Lower Temperature (0.6):** Favors "math" content. The model produces outputs that are heavily skewed towards mathematical problems and solutions, and these outputs are particularly lengthy (high token-to-instance ratio). This suggests a more focused, deterministic, and perhaps repetitive generation mode. * **Higher Temperature (1.0):** Encourages diversity. The output distribution becomes more balanced, with "code" generation becoming the primary activity by volume (tokens). The significant increase in total output volume suggests the model is more prolific and exploratory at this setting. The drastic reduction in the token share of "math" implies that while math problems are still generated, the model's responses to them are likely shorter or less elaborated compared to the code it writes. **In essence, the charts demonstrate that tuning the temperature parameter doesn't just change the style of output, but fundamentally alters the topical focus and structural length of the generated content, shifting from lengthy, math-heavy responses at low temperature to voluminous, code-centric outputs at higher temperature.** </details> Figure 8: Data distribution. ### 6.3 Response Filtering Beyond our core innovations, we employ stringent response filtering strategies to remove low-quality samples: - Length-based Filtering. We compute response lengths using the student model’s tokenizer, whose segmentation may differ substantially from the teacher’s. Responses that exceed the training context length are discarded. - Structure-based Filtering. gpt-oss-120b occasionally invokes built-in tools to answer questions. Since our current focus is on distilling long CoT reasoning capabilities—and function calling is deferred to future work—we explicitly detect and filter out any responses containing function calls. Additionally, we require every response to include both a thinking process and a final answer. gpt-oss-120b uses a harmony template to separate the reasoning trace from the final answer; for student-model compatibility, we post-process the original outputs by replacing the native delimiters with `` $<think>$ '' and `` $</think>$ ''. - Repetitive Content Filtering. We observe that gpt-oss-120b tends to generate repetitive content, particularly at lower temperatures, including repeated paragraphs, sentences, or phrases within a single response. Such repetitions can induce the trained student to produce endlessly repetitive and excessively verbose outputs during inference. To mitigate this, we rigorously filter out data containing repetitive content using regular expressions and n-gram matching, preventing the student from internalizing undesirable patterns. Finally, we acquire a total of 105K low-temperature (T=0.6) and 330K high-temperature (T=1.0) responses. The resulting data distribution across various domains is shown in Figure 8. ### 6.4 Multi-stage Training As illustrated in Figure 2, our training pipeline comprises two main stages: temperature-scheduled learning and mixed-policy distillation. During temperature-scheduled learning, the sampling data used to train DASD-4B-Thinking undergoes a two-stage filtering and training process to better capture the teacher model’s distribution while effectively supporting the student model’s learning. In the subsequent mixed-policy distillation, we construct mixed-policy data via on-policy rejection sampling and off-policy teacher revision. This hybrid strategy mitigates exposure bias by providing targeted, error-aware supervision. #### 6.4.1 Temperature-scheduled Learning To better represent the teacher model’s sequence-level distribution, we follow Section 3, and perform two-stage SFT on Qwen3-4B-Instruct-2507: first using low-temperature sampling data, followed by high-temperature sampling data. Training configurations are kept identical across both stages. We use an initial learning rate of 5e-5 that decays to 1e-5 via a cosine scheduler. We set the cutoff length to 64K and employ greedy sequence packing to accelerate training. Given the substantial GPU memory demands of 64K-context training, we leverage ZeRO-3 optimization together with Liger kernels to reduce memory consumption. Training is conducted with a global batch size of 64 over 6 epochs, and we observe consistent performance improvements across epochs. #### 6.4.2 Mixed-policy Distillation To mitigate the exposure bias, and inspired by the effectiveness of on-policy data (Chen et al., 2025a), we propose a mixed-policy revision protocol. Starting from the DAS-curated training set, we sample 50K questions. Each question is fed to the student model trained in the previous stage to generate responses. To align with the teacher’s reference length, we cap the student’s generation at 1.5 times the token count of the corresponding teacher response to the same question. Among the above student-generated solutions, we identify 15K truncated responses. For each truncated response, we discard the portion after a randomly selected position located beyond half of its total length, and then employ the teacher model to rewrite this discarded part. The teacher continuations that pass predefined quality filters are retained for the student’s further fine-tuning, yielding a total of mixed-policy 12.7K data samples. ## 7 Experimental Evaluation ### 7.1 Benchmarks We evaluate models on five complementary reasoning benchmarks: - AIME24 & AIME25 (AIME, 2025): Problem sets from that year’s American Invitational Mathematics Examination (AIME) I/II, each comprising 30 challenging problems, focusing on mathematical reasoning and requiring the correct final answer. - GPQA Diamond (GPQA-D) (Rein et al., 2023): A graduate-level benchmark of 198 expert-written multiple-choice questions spanning physics, chemistry, and biology, emphasizing ''Google-proof" deep academic reasoning. - LiveCodeBench (LCB) (Jain et al., 2024): A continuously updated coding benchmark that mitigates data contamination through strict temporal partitioning. Beyond code generation, it evaluates self-repair, executable correctness, and test-output prediction. The benchmark is released in time-based snapshots; v5 contains problems collected from October 2024 to February 2025, and v6 covers problems collected from February 2025 to May 2025. ### 7.2 Baselines We release our model, , and evaluate its performance on five public reasoning benchmarks: AIME24, AIME25, LCB and GPQA - D. We compare against two families of state-of-the-art open-source models serving as our primary baselines. All baselines were selected for their demonstrated strength in reasoning and complex problem solving, ensuring a relevant and competitive evaluation context for . Particularly, we categorize these baselines based on the accessibility of their training data. This enables a dual-perspective assessment of : (i) against top-performing models trained on private or proprietary data, and (ii) against leading models representing fully transparent, reproducible research. - Open-Weights Only. This group comprises models that release their weights publicly but keep their training data as proprietary. These models often represent the best performance available from models where the full training process is not disclosed. Our comparison set includes: - The Qwen3 series (4B-Thinking-2507, 8B, 14B, 32B) (Yang et al., 2025): The world's most popular model family featuring mainline models (8B, 14B, 32B) with switchable reasoning modes, alongside a dedicated 4B-Thinking variant optimized for complex reasoning over long contexts. - DeepSeek-R1-0528-Qwen3-8B (Guo et al., 2025): A specialized 8B reasoning model distilled from the powerful teacher model DeepSeek-R1-0528 into a Qwen3-8B backbone. - The GLM-Z1 series (32B-0414, 9B-0414) (Zeng et al., 2024): Models built on the GLM-4 architecture, enhanced via extended reinforcement learning for mathematical, logical, and coding reasoning. - The Mistral 3 series (3B, 8B) (Mistral AI Team, 2025): Compact open-weight models that emphasize efficient reasoning, strong math and coding capabilities, and multilingual generalization, suitable for low-latency, low-memory deployments. - Open-Weights & Open-Data. This group releases both model weights and the curated reasoning datasets, enabling full reproducibility and fostering community-wide study of reasoning acquisition. Included models are: - AM-thinking-v1 (Ji et al., 2025) and OpenThoughts3-7B (Guha et al., 2025): These models demonstrate different open-data strategies. AM-thinking combines SFT on 2.9M distilled examples with RL on a Qwen2.5-32B base; OpenThoughts3 achieves strong 7B-level performance via SFT alone on its publicly released 1.2M high-quality reasoning traces. - The Pai-DistillQwen-ThoughtY series (Cai et al., 2025): A set of compact models (4B, 8B) distilled from DeepSeek-R1-0528 using 365K curated examples, accompanied by full dataset release. - POLARIS-4B-Preview (An et al., 2025): A model based on Qwen3-4B that highlights the effectiveness of scaling up RL on public data to significantly improve complex, long-context reasoning. - The Nemotron family (Bercovich et al., 2025): This collection includes OpenReasoning-Nemotron-7B, distilled from DeepSeek-R1-0528 on a massive 30M example open dataset, and Nemotron-Ultra-253B, a large-scale model targeting high-end reasoning tasks. <details> <summary>x12.png Details</summary>  ### Visual Description ## Scatter Plot: AIME 2025 Performance vs. Total Parameters ### Overview This is a scatter plot comparing the performance of various AI models on the AIME 2025 benchmark against their total parameter count. The chart highlights a specific model, "DASD-4B-Thinking (Ours)," and compares it against other models categorized by their weight and data openness. ### Components/Axes * **Chart Title:** "AIME 2025 Performance vs. Total Parameters" * **Y-Axis:** "AIME 2025 Pass@1 Score". The scale runs from 50 to 90, with major tick marks every 5 units (50, 55, 60, 65, 70, 75, 80, 85, 90). * **X-Axis:** "Total Parameters". The scale is logarithmic, with labeled tick marks at 4B, 10B, 32B, and 100B (where B likely denotes billion). * **Legend:** Located in the bottom-right quadrant. It defines three model categories: * **Gray Circle:** "Open-Weights Only" * **Tan/Gold Circle:** "Open-Weights & Open-Data" * **Orange Star:** "Our Model" * **Reference Line:** A horizontal, dashed orange line extends from the "Our Model" data point across the chart at a score of approximately 83.5. ### Detailed Analysis The plot contains 13 distinct data points. Below is a list of each model, its approximate parameter count (x-axis), its approximate AIME score (y-axis), and its category based on the legend. **Models with ~4B Parameters:** 1. **DASD-4B-Thinking (Ours):** ~4B parameters, Score: ~83.5. (Orange Star - "Our Model"). This is the highest-performing model on the chart. 2. **POLARIS-4B:** ~4B parameters, Score: ~79. (Tan Circle - "Open-Weights & Open-Data"). 3. **Mistral3-3B:** ~3B parameters, Score: ~72. (Gray Circle - "Open-Weights Only"). **Models with ~7B-8B Parameters:** 4. **Qwen3-4B-Thinking:** ~7B parameters, Score: ~79. (Gray Circle - "Open-Weights Only"). 5. **Nvidia-OpenReasoning-7B:** ~7B parameters, Score: ~77. (Gray Circle - "Open-Weights Only"). 6. **Mistral3-8B:** ~8B parameters, Score: ~78. (Gray Circle - "Open-Weights Only"). 7. **DeepSeek-R1-Qwen3-8B:** ~8B parameters, Score: ~76. (Gray Circle - "Open-Weights Only"). 8. **OpenThoughts3-7B:** ~7B parameters, Score: ~53. (Tan Circle - "Open-Weights & Open-Data"). This is a notable low-performer for its size. 9. **GLM-Z1-9B:** ~9B parameters, Score: ~56. (Gray Circle - "Open-Weights Only"). **Models with ~14B-32B Parameters:** 10. **Qwen3-14B:** ~14B parameters, Score: ~70. (Gray Circle - "Open-Weights Only"). 11. **GLM-Z1-32B:** ~32B parameters, Score: ~63. (Gray Circle - "Open-Weights Only"). 12. **Qwen3-32B:** ~32B parameters, Score: ~72. (Gray Circle - "Open-Weights Only"). 13. **AM-thinking-v1:** ~32B parameters, Score: ~74. (Tan Circle - "Open-Weights & Open-Data"). **Model with ~253B Parameters:** 14. **Nvidia-Nemotron-Ultra-253B:** ~253B parameters, Score: ~72. (Tan Circle - "Open-Weights & Open-Data"). ### Key Observations 1. **Performance Outlier:** The "Our Model" (DASD-4B-Thinking) significantly outperforms all other models, achieving the highest score (~83.5) with a relatively small parameter count (~4B). 2. **Efficiency Trend:** There is no clear positive correlation between parameter count and performance. Several smaller models (e.g., POLARIS-4B, Qwen3-4B-Thinking) outperform much larger models (e.g., Nvidia-Nemotron-Ultra-253B, Qwen3-32B). 3. **Low-Performing Cluster:** Two models, OpenThoughts3-7B and GLM-Z1-9B, form a distinct low-scoring cluster with scores in the 50s, despite having parameter counts similar to mid-performing 7B-8B models. 4. **Category Distribution:** "Open-Weights Only" models (gray) are the most numerous and show the widest performance variance (scores from ~56 to ~79). "Open-Weights & Open-Data" models (tan) are fewer but include both high performers (POLARIS-4B) and the largest model on the chart. ### Interpretation The data suggests that model architecture, training methodology, or data quality (factors not directly shown) are more critical to achieving high AIME 2025 scores than sheer model size. The standout performance of "DASD-4B-Thinking" implies a significant efficiency breakthrough, achieving state-of-the-art results with a compact model. The chart effectively argues that bigger is not always better in this benchmark context. The presence of low-scoring models in the 7B-9B range indicates that this parameter size is a competitive but volatile space where implementation details lead to drastically different outcomes. The horizontal reference line from the "Our Model" point serves as a visual benchmark, emphasizing the performance gap it has established over both smaller and substantially larger competitors. </details> <details> <summary>x13.png Details</summary>  ### Visual Description ## Scatter Plot: LiveCodeBench v5 Performance vs. Total Parameters ### Overview This is a scatter plot comparing the performance of various large language models on the LiveCodeBench v5 coding benchmark against their total parameter count. The chart highlights a specific model, "DASD-4B-Thinking (Ours)," positioning it relative to other open-weight models. The data suggests an analysis of model efficiency, showing performance (y-axis) versus model size (x-axis). ### Components/Axes * **Chart Title:** "LiveCodeBench v5 Performance vs. Total Parameters" * **Y-Axis:** "LiveCodeBench v5 Pass@1 Score". The scale runs from 45 to 75, with major tick marks at 45, 50, 55, 60, 65, 70, and 75. * **X-Axis:** "Total Parameters". The scale is logarithmic, with labeled tick marks at 4B, 10B, 32B, and 100B (B = Billion). * **Legend (Bottom-Right Corner):** A box titled "Model Category" defines three data series: * **Gray Circle:** "Open-Weights Only" * **Tan/Gold Circle:** "Open-Weights & Open-Data" * **Orange Star:** "Our Model" * **Highlighted Element:** A dashed orange horizontal line extends from the "Our Model" data point (score ~70) across the chart, serving as a visual reference for its performance level. ### Detailed Analysis **Data Points (Approximate Coordinates & Category):** The following lists all labeled models, their approximate position on the chart, and their category based on color. * **Top-Left Quadrant (High Score, Low Parameters):** * **DASD-4B-Thinking (Ours):** Orange Star. Position: X ≈ 4B, Y ≈ 70. This is the highlighted model. * **Top-Right Quadrant (High Score, High Parameters):** * **AM-thinking-v1:** Tan Circle. Position: X ≈ 32B, Y ≈ 70. * **Nvidia-Nemotron-Ultra-253B:** Tan Circle. Position: X ≈ 250B (estimated, far right), Y ≈ 68. * **Qwen3-32B:** Gray Circle. Position: X ≈ 32B, Y ≈ 66. * **Middle Region (Moderate Score, Moderate Parameters):** * **Nvidia-OpenReasoning-7B:** Tan Circle. Position: X ≈ 7B, Y ≈ 64. * **Qwen3-14B:** Gray Circle. Position: X ≈ 14B, Y ≈ 63. * **Mistral3-8B:** Gray Circle. Position: X ≈ 8B, Y ≈ 62. * **DeepSeek-R1-Qwen3-8B:** Gray Circle. Position: X ≈ 8B, Y ≈ 61. * **GLM-Z1-32B:** Gray Circle. Position: X ≈ 32B, Y ≈ 59. * **Bottom Region (Lower Score, Varying Parameters):** * **Mistral3-3B:** Gray Circle. Position: X ≈ 3B, Y ≈ 55. * **GLM-Z1-9B:** Gray Circle. Position: X ≈ 9B, Y ≈ 52. * **OpenThoughts3-7B:** Tan Circle. Position: X ≈ 7B, Y ≈ 52. **Trend Verification:** * **General Trend:** There is a loose positive correlation; models with higher parameter counts (right side) tend to have higher scores (top). However, there is significant variance, especially in the 7B-32B range. * **"Our Model" Trend:** The orange star (DASD-4B-Thinking) is a clear outlier. It achieves a score (~70) comparable to the top-performing models that have 8x to 60x more parameters (e.g., AM-thinking-v1 at 32B, Nemotron at ~253B). ### Key Observations 1. **Efficiency Outlier:** The model labeled "Our Model" (DASD-4B-Thinking) demonstrates exceptional parameter efficiency, matching the performance of much larger models. 2. **Performance Clustering:** Models cluster into rough performance tiers: * **Top Tier (Score ~68-70):** Includes "Our Model," AM-thinking-v1, and Nvidia-Nemotron-Ultra-253B. * **Upper-Mid Tier (Score ~61-66):** Includes Qwen3-32B, Nvidia-OpenReasoning-7B, Qwen3-14B, Mistral3-8B, DeepSeek-R1-Qwen3-8B. * **Lower-Mid Tier (Score ~52-59):** Includes GLM-Z1-32B, Mistral3-3B, GLM-Z1-9B, OpenThoughts3-7B. 3. **Category Distribution:** Both "Open-Weights Only" (gray) and "Open-Weights & Open-Data" (tan) models are spread across all performance tiers. The top-performing group contains models from both categories plus the highlighted "Our Model." ### Interpretation This chart is likely from a research paper or technical report introducing the "DASD-4B-Thinking" model. Its primary argument is that this new model achieves state-of-the-art or competitive performance on a coding benchmark while using significantly fewer parameters than existing models. * **What the data suggests:** The plot challenges the simple "bigger is better" paradigm in LLM scaling. It suggests that architectural innovations, training data quality, or training methodologies (implied by the "Thinking" in the model name) can lead to more efficient models that punch above their weight class. * **How elements relate:** The dashed reference line from the orange star visually reinforces the central claim: this 4B model performs at the level of 32B+ models. The inclusion of various other models provides context, showing the competitive landscape and where this new model fits. * **Notable Anomalies:** The most significant anomaly is the position of "Our Model." Another point of interest is that `Nvidia-OpenReasoning-7B` (tan) outperforms several larger gray models, suggesting the "Open-Data" component may contribute to efficiency. Conversely, `GLM-Z1-32B` (gray) underperforms relative to its size, scoring lower than several smaller models. </details> Figure 9: Performance versus model size on AIME25 (left) and LCB v5 (right). Each point denotes a model, with the x-axis representing model size (in number of parameters) and the y-axis indicating benchmark score. Points positioned toward the top-left indicate superior efficiency—i.e., higher performance at smaller scale. ### 7.3 Evaluation Setup All evaluations were conducted under a unified setup. We consistently set the temperature to 1.0 and top- $p$ to $1.0$ . For every benchmark, we sampled 64 responses per question and reported the average accuracy to ensure reliable and stable evaluation results. Given the extreme difficulty of AIME24 and AIME25, we set the maximum generation length to 102,400 tokens; For LiveCodeBench and GPQA - D, the limit was set to 81,920 tokens. ### 7.4 Main Results Table 6 summarizes our evaluation, robustly validating the effectiveness of our proposed framework. The results demonstrate that, with our enhanced sequence-level distillation pipeline, complex reasoning capabilities can be efficiently transferred from a large teacher to a lightweight 4B-parameter student, yielding state-of-the-art performance for its scale. Notably, as shown in Figure 9, not only outperforms all comparable-size models but also surpasses significantly larger counterparts (e.g., 32B) on multiple key benchmarks, highlighting both the effectiveness of our approach and the high efficiency of our training data. Mathematical Reasoning (AIME24, AIME25). On the most challenging mathematical reasoning benchmarks, achieves remarkable scores of 83.3 on AIME25 and 88.5 on AIME24, establishing state-of-the-art performance among all listed models. These scores confirm that delivers top-tier reasoning capability, even when compared to larger scale models. This exceptional efficiency is visualized in Figure 9, where is clearly positioned in the top-left, achieving superior performance at a fraction of the parameter cost of its competitors. Notably, 's performance: - In the ``Open-Weights & Open-Data'' category, our 4B model demonstrates clear superiority. It substantially outperforms the 32B AM-thinking-v1 on AIME25 (83.3 vs 74.4) and AIME24 (88.5 vs 85.3). This is particularly noteworthy given that AM-thinking-v1 was trained on 2.9M examples, whereas achieves this result using only 448K examples—a dataset roughly 6 times smaller. It also surpasses other strong open-data models, including NVIDIA-OpenReasoning-Nemotron-7B (84.7/78.2), which used a massive 30M dataset, and even NVIDIA-Nemotron-Ultra-253B (80.8/72.5), a model over 60 times its size. - In the ``Open-Weights Only'' category, also sets a new benchmark for compact reasoning models, decisively outperforming the strong Qwen3-4B-Thinking-2507 on AIME25 (83.3 vs. 81.3). It further exceeds several ``Open-Weights Only'' models of medium to large scale, including Qwen3-32B (81.4/72.9) and GLM-Z1-32B (80.8/63.6). These results strongly demonstrate that our refined, data-efficient sequence-level distillation pipeline effectively enhances ’s reasoning capabilities, enabling it to match or surpass models 8 to 60 times larger in parameter count. Table 6: Comparison across AIME24, AIME25, LiveCodeBench (v5/v6), and GPQA-D. | | AIME24 | AIME25 | LCB v5 | LCB v6 | GPQA-D | | --- | --- | --- | --- | --- | --- | | Open-Weights Only | | | | | | | Qwen3-4B-Thinking-2507 | - | 81.3 | - | 55.2 | 65.8 | | Qwen3-14B | 79.3 | 70.4 | 63.5 | - | 64.0 | | Qwen3-32B | 81.4 | 72.9 | 65.7 | - | 68.4 | | DeepSeek - R1 - 0528 - Qwen3 - 8B | 86.0 | 76.3 | 60.5 | - | 61.1 | | GLM - Z1 - 32B - 0414 | 80.8 | 63.6 | 59.1 | - | 66.1 | | GLM - Z1 - 9B - 0414 | 76.4 | 56.6 | 51.8 | - | 58.5 | | Mistral3 - 3B | - | 72.1 | 54.8 | - | 53.4 | | Mistral3 - 8B | - | 78.7 | 61.6 | - | 66.8 | | Open-Weights & Open-Data | | | | | | | AM - thinking - v1 | 85.3 | 74.4 | 70.3 | - | - | | POLARIS - 4B - Preview | 81.2 | 79.4 | - | - | - | | OpenThoughts3 - 7B | 69.0 | 53.3 | 51.7 | - | 53.7 | | Pai - DistillQwen - ThoughtY - 4B | 76.7 | - | - | - | 56.1 | | Pai - DistillQwen - ThoughtY - 8B | 76.7 | - | - | - | 62.1 | | NVIDIA - OpenReasoning - Nemotron - 7B | 84.7 | 78.2 | 63.9 | - | 61.4 | | NVIDIA - Nemotron - Ultra - 253B | 80.8 | 72.5 | 68.1 | - | 76.0 | | DASD-4B-Thinking (Ours) | 88.5 | 83.3 | 69.3 | 67.5 | 68.4 | Coding (LiveCodeBench). In code generation, scores 69.3 on LCB v5 and 67.5 on LCB v6, again demonstrating exceptional efficiency. On LCB v5, this score not only surpasses DeepSeek-R1-0528-Qwen3-8B (60.5) and Qwen3-14B (63.5), but also the NVIDIA-OpenReasoning-Nemotron-7B (63.9). On LCB v6, it outperforms the strong Qwen3-4B-Thinking-2507 (67.5 vs. 55.2). Our 4B model even surpasses Qwen3-32B (65.7), highlighting the high efficiency of our method in transferring complex code generation capabilities. Scientific QA (GPQA-D). achieves a remarkable 68.4 on GPQA-D. It not only outperforms its same-size counterparts but also closely approaches the performance of substantially larger models, e.g., Qwen3-32B (68.4) and NVIDIA-Nemotron-Ultra-253B (76.0). While GPQA is notoriously challenging for compact models due to its heavy reliance on parametric knowledge, our 4B model successfully narrows the gap to these massive baselines. This result strongly suggests that our innovations effectively maximize the utilization of limited capacity for scientific reasoning. Summary. Overall, delivers state-of-the-art performance across mathematical, coding, and scientific reasoning benchmarks, robustly validating the effectiveness and data efficiency of our sequence-level distillation framework. It provides new insights and a fresh perspective for developing compact, high-performing, and fully open reasoning models. ### 7.5 Ablations over training stages Sections 3, 4, and 5 have validated the individual contributions of our three core components through isolated experiments. In this section, we complement those findings with a holistic ablation of the full training pipeline, examining how performance evolves after each sequential stage. Results are summarized in Table 7. Table 7: Ablations over training stages on AIME24, AIME25, LiveCodeBench v5, LiveCodeBench v6, and GPQA-D. | | AIME24 | AIME25 | LCB v5 | LCB v6 | GPQA-D | | --- | --- | --- | --- | --- | --- | | Qwen3-4B-Instruct-2507 | - | 47.4 | - | 35.1 | 62.5 | | + Low-Temperature Training | 84.2 | 74.0 | 56.6 | 50.6 | 67.7 | | + High-Temperature Training | 87.7 | 83.0 | 68.4 | 67.2 | 67.6 | | + Mixed-Policy Distillation | 88.5 | 83.3 | 69.3 | 67.5 | 68.4 | Starting from the Qwen3-4B-Instruct-2507 baseline, we observe consistent performance improvements across the three stages: Low-temperature training (with DAS) delivers substantial initial gain, boosting AIME25 from 47.4% to 74.0% (+26.6%) and LCB v6 from 35.1% to 50.6% (+15.5%). This confirms that stable, low-variance gradient signals during early training are critical for establishing a solid reasoning foundation. High-temperature training (with DAS) further enhances performance across key benchmarks, advancing LCB v5 by +11.8% and LCB v6 by +16.6%, while also providing a notable +9.0% gain on AIME25. This demonstrates that diverse exploration under higher temperature effectively expands the policy's solution coverage once a stable baseline has been established. Mixed-policy distillation consistently yields performance gains even on top of an already strong model across all benchmarks (e.g., +0.8% on AIME24, +0.3% on AIME25, +0.9% on LCB v5, +0.3% on LCB v6, +0.8% on GPQA-D), supporting the effectiveness of mixed-policy distillation in addressing the exposure-bias issue with minimal training overhead. ### 7.6 Effect of Divergence-aware Sampling on Data Distribution <details> <summary>x14.png Details</summary>  ### Visual Description ## Histograms: Comparison of Sampling Distributions at Low vs. High Temperature ### Overview The image displays two side-by-side histograms comparing the probability distributions of samples generated by two methods—"Random" and "Divergence-aware Sampling"—under two different conditions: "low temperature" (left panel) and "high temperature" (right panel). Each panel includes a main density histogram and a supplementary rug plot below it. ### Components/Axes * **Titles:** "low temperature" (left panel), "high temperature" (right panel). * **Y-axis (both panels):** Labeled "Density". The scale is not numerically marked, indicating relative frequency. * **X-axis (both panels):** Labeled "Probability Distribution of Samples". The scale runs from 0.0 to 1.0, with major ticks at 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0. * **Legend (both panels):** Located in the top-left corner of each plot. * Gray square: "Random" * Orange square: "Divergence-aware Sampling" * **Rug Plots:** Located directly below each histogram, aligned with the x-axis. They show individual data points as vertical lines. Gray lines correspond to the "Random" method, and orange lines correspond to "Divergence-aware Sampling". ### Detailed Analysis **1. Low Temperature Panel (Left):** * **Trend Verification:** The "Divergence-aware Sampling" (orange) distribution is sharply peaked and narrow. The "Random" (gray) distribution is broader and shorter, but its central mass overlaps significantly with the orange distribution. * **Data Points & Distribution:** * **Divergence-aware Sampling (Orange):** The distribution is unimodal and highly concentrated. It peaks sharply in the interval approximately between 0.7 and 0.8. The bulk of its density lies between ~0.6 and ~0.9. The rug plot shows a dense cluster of orange lines in this same range. * **Random (Gray):** The distribution is wider and flatter. Its central tendency appears to be in a similar region as the orange distribution (around 0.7-0.8), but it has significant density extending from approximately 0.5 to 0.9. The gray rug lines are more spread out than the orange ones. **2. High Temperature Panel (Right):** * **Trend Verification:** Both distributions are wider and more overlapping compared to the low-temperature case. The "Divergence-aware Sampling" (orange) distribution is still somewhat more concentrated than the "Random" (gray) one, but the distinction is less pronounced. * **Data Points & Distribution:** * **Divergence-aware Sampling (Orange):** The distribution is broader than in the low-temperature case. It appears bimodal or has a wide plateau, with high density spanning from approximately 0.4 to 0.7. The peak is less defined. The orange rug lines are spread across this wider interval. * **Random (Gray):** This distribution is the broadest of all. It spans from roughly 0.3 to 0.9, with a central mass around 0.5 to 0.7. It shows a notable secondary concentration or shoulder around 0.7-0.8. The gray rug lines are very widely dispersed. ### Key Observations 1. **Effect of Temperature:** Increasing the temperature parameter dramatically increases the variance (spread) of the sample distributions for both methods. 2. **Method Comparison:** At both temperatures, "Divergence-aware Sampling" produces a more concentrated distribution of sample probabilities than "Random" sampling. This effect is much stronger at low temperature. 3. **Distribution Shift:** The central tendency of the "Divergence-aware Sampling" distribution shifts leftward (toward lower probability values) as temperature increases, moving from a peak near 0.75 (low temp) to a broader mass centered around 0.55 (high temp). The "Random" distribution's center shifts less dramatically. 4. **Overlap:** The overlap between the two methods' distributions is significantly greater at high temperature. ### Interpretation This visualization demonstrates the impact of a "temperature" parameter on sampling behavior in a probabilistic model. Temperature typically controls the randomness or entropy of sampling; low temperature leads to more deterministic, confident sampling (peaked distributions), while high temperature leads to more exploratory, uniform sampling (broad distributions). The key finding is that **"Divergence-aware Sampling" consistently yields samples with a narrower range of probabilities compared to random sampling, acting as a regularizing or focusing mechanism.** This suggests the method is effective at avoiding extreme or low-probability samples. The effect is most potent at low temperature, where it creates a very sharp peak. At high temperature, where the system is inherently more random, the method still provides some constraining influence, but its advantage over pure random sampling is reduced. The leftward shift of the orange distribution at high temperature indicates that when forced to be more random, the divergence-aware method tends to select samples with moderately lower probabilities than the random baseline, possibly avoiding the high-probability modes that random sampling might still frequently hit. </details> Figure 10: Comparison of response probability distributions with/without divergence-aware sampling using different temperatures. To broaden coverage of the teacher’s output modes, we employ temperature-scheduled learning to collect both low-temperature samples (sharper, more concentrated distributions that cover a narrower probability range around high-probability regions) and high-temperature samples (flatter, broader densities that capture rarer teacher modes). To better identify the target sequence-level distribution that supports effective student learning, we further apply divergence-aware sampling to both data subsets. As illustrated in Figure 10, divergence-aware sampling induces negligible perturbation to the underlying response probability distribution, confirming its orthogonality to temperature scheduling. This decoupling underpins the strong synergy observed when combining the two strategies, as evidenced by the substantial performance gains in Section 7.5. ### 7.7 Evaluation on MoE Models To assess the scalability and architectural robustness of our distillation framework, we further extend it to Mixture-of-Experts (MoE) students, which increase model capacity while maintaining inference efficiency via sparse expert routing. We select Qwen3-30B-A3B-Instruct-2507 as the student model. For this investigation, we conduct a preliminary evaluation by applying only the first stage of our pipeline (i.e., Low-Temperature Training with DAS), yielding the model denoted as DASD-30B-A3B-Thinking-Preview. Crucially, to test the cross-architecture transferability of our data, we do not re-collect or re-curate training samples; instead, we directly reuse the exact dataset curated for the Qwen3-4B student. Table 8 summarizes the results. All baseline results are quoted from the corresponding official technical reports. Table 8: Performance comparison on MoE models. | Model | AIME25 | LCB v6 | GPQA-D | Average | | --- | --- | --- | --- | --- | | gpt-oss-20b | 91.7 | 61.0* | 71.5 | 74.7 | | Qwen3-30B-A3B-Thinking-2507 | 85.0 | 66.0 | 73.4 | 74.8 | | NVIDIA-Nemotron-3-Nano-30B-A3B | 89.1 | 68.3 | 73.0 | 76.8 | | DASD-30B-A3B-Thinking-Preview (Ours) | 86.7 | 72.8 | 72.3 | 77.3 | * indicates the number is taken from the technical report of NVIDIA-Nemotron-3. Despite being a preview version that has not been thoroughly trained, DASD-30B-A3B-Thinking-Preview already demonstrates strong competitiveness against powerful open MoE baselines. Compared with Qwen3-30B-A3B-Thinking-2507, it achieves robust gains across key benchmarks, boosting AIME25 to 86.7% (+1.7%) and LCB v6 to 72.8% (+6.8%), while maintaining competitive performance on GPQA-D. When evaluated against gpt-oss-20b, our model secures a substantial lead in coding tasks, elevating the LCB v6 score to 72.8% (+11.8%) and driving the overall average to 77.3% (+2.6%). We further compare with NVIDIA-Nemotron-3-Nano-30B-A3B (NVIDIA, 2025): according to its technical report, it is trained with a large-scale SFT corpus (18M) and additional RL, whereas our model uses only the 105K distilled dataset from the first stage of our pipeline, with no extra data or RL involved. Even under this much lighter training recipe, DASD-30B-A3B-Thinking-Preview delivers stronger coding performance on LCB v6 (+4.5; 72.8 vs. 68.3) and a higher average score (+0.5; 77.3 vs. 76.8), highlighting that careful pipeline design can achieve superior efficiency-quality trade-offs at MoE scale. ## 8 Conclusion and Future Work In this technical report, we introduce DASD-4B-Thinking, a high-performance large reasoning model developed through large-scale knowledge distillation. We critically re-examine the prevailing paradigm of SFT on teacher-generated responses and identify three key limitations: (i) inadequate coverage of the teacher’s sequence-level distribution; (ii) misalignment between the teacher’s output distribution and the student’s learning capacity; and (iii) exposure bias stemming from teacher forcing during training versus autoregressive inference at test time. To address these challenges, we propose a comprehensive data construction and training pipeline built upon three core innovations: temperature-scheduled learning, divergence-aware sampling, and mixed-policy distillation. Leveraging this pipeline and training on only 448K samples, DASD-4B-Thinking achieves state-of-the-art performance across the majority of reasoning benchmarks—consistently outperforming models of comparable scale and, notably, surpassing several larger counterparts. Complementing this, we also release DASD-30B-A3B-Thinking-Preview, a MoE variant that attains competitive or superior results across the same benchmarks. To promote reproducibility and community advancement, we open-source both the curated dataset and the trained models. Looking ahead, we outline the following directions for future work. First, we will explore distribution-aware reweighting during SFT, leveraging the teacher model’s sequence-level output probabilities to more faithfully approximate its target distribution, thereby improving both distillation effectiveness and data efficiency. Second, we aim to further refine the mixed-policy distillation approach to enhance training efficiency and stability. Finally, we plan to integrate complementary agentic capabilities—such as knowledge retrieval and tool use—to progressively develop more powerful, domain-adapted reasoning models capable of handling complex, real-world tasks. ## 9 Acknowledgement We would like to thank the following individuals (listed in alphabetical order by last name) for their valuable discussions and for their help with experiments and evaluation: Xin Guo, Chenxi Huang, Zhen Huang, Yao Liu, Rui Miao, Ge Teng, Jingyi Shen, Xu Shen, Chaoqun Wan, Yue Xin, Xiaosong Yuan. ## References - R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. R. Garea, M. Geist, and O. Bachem (2024) On-policy distillation of language models: learning from self-generated mistakes. In International Conference on Learning Representations, Cited by: §1, §4. - S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, H. Bao, et al. (2025) Gpt-oss-120b & gpt-oss-20b model card. CoRR abs/2508.10925. Cited by: §1. - W. U. Ahmad, S. Narenthiran, S. Majumdar, A. Ficek, S. Jain, J. Huang, V. Noroozi, and B. Ginsburg (2025) OpenCodeReasoning: advancing data distillation for competitive coding. CoRR abs/2504.01943. Cited by: §1, 2nd item. - AIME (2025) AIME problems and solutions. External Links: Link Cited by: 1st item. - C. An, Z. Xie, X. Li, L. Li, J. Zhang, S. Gong, M. Zhong, J. Xu, X. Qiu, M. Wang, and L. Kong (2025) POLARIS: a post-training recipe for scaling reinforcement learning on advanced reasoning models. External Links: Link Cited by: 3rd item. - A. Bercovich, I. Levy, I. Golan, M. Dabbah, R. El-Yaniv, O. Puny, I. Galil, Z. Moshe, T. Ronen, N. Nabwani, et al. (2025) Llama-nemotron: efficient reasoning models. arXiv preprint arXiv:2505.00949. Cited by: §1, 4th item. - W. Cai, C. Wang, J. Yan, J. Huang, and X. Fang (2025) Reasoning with omnithought: A large cot dataset with verbosity and cognitive difficulty annotations. CoRR abs/2505.10937. Cited by: §1, 2nd item. - M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin (2021) Emerging properties in self-supervised vision transformers. In IEEE/CVF International Conference on Computer Vision, pp. 9630–9640. Cited by: §3. - H. Chen, N. Razin, K. Narasimhan, and D. Chen (2025a) Retaining by doing: the role of on-policy data in mitigating forgetting. CoRR abs/2510.18874. Cited by: §5, §6.4.2. - Y. Chen, Z. Yang, Z. Liu, C. Lee, P. Xu, M. Shoeybi, B. Catanzaro, and W. Ping (2025b) AceReason-nemotron: advancing math and code reasoning through reinforcement learning. CoRR abs/2505.16400. Cited by: §1, 1st item, §6.1. - Y. Gu, L. Dong, F. Wei, and M. Huang (2024) MiniLLM: knowledge distillation of large language models. In International Conference on Learning Representations, Cited by: §4. - E. K. Guha, R. Marten, S. Keh, N. Raoof, G. Smyrnis, H. Bansal, M. Nezhurina, J. Mercat, T. Vu, Z. Sprague, et al. (2025) OpenThoughts: data recipes for reasoning models. CoRR abs/2506.04178. Cited by: §1, 1st item. - D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. (2025) DeepSeek-r1 incentivizes reasoning in llms through reinforcement learning. Nature 645 (8081), pp. 633–638. Cited by: §1, §4, 2nd item. - Z. He, T. Liang, J. Xu, Q. Liu, X. Chen, Y. Wang, L. Song, D. Yu, Z. Liang, W. Wang, Z. Zhang, R. Wang, Z. Tu, H. Mi, and D. Yu (2025) DeepMath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning. CoRR abs/2504.11456. Cited by: §1. - D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song, and J. Steinhardt (2021) Measuring coding challenge competence with APPS. In Neural Information Processing Systems Track on Datasets and Benchmarks, Cited by: 2nd item. - G. E. Hinton, O. Vinyals, and J. Dean (2015) Distilling the knowledge in a neural network. CoRR abs/1503.02531. Cited by: §1, §1, §2. - A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi (2020) The curious case of neural text degeneration. In International Conference on Learning Representations, Cited by: §3. - Hugging Face (2025) Open r1: a fully open reproduction of deepseek-r1. External Links: Link Cited by: §1. - N. Jain, K. Han, A. Gu, W. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica (2024) LiveCodeBench: holistic and contamination free evaluation of large language models for code. CoRR abs/2403.07974. Cited by: 3rd item. - E. Jang, S. Gu, and B. Poole (2017) Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, Cited by: §3. - Y. Ji, X. Tian, S. Zhao, H. Wang, S. Chen, Y. Peng, H. Zhao, and X. Li (2025) AM-thinking-v1: advancing the frontier of reasoning at 32b scale. CoRR abs/2505.08311. Cited by: 4th item, §6.1, 1st item. - J. Jung, S. Han, X. Lu, S. Hallinan, D. Acuna, S. Prabhumoye, M. Patwary, M. Shoeybi, B. Catanzaro, and Y. Choi (2025) Prismatic synthesis: gradient-based data diversification boosts generalization in LLM reasoning. CoRR abs/2505.20161. Cited by: §1. - A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, L. Rouillard, et al. (2025) Gemma 3 technical report. CoRR abs/2503.19786. Cited by: §1. - Y. Kim and A. M. Rush (2016) Sequence-level knowledge distillation. In Conference on Empirical Methods in Natural Language Processing, pp. 1317–1327. Cited by: §1, §1, §2, §2. - Z. Lei, Z. Tan, S. Wang, Y. Zhu, Z. Chen, Y. Dong, and J. Li (2025) Learning from diverse reasoning paths with routing and collaboration. CoRR abs/2508.16861. Cited by: §1, §3. - H. Li, K. Yang, Y. Chu, H. Liu, and J. Tang (2025a) Exploring solution divergence and its effect on large language model problem solving. CoRR abs/2509.22480. Cited by: §1. - J. Li, E. Beeching, L. Tunstall, B. Lipkin, R. Soletskyi, S. C. Huang, K. Rasul, L. Yu, A. Jiang, Z. Shen, Z. Qin, B. Dong, L. Zhou, Y. Fleureau, G. Lample, and S. Polu (2024) NuminaMath. Numina. Note: [https://huggingface.co/AI-MO/NuminaMath-CoT](https://github.com/project-numina/aimo-progress-prize/blob/main/report/numina_dataset.pdf) Cited by: 1st item. - R. Li, J. Fu, B. Zhang, T. Huang, Z. Sun, C. Lyu, G. Liu, Z. Jin, and G. Li (2023a) TACO: topics in algorithmic code generation dataset. CoRR abs/2312.14852. Cited by: 2nd item. - X. Li, Y. Xiao, D. Ng, H. Ye, Y. Deng, X. Lin, B. Wang, Z. Mo, C. Zhang, Y. Zhang, Z. Yang, R. Li, L. Lei, S. Xu, H. Zhao, W. Chen, F. Ji, and L. Bing (2025b) MiroMind-m1: an open-source advancement in mathematical reasoning via context-aware multi-stage policy optimization. CoRR abs/2507.14683. Cited by: §1. - Y. Li, Y. Emad, K. Padthe, J. Lanchantin, W. Yuan, T. Nguyen, J. Weston, S. Li, D. Wang, I. Kulikov, and X. Li (2025c) NaturalThoughts: selecting and distilling reasoning traces for general reasoning tasks. CoRR abs/2507.01921. Cited by: §1. - Y. Li, D. H. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. D. Lago, T. Hubert, P. Choy, C. de Masson d'Autume, I. Babuschkin, X. Chen, P. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. S. Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals (2022) Competition-level code generation with alphacode. CoRR abs/2203.07814. Cited by: 2nd item. - Z. Li, X. Li, L. Yang, B. Zhao, R. Song, L. Luo, J. Li, and J. Yang (2023b) Curriculum temperature for knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, pp. 1504–1512. Cited by: §3. - K. Liu, S. Yan, R. Miao, B. Wang, C. Shen, J. Zhang, and J. Ye (2025) Where did this sentence come from? tracing provenance in llm reasoning distillation. arXiv preprint arXiv:2512.20908. Cited by: §4, §4. - K. Lu and T. M. Lab (2025) On-policy distillation. Thinking Machines Lab: Connectionism. Note: https://thinkingmachines.ai/blog/on-policy-distillation External Links: Document Cited by: §1. - J. Mattern, S. Jaghouar, M. Basra, J. Straube, M. D. Ferrante, F. Gabriel, J. M. Ong, V. Weisser, and J. Hagemann (2025) SYNTHETIC-1: two million collaboratively generated reasoning traces from deepseek-r1. External Links: Link Cited by: §1. - Mistral AI Team (2025) Mistral 3. Note: https://mistral.ai/news/mistral-3 Accessed: 2025-12-03 Cited by: 4th item. - I. Moshkov, D. Hanley, I. Sorokin, S. Toshniwal, C. Henkel, B. Schifferer, W. Du, and I. Gitman (2025) AIMO-2 winning solution: building state-of-the-art mathematical reasoning models with openmathreasoning dataset. CoRR abs/2504.16891. Cited by: §1. - N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. J. Candès, and T. Hashimoto (2025) S1: simple test-time scaling. CoRR abs/2501.19393. Cited by: §1. - Nvidia (2024) OpenScience. Nvidia. Note: [https://huggingface.co/datasets/nvidia/OpenScience](https://huggingface.co/datasets/nvidia/OpenScience) Cited by: 3rd item, §6.1. - NVIDIA (2025) Nemotron 3 Nano: open, efficient mixture-of-experts hybrid Mamba-Transformer model for Agentic reasoning. Note: Technical report External Links: Link Cited by: §7.7. - M. Ranzato, S. Chopra, M. Auli, and W. Zaremba (2016) Sequence level training with recurrent neural networks. In International Conference on Learning Representations, Cited by: §5. - D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman (2023) GPQA: a graduate-level Google-proof Q&A benchmark. CoRR abs/2311.12022. Cited by: 2nd item. - N. Team (2025) Sky-t1: train your own o1 preview model within $450. Note: https://novasky-ai.github.io/posts/sky-t1Accessed: 2025-01-09 Cited by: §1. - L. Wen, Y. Cai, F. Xiao, X. He, Q. An, Z. Duan, Y. Du, J. Liu, L. Tang, X. Lv, H. Zou, Y. Deng, S. Jia, and X. Zhang (2025) Light-r1: curriculum sft, DPO and RL for long COT from scratch and beyond. CoRR abs/2503.10460. Cited by: §1. - X. Wu, X. Jiang, H. Li, J. Zhai, D. Liu, Q. Hao, H. Liu, Z. Yang, J. Xie, N. Gu, J. Yang, K. Zhang, Y. Bao, and J. Wang (2025) Beyond scaling law: A data-efficient distillation framework for reasoning. CoRR abs/2508.09883. Cited by: §1. - J. Yan, L. Liu, Y. Pan, S. Chen, Y. Xiang, and B. Tang (2025) Towards efficient cot distillation: self-guided rationale selector for better performance with fewer rationales. CoRR abs/2509.23574. Cited by: §3. - A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. (2025) Qwen3 technical report. CoRR abs/2505.09388. Cited by: §1, §1, 1st item. - Y. Ye, Z. Huang, Y. Xiao, E. Chern, S. Xia, and P. Liu (2025) LIMO: less is more for reasoning. CoRR abs/2502.03387. Cited by: §1. - A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Rojas, G. Feng, H. Zhao, H. Lai, H. Yu, et al. (2024) ChatGLM: a family of large language models from GLM-130B to GLM-4 all tools. CoRR abs/2406.12793. Cited by: 3rd item. - H. Zhao, H. Wang, Y. Peng, S. Zhao, X. Tian, S. Chen, Y. Ji, and X. Li (2025) 1.4 million open-source distilled reasoning dataset to empower large language model training. CoRR abs/2503.19633. Cited by: §1. - J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. L. Yuille, and T. Kong (2021) IBOT: image BERT pre-training with online tokenizer. CoRR abs/2111.07832. Cited by: §3.