# Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models
**Authors**: Nanbeige LLM Lab, Boss Zhipin
<details>
<summary>x1.png Details</summary>
### Visual Description
Icon/Small Image (195x51)
</details>
## Abstract
We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, we extend the boundary of the scaling law for small language models. In pre-training, we design a Fine-Grained Warmup-Stable-Decay (FG-WSD) training scheduler, which progressively refines data mixtures across stages to boost model performance. In post-training, to improve the quality of the SFT data, we design a joint mechanism that integrates deliberative generation refinement and chain-of-thought reconstruction, yielding substantial gains on complex tasks. Following SFT, we employ our flagship reasoning model to distill Nanbeige4-3B through our proposed Dual Preference Distillation (DPD) method, which leads to further performance gains. Finally, a multi-stage reinforcement learning phase was applied, leveraging verifiable rewards and preference modeling to strengthen abilities on both reasoning and human alignment. Extensive evaluations show that Nanbeige4-3B not only significantly outperforms models of comparable parameter scale but also rivals much larger models across a wide range of benchmarks. The model checkpoints are available at https://huggingface.co/Nanbeige.
<details>
<summary>figures/nbg_performance.png Details</summary>
### Visual Description
\n
## Bar Chart: Model Performance on Various Benchmarks
### Overview
The image presents a bar chart comparing the performance of several language models (NanBeigel-3B, Qwen-3-4B-2507, Qwen3-14B, Qwen3-30B-A3B-2507) across six different benchmarks: AIME 2024, AIME 2025, GPQA-Diamond, SuperGPQA, BFCL-v4, Fullstack Bench, ArenaHard-v2, and Multi-Challenge. The performance metric appears to be a score, likely accuracy or a similar measure.
### Components/Axes
* **X-axis:** Represents the different language models: NanBeigel-3B, Qwen-3-4B-2507, Qwen3-14B, Qwen3-30B-A3B-2507.
* **Y-axis:** Represents the performance score, ranging from approximately 20 to 90. The scale is not explicitly labeled, but can be inferred from the values displayed on the bars.
* **Chart Title:** The chart is divided into sections, each with a title indicating the benchmark being evaluated: Mathematical Reasoning (AIME 2024, AIME 2025), Scientific Reasoning (GPQA-Diamond, SuperGPQA), Tool Use & Coding (BFCL-v4, Fullstack Bench), and Human Preference Alignment (ArenaHard-v2, Multi-Challenge).
* **Bars:** Each bar represents the performance of a specific model on a specific benchmark. The color of the bars is consistent within each benchmark section.
### Detailed Analysis
**Mathematical Reasoning**
* **AIME 2024:**
* NanBeigel-3B: Approximately 90.4
* Qwen-3-4B-2507: Approximately 83.3
* Qwen3-14B: Approximately 79.3
* Qwen3-30B-A3B-2507: Approximately 81.4, 76.0
* **AIME 2025:**
* NanBeigel-3B: Approximately 89.2
* Qwen-3-4B-2507: Approximately 81.3
* Qwen3-14B: Approximately 70.4
* Qwen3-30B-A3B-2507: Approximately 72.9, 85.0
**Scientific Reasoning**
* **GPQA-Diamond:**
* NanBeigel-3B: Approximately 82.2
* Qwen-3-4B-2507: Approximately 67.2
* Qwen3-14B: Approximately 62.0
* Qwen3-30B-A3B-2507: Approximately 64.0, 68.7
* **SuperGPQA:**
* NanBeigel-3B: Approximately 53.2
* Qwen-3-4B-2507: Approximately 46.7
* Qwen3-14B: Approximately 39.1
* Qwen3-30B-A3B-2507: Approximately 54.1, 50.8, 46.8
**Tool Use & Coding**
* **BFCL-v4:**
* NanBeigel-3B: Approximately 53.8
* Qwen-3-4B-2507: Approximately 44.9
* Qwen3-14B: Approximately 45.4
* Qwen3-30B-A3B-2507: Approximately 47.9, 48.6, 42.2
* **Fullstack Bench:**
* NanBeigel-3B: Approximately 58.2
* Qwen-3-4B-2507: Approximately 48.0
* Qwen3-14B: Approximately 47.1
* Qwen3-30B-A3B-2507: Approximately 51.5, 54.4
**Human Preference Alignment**
* **ArenaHard-v2:**
* NanBeigel-3B: Approximately 60.0
* Qwen-3-4B-2507: Approximately 40.5
* Qwen3-14B: Approximately 39.9
* Qwen3-30B-A3B-2507: Approximately 26.4
* **Multi-Challenge:**
* NanBeigel-3B: Approximately 49.4
* Qwen-3-4B-2507: Approximately 41.2
* Qwen3-14B: Approximately 35.8
* Qwen3-30B-A3B-2507: Approximately 36.4, 39.2
### Key Observations
* **NanBeigel-3B consistently outperforms other models** across most benchmarks, particularly in Mathematical and Scientific Reasoning.
* **Qwen3-30B-A3B-2507 often shows the second-best performance**, frequently surpassing Qwen-3-4B-2507 and Qwen3-14B.
* **Qwen3-14B generally performs worse than Qwen-3-4B-2507**, except in some instances within the Tool Use & Coding category.
* **Performance varies significantly across benchmarks.** Models that excel in one area may not perform as well in others.
* The presence of multiple bars for Qwen3-30B-A3B-2507 in some benchmarks suggests multiple runs or variations of the model were evaluated.
### Interpretation
The data suggests that NanBeigel-3B is the strongest performing model among those tested, demonstrating superior capabilities in mathematical, scientific, and general reasoning tasks. Qwen3-30B-A3B-2507 represents a significant step up in performance compared to the smaller Qwen models, indicating that increasing model size generally leads to improved results. The varying performance across benchmarks highlights the importance of evaluating models on a diverse set of tasks to gain a comprehensive understanding of their strengths and weaknesses. The multiple bars for Qwen3-30B-A3B-2507 could indicate the impact of different training configurations or data subsets on the model's performance. The relatively lower scores on the Human Preference Alignment benchmarks suggest that these tasks may be more challenging for the models, or that the evaluation metrics do not fully capture human preferences. Overall, the chart provides valuable insights into the relative performance of these language models and can inform future research and development efforts.
</details>
Figure 1: Performance Comparison between Nanbeige4-3B-Thinking and Qwen series models.
## 1 Introduction
In recent years, we have witnessed the emergence of capable large-scale language models ranging from hundreds of billions to trillions of parameters yang2025qwen3; DBLP:journals/corr/abs-2508-06471; deepseekai2025deepseekr1incentivizingreasoningcapability; lingteam2025activationboostedscalinggeneral. They have demonstrated remarkable reasoning abilities and have become a pivotal driving force in the evolution of artificial intelligence. Despite their impressive performance, these models come with substantial inference costs in deployment and high training expenses for research, whether for full-scale replication or fine-tuning. This scenario underscores the importance of exploring the potential of Small Language Models (SLMs) as a resource-efficient alternative.
In this work, we present the Nanbeige4-3B model family. Despite their compact size, they exhibit remarkably strong and well-balanced capabilities in mathematics, scientific reasoning, human preference alignment, creative writing, and tool use—substantially outperforming many larger models. These results highlight the effectiveness of our training methodology and clearly demonstrate that small, well-engineered models can achieve performance that surpasses far larger models.
To provide a clear sense of the model’s capability, we compare Nanbeige4-3B against the Qwen3 series across multiple parameter scales. For the base models, we conduct several post-training runs using identical SFT datasets, where Nanbeige4-3B-Base significantly outperforms Qwen3-8B-Base. For the reasoning model, Nanbeige4-3B-Thinking demonstrates overall performance that surpasses Qwen3-14B on average. Notably, it even outperforms the substantially larger Qwen3-32B and Qwen3-30B-A3B on mathematical and scientific reasoning, as well as on specific dimensions concerning tool use and human preference alignment. Moreover, in terms of creative writing ability, Nanbeige4-3B-Thinking approaches the performance of several state-of-the-art large models, as elaborated in the WritingBench Leaderboard November 2025.
Table 1: Performance comparison between Nanbeige4-3B-Thinking and other large parameter-scale models on WritingBench wu2025writingbenchcomprehensivebenchmarkgenerative. Scores are taken from the official leaderboard.
| Model | Overall | Academic & Engineering | Finance & Business | Politics & Law | Literature & Arts | Education | Advertising & Marketing |
| --- | --- | --- | --- | --- | --- | --- | --- |
| GPT-5 | 83.87 | 84.46 | 83.78 | 82.21 | 85.26 | 85.64 | 82.34 |
| Qwen-235B-A22B-Thinking | 82.34 | 83.43 | 83.44 | 79.67 | 82.68 | 84.52 | 80.94 |
| Doubao-Seed-1.6-Thinking | 79.29 | 80.79 | 79.74 | 78.05 | 78.37 | 81.48 | 77.95 |
| Gemini-2.5-Pro | 79.26 | 79.59 | 78.24 | 78.69 | 80.92 | 81.30 | 77.27 |
| Nanbeige4-3B-Thinking-2511 | 79.03 | 80.77 | 82.37 | 77.41 | 76.05 | 80.81 | 76.55 |
| Deepseek-R1-0528 | 78.92 | 78.79 | 78.38 | 77.27 | 80.91 | 80.41 | 78.44 |
| Grok-4 | 74.65 | 75.44 | 73.99 | 73.08 | 74.66 | 77.48 | 74.73 |
| O4-mini | 72.90 | 76.04 | 73.19 | 71.26 | 69.87 | 75.91 | 72.66 |
| O3-mini | 68.02 | 69.52 | 68.61 | 66.92 | 66.54 | 71.48 | 65.95 |
In this report, we present a comprehensive overview of the model training pipeline—from the pre-training to the post-training process, and highlight the techniques that most substantially contribute to our model’s performance improvements. Specifically, Nanbeige4-3B is featured with the following technologies to improve model capabilities:
Pre-training with Hybrid Data Filtering and Fine-Grained WSD.
- Hybrid Data Filtering. To enable more precise filtering of high-quality data, we develop a hybrid strategy combining tagging-based scoring su2025nemotroncctransformingcommoncrawl; zhao2024decoratelmdataengineeringcorpus with retrieval-based recalling DBLP:journals/corr/abs-2401-14624, obtaining the final comprehensive training corpus consists of 23 trillion tokens high-quality data.
- Fine-Grained WSD. We introduce a Fine-Grained Warmup-Stable-Decay (FG-WSD) scheduler to maximize the utility of high-quality data. This approach employs a fine-grained and quality-progressive data curriculum, in which the stable stage is partitioned into multiple phases with progressively refined data mixtures. Evaluations demonstrate that our method yields substantial improvements over the vanilla WSD scheduler hu2024minicpmunveilingpotentialsmall; liu2024deepseek.
Post-training with Multi-Stage SFT, Distillation, and Reinforcement Learning.
- Multi-Stage SFT. We first fine-tune our base model on over 30 million cleaned samples related to math, science, and code to establish strong reasoning capabilities. Subsequently, we apply curriculum learning with increasingly diverse and more challenging instructions to ensure robust performance across a wide range of tasks. To further improve the quality of SFT responses, we introduce an innovative approach that combines deliberative learning with Chain-of-Thought (CoT) reconstruction: we first refine the Solution component to make it superior, then reconstruct a corresponding CoT that logically leads to this improved solution. This yields high-quality SFT training examples that significantly outperform those generated via rejection sampling.
- Distillation. Following SFT, we employ the Nanbeige flagship reasoning model as the teacher to distill the Nanbeige4-3B student model, with our proposed Dual Preference Distillaition (DPD) method. In particular, we innovate the loss function design: on the one hand, the student model learns to mimic the teacher’s output distribution as a policy model; on the other hand, it is simultaneously trained to distinguish between high-quality and low-quality responses.
- Reinforcement Learning. Building upon the distilled model, we conduct multi-stage RL training to further boost performance. We design a suite of reward and verification strategies tailored to different training phases—including STEM reasoning, coding, and human preference alignment—ensuring consistent and stable improvements throughout.
Building on the techniques described above, we develop a base model Nanbeige4-3B-Base, and a reasoning-enhanced model Nanbeige4-3B-Thinking. To support the community and foster open, reproducible research in LLMs, we open-source the Nanbeige4-3B model family. We hope this will empower researchers and developers to explore advanced training methodologies, accelerate innovation in reasoning-centric models, and contribute to a more open and collaborative AI ecosystem.
## 2 Pre-Training
In this section, we first describe the construction of our pre-training dataset and the corresponding training recipe, followed by an evaluation of the Nanbeige4-3B-Base model’s performance.
### 2.1 Pre-Training Data
During the construction of the pre-training corpus for Nanbeige4-3B, we extensively collect high-quality and diverse data. Our corpus encompasses a diverse range of web pages, scholarly articles, books, source code, and other materials. To strengthen the grasp of the human world, we not only extract clean text from HTML documents but also develop a highly efficient PDF text-extraction pipeline. We further augment the pre-training mix with synthetic data that target specialized competencies: these take the form of question–answer pairs, textbooks, lecture notes, and long chain-of-thought samples, which constitute 15% of the total pre-training tokens.
### 2.2 Pre-Training Recipe
In developing the training recipe for Nanbeige4-3B, we focus on two key issues:
1. How to identify high-quality data while filtering out low-quality samples.
1. How to make full use of the selected data to further improve performance.
In the subsequent parts of this section, we first introduce how data quality is assessed and annotated, and then present how these quality ratings are utilized during training to enhance model performance.
#### 2.2.1 Data Quality Identify
To retain high-quality samples while filtering out low-quality ones, we access the quality of each data entry in the corpus using two complementary strategies: (1) multi-dimensional tagging that inspects intrinsic attributes, and (2) similarity-based scoring against a curated set of high-quality seed data to assess extrinsic alignment.
Multi-dimensional tagging. We adopt a similar workflow as in prior research on pre-training data quality assessment hu2024minicpmunveilingpotentialsmall; su2025nemotroncctransformingcommoncrawl. This involves constructing a comprehensive labeling system, sampling and annotating data using a strong model, distilling the annotations to a smaller model for scalable annotation, and finally applying weighted ranking to select high-quality samples. Our labeling framework spans two key aspects: format and content. Initially, we define over 60 dimensions, including knowledge density, reasoning density, and text fluency, etc. Based on fine-grained experiments and multiple criteria such as inter-dimension similarity, we further filter these candidates and ultimately retain a final set of 20 dimensions. We annotate all pre-training data along these 20 dimensions and further perform extensive validation and optimization experiments to refine their effectiveness. Empirical analyses reveal two key findings. First, content-related labels are markedly more predictive of data quality than format-related ones. Second, a fine-grained 0–9 scoring scheme provides substantially more accurate data selection than a binary 0–1 labeling strategy.
Similarity-based scoring. We build a retrieval database containing hundreds of billions of entries, supporting efficient hybrid text-based and vector-based retrieval. With this robust retrieval infrastructure, we continuously iterate on seed data curation, as well as the retrieval methodologies. For the seed data, we prioritize samples that rank high within our quality tagging framework while ensuring data provenance from reliable and authoritative sources. This dual-criteria approach guarantees that our foundational dataset maintains both exceptional quality standards and trustworthy origins. For the retrieval methodology, we conduct extensive experiments on how to balance between similarity scores and quality assessments. We discover that applying retrieval strategies on top of our quality labeling system enables more precise identification and selection of high-quality data.
By combining multi-dimensional tagging with similarity-based scoring, we filter out tens of trillions of tokens that do not meet our criteria and retain 12.5 trillion high-quality training data. From these 12.5 tokens, we further select 6.5 trillion tokens of even higher quality for up-sampling two or more epochs, ultimately forming our final 23 trillion token training corpus.
#### 2.2.2 Data Utility Scheduler
To fully leverage the selected data for further performance improvement, we introduce a novel learning rate scheduler named Fine-Grained Warmup-Stable-Decay (FG-WSD). This scheduler increases the learning rate during an initial warm-up phase, maintains it across multiple carefully designed stable stages, and finally applies a smooth decay. The multi-stage stable phases provide greater room for exploration based on the previously obtained data scores, while preventing the undesirable coupling between data ordering and learning rate changes.
Cosine Decay vs. WSD vs. FG-WSD Pre-training learning rate schedulers generally fall into two categories: warmup-cosine-decay touvron2023llamaopenefficientfoundation; touvron2023llama2openfoundation; grattafiori2024llama3herdmodels and warmup-stable-decay (WSD) hu2024minicpmunveilingpotentialsmall; liu2024deepseek; minimax2025minimaxm1scalingtesttimecompute. Through experimental validation, we find that when the data quality during the annealing phase is sufficiently high, warmup-stable-decay significantly outperforms warmup-cosine-decay. Consequently, we adopt the WSD approach as our foundational scheduling strategy. Building upon this baseline, we further introduce an enhanced variant: FG-WSD (Fine-Grained WSD). Recognizing that the WSD schedule maintains a constant learning rate during the stable phase, we optimize the data resampling strategy accordingly. Rather than uniformly sampling high-quality data throughout the entire training process, FG-WSD divides training into multiple fine-grained stages and progressively increases the proportion of higher-quality data mixtures in later stages.
Preliminary Experiments. We verify the effectiveness of FG-WSD on a 1B-parameter model with a fixed 100B-token corpus in the decay stage. The stable phase consumes one epoch of 500B medium-quality (MQ) tokens and two epochs of 250B high-quality (HQ) tokens, for a total of 1T stable-phase tokens. Under vanilla WSD, these 1T tokens are shuffled uniformly: every sample is drawn from a static 1:1 mixture of HQ and MQ data. FG-WSD, by contrast, splits the stable phase into two contiguous stages. Stage 1 processes 750B tokens composed of one epoch of HQ data (250B tokens) plus one epoch of MQ data (500B tokens). Stage 2 then continues for an additional 250B-token containing only HQ subset.
Table 2: Performance comparison between vanilla WSD scheduler and our proposed Fine-Grained WSD scheduler. The experiment is implemented on 1B parameter scale model with 1T tokens.
| Learing Rate Scheduler | GSM8k | Cmath | BBH | MMLU | CMMLU | MMLU-Pro |
| --- | --- | --- | --- | --- | --- | --- |
| Vanallia WSD | 27.1 | 34.5 | 29.3 | 49.2 | 50.3 | 16.87 |
| Fine-Grained WSD | 34.3 | 39.5 | 31.6 | 50.6 | 51.9 | 18.64 |
Table 3: Training stages of the FG-WSD scheduler in Nanbeige4-3B-Base, with corresponding data usage and learning rates.
| Stage | Training Tokens | Learning Rate |
| --- | --- | --- |
| Warmup Stage | 0.1T | 0 $\rightarrow$ 4.5e-4 |
| Diversity-Enriched Stable Stage | 12.4T | Constant 4.5e-4 |
| High-Quality Stable Stage | 6.5T | Constant 4.5e-4 |
| Decay Stage | 4T | 4.5e-4 $\rightarrow$ 1.5e-6 |
As shown in Table 2, our FG-WSD method outperforms the vanilla WSD approach across all benchmarks For GSM8k, Cmath, and BBH, we run 3-shot evaluation. For MMLU, CMMLU, and MMLU-Pro, we run 5-shot evaluation. Notably, the improvement is more pronounced on mathematical and reasoning tasks (e.g., GSM8K cobbe2021trainingverifierssolvemath, CMATH wei2023cmathlanguagemodelpass, and BBH suzgun2022challengingbigbenchtaskschainofthought) compared to knowledge and science benchmarks (e.g., MMLU hendrycks2021measuringmassivemultitasklanguage, CMMLU li2024cmmlumeasuringmassivemultitask, and MMLU-Pro wang2024mmluprorobustchallengingmultitask). This is because our quality scoring framework prioritizes reasoning density over knowledge density during high-quality data selection.
After validating the effectiveness of FG-WSD using a 1B-parameter model trained on 1T tokens, we scale the method up to the full training corpus of Nanbeige4-3B-Base. The overall training pipeline is organized into four well-defined stages: Warmup, Diversity-Enriched Stable, High-Quality Stable, and Decay. The amount of training tokens allocated to each stage, as well as the corresponding learning-rate schedules, are summarized in Table 3. A noteworthy aspect of the final decay stage is the use of the ABF (Adjusting Base Frequency) method xiong2023effectivelongcontextscalingfoundation, which enables us to extend the model’s context length to 64K. This long-context capability allows the model to fully ingest synthetic long chain-of-thought traces, books, academic articles, and large-scale code repositories without truncation, ensuring that high-value long-form data is preserved throughout training.
### 2.3 Post-SFT Evaluation for Base Model
Beyond evaluating base-model capabilities through few-shot evaluation, post-SFT downstream performance offers a more informative indicator, as virtually all production systems ultimately rely on instruction-tuned variants. We therefore fine-tune Nanbeige4-3B-Base and the open-source Qwen3-Base series with an identical supervised fine-tuning (SFT) pipeline and compared the resulting checkpoints on six representative benchmarks: AIME 2024, AIME 2025, GPQA rein2023gpqagraduatelevelgoogleproofqa, MATH-500 lightman2023letsverifystepstep, LiveCodeBench-V5, and LiveCodeBench-V6 jain2024livecodebenchholisticcontaminationfree.
To ensure robustness, we repeat the procedure with three independent SFT datasets, including Nemotron-Post-Training-Dataset-V1 NemotronPostTrainingDatasetV1, Ring-Lite-SFT-Data ringteam2025ringlitescalablereasoningc3postabilized, and OpenThoughts-3 guha2025openthoughtsdatarecipesreasoning, aggregating 9 SFT training runs in total. For each dataset, we randomly sample 500,000 instances and train for 2 epochs. To ensure a rigorously controlled and fair comparison, we keep all training hyperparameters identical across the different base models.
Table 4: Comparison of base models on reasoning benchmarks after fine-tuning.
| SFT Dataset | Base Model | AIME 24 | AIME 25 | Math-500 | GPQA | LCB-V5 | LCB-V6 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Nemotron-Post-Training-V1 | Qwen3-4B-Base | 24.6 | 25.0 | 90.4 | 44.6 | 15.9 | 17.0 |
| Qwen3-8B-Base | 37.9 | 29.6 | 91.1 | 48.9 | 27.6 | 28.1 | |
| Nanbeige4-3B-Base | 52.9 | 40.8 | 93.4 | 53.4 | 35.9 | 34.0 | |
| Ring-Lite-SFT | Qwen3-4B-Base | 40.4 | 31.3 | 93.6 | 51.4 | 20.7 | 22.5 |
| Qwen3-8B-Base | 50.0 | 35.8 | 94.4 | 55.1 | 30.2 | 29.5 | |
| Nanbeige4-3B-Base | 56.8 | 45.3 | 95.5 | 57.7 | 33.3 | 33.2 | |
| Openthoughts3 | Qwen3-4B-Base | 52.9 | 42.1 | 93.2 | 49.6 | 27.2 | 27.5 |
| Qwen3-8B-Base | 60.4 | 47.1 | 95.0 | 55.3 | 35.2 | 34.4 | |
| Nanbeige4-3B-Base | 62.4 | 49.2 | 94.6 | 56.9 | 40.9 | 38.8 | |
As shown in Table 4, across all experimental settings, Nanbeige4-3B-Base not only delivers substantial improvements over Qwen3-4B-Base, but also consistently outperforms the much larger Qwen3-8B-Base, despite being only half its size. These results indicate that our model provides a markedly stronger initialization for developing downstream reasoning models.
## 3 Post-Training
In this section, we introduce each stage of the post-training process in detail. The overall post-training pipeline for Nanbeige4-3B-Thinking is illustrated in Figure 2. Initially, we conduct two fine-tuning stages—cold-start SFT and overall SFT—to equip the model with fundamental reasoning capabilities across a wide range of tasks. Subsequently, a knowledge-distillation phase is employed to further enhance performance. Finally, the model undergoes multi-stage RL training to achieve additional improvements in multiple domain skills.
<details>
<summary>x2.png Details</summary>
### Visual Description
\n
## Diagram: Model Training Pipeline
### Overview
The image depicts a four-stage pipeline for training a model, progressing from "Cold Start SFT" to "RL". Each stage is represented by a grey rectangle with a title and a more detailed description below. Arrows indicate the flow of the process from left to right.
### Components/Axes
The diagram consists of four sequential stages:
1. **Cold Start SFT:** "Foundational Reasoning Skills" (Math/Code/STEM)
2. **Overall SFT:** "General/Curriculum Learning" (General Conversation/Agent/Reasoning Curriculum Data)
3. **Distillation:** "Dual-Level Preference Distillation" (Large Model -> Small Model)
4. **RL:** "Multi-Stage RL With Robust Reward System" (STEM/Code/Human Preference Alignment)
The arrows connecting the stages indicate the progression of the training process.
### Detailed Analysis or Content Details
The diagram outlines a sequence of model training techniques:
* **Stage 1: Cold Start SFT** - Focuses on building foundational reasoning skills using data related to Math, Code, and STEM fields.
* **Stage 2: Overall SFT** - Expands the training to include general conversation, agent behavior, and reasoning curriculum data.
* **Stage 3: Distillation** - Employs a dual-level preference distillation technique, transferring knowledge from a large model to a smaller model.
* **Stage 4: RL** - Utilizes multi-stage reinforcement learning with a robust reward system, incorporating STEM, Code, and human preference alignment.
The text within each stage provides specific details about the training approach and the types of data used.
### Key Observations
The pipeline demonstrates a progression from foundational skills to more complex training techniques. The inclusion of "STEM/Code" in multiple stages suggests a strong emphasis on these areas. The final stage incorporates human preference alignment, indicating a focus on creating models that align with human values.
### Interpretation
This diagram illustrates a comprehensive approach to model training, starting with a strong foundation in reasoning skills and progressively incorporating more advanced techniques like distillation and reinforcement learning. The pipeline appears designed to create models that are not only capable of general conversation and agent behavior but also possess strong reasoning abilities and align with human preferences. The emphasis on STEM and code throughout the process suggests a focus on building models that excel in technical domains. The use of distillation indicates an effort to create efficient models without sacrificing performance. The overall structure suggests a deliberate and iterative approach to model development.
</details>
Figure 2: Nanbeige4-3B-Thinking Post-Training Pipeline
### 3.1 Cold Start Supervised Fine-tuning
On top of the base model, we first perform a cold-start supervised fine-tuning (SFT) stage to establish a robust foundation for reasoning. This stage focuses on high-quality reasoning data. After systematic cleaning and filtering, we collect approximately 30 million QA samples of mathematical, code, and subject-area problem-solving and reasoning. Based on this dataset, we construct a training corpus with a context length of 32K tokens, comprising approximately 50% mathematical reasoning, 30% scientific reasoning, and 20% code-related tasks. The objective of the cold-start stage is to strengthen the model’s chain-of-thought reasoning and structured response abilities wei2022chainofthought; wang2022selfconsistency, providing a solid foundation for subsequent capability expansion.
Scaling SFT Instructions. While some recent technical reports suggest that tens or hundreds of thousands of high-quality instructions are sufficient for supervised-finetuning, our experiments on Nanbeige4-3B reveal a different trend. As shown in Figure 3, when controlling for data distribution and overall quality, scaling the cold-start SFT instruction set from hundreds of thousands to tens of millions of examples continues to produce substantial improvements on challenging reasoning benchmarks such as AIME 2025 and GPQA-Diamond, without a clear early saturation point. Motivated by this empirical observation, we explicitly adopt a Scaling SFT Instructions strategy. Rather than relying on a compact instruction set, we train the model during the cold-start stage on tens of millions of carefully curated instructions. This large-scale instruction regime provides the model with a stronger initial reasoning prior and stabilizes its chain-of-thought generation policy, thereby creating a more reliable foundation for subsequent training stages.
<details>
<summary>x3.png Details</summary>
### Visual Description
## Line Chart: Performance Comparison
### Overview
This image presents a line chart comparing the performance of two entities, "GPQA-Diamond" and "AIME 2025", across a range of values from 0M to 35M on the x-axis. The y-axis represents a performance metric ranging from approximately 0.40 to 0.80. The chart displays the trend of these metrics as the x-axis value increases.
### Components/Axes
* **X-axis:** Labeled "0M" to "35M" in increments, representing a numerical scale.
* **Y-axis:** Ranges from approximately 0.40 to 0.80, representing a performance score. The scale is linear.
* **Legend:** Located in the top-left corner.
* "GPQA-Diamond" - Represented by an orange line with diamond markers.
* "AIME 2025" - Represented by a light blue line with circular markers.
* **Gridlines:** Horizontal gridlines are present to aid in reading the y-axis values.
### Detailed Analysis
**GPQA-Diamond (Orange Line):**
The GPQA-Diamond line starts at approximately 0.67 at 0M, dips to around 0.62 at 2M, then rises steadily to a peak of approximately 0.76 at 12M. It then fluctuates, reaching approximately 0.78 at 30M and ending at approximately 0.77 at 35M. The overall trend is upward, with some oscillations.
* 0M: ~0.67
* 2M: ~0.62
* 5M: ~0.65
* 8M: ~0.69
* 10M: ~0.73
* 12M: ~0.76
* 15M: ~0.74
* 18M: ~0.70
* 20M: ~0.72
* 22M: ~0.74
* 25M: ~0.75
* 28M: ~0.73
* 30M: ~0.78
* 35M: ~0.77
**AIME 2025 (Light Blue Line):**
The AIME 2025 line begins at approximately 0.42 at 0M and increases rapidly to around 0.58 at 5M. It continues to rise, reaching approximately 0.70 at 15M, then fluctuates between approximately 0.70 and 0.73, ending at approximately 0.72 at 35M. The overall trend is upward, but less pronounced than GPQA-Diamond.
* 0M: ~0.42
* 2M: ~0.48
* 5M: ~0.58
* 8M: ~0.61
* 10M: ~0.65
* 12M: ~0.68
* 15M: ~0.70
* 18M: ~0.67
* 20M: ~0.69
* 22M: ~0.71
* 25M: ~0.73
* 28M: ~0.72
* 30M: ~0.72
* 35M: ~0.72
### Key Observations
* GPQA-Diamond consistently outperforms AIME 2025 across the entire range of values.
* AIME 2025 shows a steeper initial increase in performance compared to GPQA-Diamond.
* Both lines exhibit fluctuations, suggesting the performance is not consistently linear.
* The performance gap between the two entities appears to widen as the x-axis value increases beyond 20M.
### Interpretation
The chart demonstrates a comparison of two systems or algorithms, GPQA-Diamond and AIME 2025, over a range of input values (represented by the x-axis). The y-axis likely represents a performance metric such as accuracy, efficiency, or a similar quantifiable measure.
The initial rapid improvement of AIME 2025 suggests it may be more sensitive to initial data or require a shorter "warm-up" period. However, GPQA-Diamond ultimately achieves and maintains a higher level of performance. The fluctuations in both lines could be due to variations in the input data, inherent randomness in the algorithms, or limitations in the evaluation methodology.
The widening gap between the two lines as the x-axis value increases suggests that GPQA-Diamond scales better or is more robust to larger input values. This could indicate that GPQA-Diamond is a more suitable choice for applications requiring consistent high performance across a wide range of inputs. The chart provides valuable insights into the relative strengths and weaknesses of each system, aiding in informed decision-making regarding their deployment.
</details>
Figure 3: Cold-start SFT scaling (0.5M–35M) on NBG4-3B reasoning benchmarks.
### 3.2 Overall Supervised Fine-Tuning
After the model acquires initial reasoning capabilities, we perform an Overall SFT stage ouyang2022instructgpt; wang2022selfinstruct; zhou2023lima to further enhance its general abilities and task diversity. The training corpus combines general conversation and writing data (covering everyday dialogue and multiple genres), agent-style interaction data (tool use, task decomposition, planning, and execution), harder reasoning data that targets the weak spots revealed in the cold-start stage, and code-related tasks that reinforce programming and code understanding. Using a 64K context length, we mix the data as roughly 40% mathematical and subject-specific reasoning, 30% general QA and writing, 20% agent scenarios, and 10% coding tasks, which systematically improves the model’s general dialogue ability, task execution, and adaptation to diverse application scenarios while maintaining strong reasoning performance. In the following section, we provide a detailed description of our data processing pipeline.
<details>
<summary>x4.png Details</summary>
### Visual Description
\n
## Diagram: Solution Refinement and CoT Reconstruction
### Overview
The image presents a diagram illustrating two interconnected processes: Solution Refinement and Chain-of-Thought (CoT) Reconstruction. The diagram uses a flowchart style to depict the flow of information and iterative loops within each process. The overall purpose appears to be to demonstrate a methodology for improving the quality of solutions, potentially in the context of large language models or AI systems.
### Components/Axes
The diagram is divided into two main sections, labeled "①Solution Refinement" and "② CoT Reconstruction". Each section contains several rectangular boxes representing different stages or components. Arrows indicate the direction of flow between these components. Key components include:
* **Instruction:** The starting point for Solution Refinement.
* **Teacher Generate:** Generates a draft solution based on the instruction.
* **Dynamic Evaluation Checklist:** Provides criteria for evaluating the draft.
* **Multi-Model Evaluator:** Scores and critiques the draft based on the checklist.
* **Answer Revision Model:** Rewrites the answer based on the critique.
* **High-Quality Final Solution:** The output of the refinement process.
* **Construct Input:** Combines prompt and solution for CoT reconstruction.
* **CoT Completion Model:** Completes the chain of thought.
* **Generate Summary:** Creates a summary of the CoT.
* **Generate CoT:** Generates the full chain of thought.
* **Thinking Model SFT Data:** The output of the CoT reconstruction process.
The diagram also includes labels for the flow of information: "Draft", "Criteria", "Critique", and "New Candidate". The iterative loop in Solution Refinement is explicitly labeled "Iterative Critique & Refinement (Loop)".
### Detailed Analysis or Content Details
The diagram illustrates a cyclical process for Solution Refinement. An "Instruction" initiates the process, leading to a "Teacher Generate" component producing a "Draft". This draft is then evaluated by a "Multi-Model Evaluator" using a "Dynamic Evaluation Checklist". The evaluator provides a "Critique" which is fed to an "Answer Revision Model" to produce a "New Candidate". This loop continues iteratively until a "High-Quality Final Solution" is achieved.
The CoT Reconstruction process begins with "Construct Input" (Prompt + Solution). This input is processed by a "CoT Completion Model", followed by "Generate Summary" and finally "Generate CoT", resulting in "Thinking Model SFT Data".
The two processes are connected. The "Dynamic Evaluation Checklist" feeds into the CoT Reconstruction process, suggesting that the criteria used for refining the solution also inform the construction of the chain of thought.
### Key Observations
* The Solution Refinement process is explicitly iterative, indicated by the labeled loop.
* The CoT Reconstruction process appears to be a linear flow of steps.
* The diagram emphasizes the importance of evaluation and feedback in both processes.
* The connection between the two processes suggests a synergistic relationship, where refining the solution also enhances the understanding of the reasoning behind it.
* The diagram does not contain any numerical data or specific values. It is a conceptual representation of a process.
### Interpretation
The diagram illustrates a sophisticated approach to improving the quality of AI-generated solutions. The Solution Refinement process, with its iterative critique and revision loop, suggests a commitment to rigorous evaluation and continuous improvement. The inclusion of a "Dynamic Evaluation Checklist" indicates that the evaluation criteria are not fixed but can be adapted based on the specific task or context.
The CoT Reconstruction process highlights the importance of understanding the reasoning behind a solution. By generating a chain of thought, the system can provide transparency and explainability, which are crucial for building trust and identifying potential errors.
The connection between the two processes suggests that refining the solution and understanding the reasoning behind it are mutually reinforcing activities. A well-refined solution is more likely to be based on sound reasoning, and a clear chain of thought can help to identify areas where the solution needs improvement.
The diagram is likely intended for an audience familiar with machine learning concepts, such as large language models, evaluation metrics, and chain-of-thought prompting. It represents a high-level overview of a complex system and does not delve into the technical details of each component. The diagram's focus on iterative refinement and explainability suggests a commitment to responsible AI development.
</details>
Figure 4: Deliberative generation refinement and CoT completion
Solution Refinement. To enhance the model’s overall output quality on complex tasks, we develop a unified mechanism that combines deliberative generation refinement with chain completion tencent2025hunyuanturbos. Specifically, for each instruction, the system first constructs a tailored multi-dimensional evaluation checklist by selecting appropriate criteria from a predefined pool according to the instruction’s semantics and task type, and then adopts this checklist as an explicit constraint for the current evaluation round. The checklist selectively integrates criteria such as correctness, completeness, consistency, executability, and safety, and further refines each criterion into concrete checkpoints to enable fine-grained assessment of candidate responses. In the next iteration, the system dynamically selects from a pool of teacher models the one that performs best on the current instruction and has it co-generate candidate answers with the current SFT/Thinking model. An evaluation model then conducts cross-evaluation and comparative scoring of all candidates against a predefined checklist, producing structured feedback that highlights error locations, missing steps, and optimization suggestions. This feedback drives the Thinking model through iterative generate–review–revise cycles, continuously improving the quality of the solution.
CoT Reconstruction. After multiple rounds of deliberation and rewriting, although the final solution quality is greatly improved, the original chain of thought is often disrupted or lost, making it difficult to obtain supervision signals that simultaneously provide a high-quality final answer and a stable, learnable reasoning process. To address this, we additionally train a chain-completion model. When constructing training data for the Thinking model, we feed this model with the concatenation of the instruction and the final solution after multi-round refinement. Empirically, generating a summarized first improves followability, so the model first generates a brief summary chain of thought and then produces an explicit chain of thought that is consistent with the final answer. Finally, we concatenate the completed chain of thought and the final answer as the target output and use it as the training sample for the Thinking model, thus restoring a structured and well-aligned reasoning supervision signal while preserving answer quality.
After integrating the above mechanism into the Overall SFT stage, the model’s alignment with human preferences is significantly improved: on the Arena-Hard v2 benchmark, the score improves by 16%, with no degradation in reasoning capability li2024arenahard.
Function Call Supporting. Our model provides native support for the function-call paradigm, enabling seamless tool invocation through formally defined and standardized parameter specifications. During the overall SFT stage, we proportionally increase the amount of function-call (FC) data to further strengthen this capability. To enhance the model’s proficiency in function calling, we incorporate both open-source and synthetic data. For the open-source portion, we unify data formats and reconstruct responses so that each sample strictly adheres to our function-call schema and includes an explicit reasoning path. For the synthetic portion, we deploy real-world environments and leverage strong models (e.g., Nanbeige3.5-Pro) to generate high-quality trajectories via rejection sampling. In addition, we adopt a multi-agent framework to simulate realistic user–assistant–tool interactions, creating data that span diverse scenarios and a wide range of difficulty levels yang2025toolmind.
<details>
<summary>x5.png Details</summary>
### Visual Description
\n
## Diagram: Model Training Pipeline
### Overview
This diagram illustrates a model training pipeline involving a teacher model, a student model, and a preference filter, culminating in a joint loss function. The pipeline appears to be focused on refining a student model using knowledge distillation and direct preference optimization (DPO).
### Components/Axes
The diagram consists of several interconnected blocks representing different stages of the training process. Key components include:
* **Query:** The initial input to the system.
* **Teacher Model (NBG3.5-Pro):** A pre-trained model providing guidance.
* **Student Model (NBG4-3B-SFT):** The model being trained.
* **Student Model (NBG4-3B):** The model being updated.
* **Negative Sample:** A sample representing an undesirable output.
* **Positive Sample:** A sample representing a desired output.
* **Pair-wise Preference Filter:** A component ensuring the positive sample is preferred over the negative sample.
* **Training Data:** The data used to update the student model.
* **Teacher Logits:** The output of the teacher model.
* **Student Logits:** The output of the student model.
* **Sequence-level DPO Loss:** A loss function optimizing the model based on sequence-level preferences.
* **Positive Token-level Distillation:** A distillation process focusing on positive samples.
* **Negative Token-level Distillation:** A distillation process focusing on negative samples.
* **Joint Loss:** The combined loss function.
* **Update:** An arrow indicating the direction of model updates.
### Detailed Analysis or Content Details
The diagram shows a flow of information starting with a "Query" which is fed into both the "Teacher Model (NBG3.5-Pro)" and the "Student Model (NBG4-3B-SFT)". The teacher model outputs "Teacher Logits". The student model generates both "Negative Sample" and "Positive Sample". These samples are then passed through a "Pair-wise Preference Filter" which ensures "Ensure Positive >> Negative". The output of this filter is "Training Data".
The "Training Data" and "Teacher Logits" are then used in two distillation processes and a DPO loss calculation.
* **Sequence-level DPO Loss:** This block shows a horizontal bar with "Maximize Margin" labeled above it. The bar is divided into two sections: a light gray section labeled "Positive Score" with a smiling face icon, and a dark gray section labeled "Negative Score" with a frowning face icon.
* **Positive Token-level Distillation:** This block shows two overlapping bell curves. The left curve is gray and labeled "Teacher Prob". The right curve is blue and labeled "Student Prob". The space between the curves is labeled "KL". The text "Minimize" is above the block.
* **Negative Token-level Distillation:** Similar to the positive distillation, this block shows two overlapping bell curves. The left curve is gray and labeled "Teacher Prob". The right curve is blue and labeled "Student Prob". The space between the curves is labeled "KL". The text "Minimize" is above the block.
Finally, these three loss components are combined into a "Joint Loss" represented by the equation: `L_DPO + L_pos + L_neg`. An arrow labeled "Update" points from the "Joint Loss" block back to the "Student Model (NBG4-3B)" block.
### Key Observations
The diagram highlights a multi-faceted training approach combining direct preference optimization (DPO) at the sequence level with token-level distillation for both positive and negative samples. The use of a teacher model suggests a knowledge distillation strategy. The preference filter is crucial for ensuring the training data reflects desired model behavior.
### Interpretation
This diagram depicts a sophisticated model training pipeline designed to align a student model with a teacher model and human preferences. The DPO loss encourages the student model to generate outputs preferred by humans, while the distillation losses help the student model mimic the teacher model's behavior at the token level. The preference filter ensures that the training data is consistent with the desired preference ordering. The joint loss function balances these different objectives, leading to a refined student model. The use of both sequence-level and token-level approaches suggests a focus on both overall coherence and fine-grained accuracy. The model names (NBG3.5-Pro, NBG4-3B-SFT, NBG4-3B) indicate specific versions or configurations of the models involved. The diagram doesn't provide quantitative data, but rather illustrates the *process* of training. It suggests a focus on reinforcement learning from human feedback (RLHF) or a similar preference-based learning paradigm.
</details>
Figure 5: Overview of the Dual-Level Preference Distillation (DPD) framework
### 3.3 Dual-level Preference Distillation
We propose Dual-Level Preference Distillation (DPD), a joint training framework that harmonizes token-level knowledge distillation with sequence-level preference optimization. In this framework, Direct Preference Optimization (DPO) rafailov2023dpo acts as a sequence-level decision-boundary regularizer, maximizing the margin between positive and negative responses. Simultaneously, we introduce token-level supervision from the teacher model’s probability distribution on both sample types. This dual-granularity setup improves instruction-following behavior and subjective preference alignment, and at the same time, substantially enhances the model’s complex reasoning capabilities.
For data construction, positive samples are obtained by sampling multiple responses from the teacher model Nanbeige3.5-Pro for the same instruction, followed by model-based scoring and rule-based filtering to select the highest-scoring answer. Negative samples are generated by sampling from the 3B student model under training. These candidates are then passed through automatic evaluation and rule-based checks, and only those whose quality and scores are significantly worse than the corresponding positive sample are retained.
For the optimization objective, we use a joint loss that combines token-level probability distillation with a sequence-level DPO preference loss. On positive samples, the student model is trained to match the Nanbeige3.5 Pro probability distribution at each token. On negative samples, we also apply a distillation loss, where the teacher provides a reference distribution for the incorrect responses generated by the student, reducing the probability of highly confident erroneous tokens and increasing the probability of under-estimated but reasonable alternatives. This design enhances the model’s ability to correct its own mistakes and to recognize errors. The sequence level DPO preference loss, implemented as a margin constraint, explicitly enlarges the score gap between positive and negative responses, thereby sharpening the decision boundary and improving style alignment.
Using the Nanbeige4-3B SFT model as the baseline, training with this framework yields consistent and substantial relative gains on multiple internal and public benchmarks: around an 8% improvement on challenging mathematical benchmarks such as AIME 2024 and AIME 2025 emergentmind2025aime, about a 10% gain on scientific reasoning tasks such as GPQA rein2023gpqa, and roughly a 30% improvement on BFCL V4, which emphasize tool use and problem decomposition patil2025bfcl. At the same time, Arena-style subjective preference evaluations still show an improvement of around 8% li2024arenahard. Furthermore, incorporating an RL phase on top of this distillation framework yields substantially larger gains compared to initiating RL directly from the SFT baseline.
### 3.4 Reinforcement Learning
Large-scale supervised fine-tuning and distillation provide a strong foundation for the reasoning ability of NBG4-3B. We further bootstrap both its reasoning capability and human preference alignment through reinforcement learning (RL). Instead of conducting a single-stage RL procedure on a mixed corpus spanning multiple knowledge domains yang2025qwen3, we adopt a multi-stage RL framework, where each stage targets a specific ability dimension. To fully exploit the potential of each stage, we perform on-policy data filtering with the latest model before every RL phase. For each phase, we adopt appropriate reward models or verifiers based on the features of the training data.
#### 3.4.1 On-Policy Data Filtering
During multi-stage RL training, the model’s reasoning ability improves after each stage. Due to cross-domain knowledge transfer, samples that were previously challenging may become trivial in later stages and contribute little to further improvement. To maintain a high-quality learning signal, we apply on-policy data filtering before each RL stage using the model from the previous stage.
Concretely, we use the model from the preceding stage to compute the avg@16 accuracy for every question and retain only those samples whose pass rate lies strictly between 10% and 90%. This focuses training on problems that are neither trivial nor unsolvable, thereby maximizing the effectiveness of each RL update.
#### 3.4.2 Multi-Stage Reinforcement Learning
To enhance reasoning ability across domains, we employ multi-stage RL rather than a single mixed-corpus paradigm. While mixed-corpus training can yield strong cross-domain transfer cheng2025revisitingreinforcementlearningllm, we observe that it often slows progress in specific domains: the model may spend many updates to achieve marginal gains in more challenging skills. For example, when jointly training on advanced mathematics and competitive programming data, the model tends to improve more on mathematics than on competitive coding. To address this imbalance, we organize RL into multiple stages, each focused on a single domain. This allows the model to concentrate its capacity on domain-specific skills at each stage while still benefiting from cross-domain transfer over the full training pipeline.
We adopt on-policy GRPO shao2024deepseekmathpushinglimitsmathematical for each RL stage, with several enhancements for stablized training. In particular, we remove the KL penalty term and mask the loss for truncated sequences, following the insights of DAPO yu2025dapoopensourcellmreinforcement. We organize the training into three RL stages as follows.
STEM RL with Agentic Verifier. We use STEM-focused RL as the first stage, motivated by prior findings that STEM training provides strong cross-domain transfer. Our dataset consists of question–answer pairs in mathematics and the natural sciences. For mathematical problems, we collect diverse data from open-source datasets. For science problems spanning physics, chemistry, and biology, we use proprietary competition-level collections. For problems with multiple sub-parts, we rewrite each sub-problem into a self-contained question with a complete context.
In STEM domains, reference answers and model output may express equivalent numerical results in different symbolic forms. To provide accurate training signals, we employ a tool-augmented verifier that calls Python interpreter to perform exact computation and simplification CosineVerifier. This agentic verifier enables robust judgments that go beyond string-matching rules.
Practical Coding RL with Synthetic Test Functions. This stage aims to enhance practical coding ability across multiple programming languages and task scenarios. We design a multi-agent system to synthesize problems paired with executable test functions. To ensure the correctness and completeness of the synthetic data chou2025autocodebenchlargelanguagemodels, we adopt a reverse-generation procedure: we first synthesize the solution and its corresponding test functions, and only then generate the natural-language problem description. Concretely, it first retrieves high-quality code snippets from GitHub, then refines or evolves these snippets into self-contained, verifiable solutions and produces paired public and private test functions. Finally, all candidate triples (problem, solution, test function) are validated via sandboxed execution to guarantee reliability. During RL training, these test functions are executed to provide a binary reward signal based on whether the generated solution passes all tests.
Human Preference Alignment RL with Pairwise Reward Model. In the final stage, we focus on aligning the model with human preferences on tasks such as creative writing and role-playing. Since these tasks prefer open-ended responses without fixed reference answers, many prior works rely on general-purpose language models to score the human preference alignment of candidate responses li2024crowdsourced. However, using a general language model as a reward model has two key drawbacks: (i) it often requires a lengthy chain of thought before reaching a final verdict, which is highly time-consuming; and (ii) RL training is prone to reward hacking. To address these issues, we train a pairwise reward model that can express preferences using only a few tokens, while exhibiting strong resistance to reward hacking.
During reinforcement learning (RL) training, we first sample diverse instructions that are both challenging and clearly specified. We then prompt strong baseline models (e.g., Nanbeige3.5-Pro) to generate high-quality reference responses. Each rollout produced by the policy model Nanbeige4-3B is paired with its corresponding reference response, and the pairwise reward model assigns a preference-based score that serves as the reward signal for policy optimization.
### 3.5 Post-Training Evaluation
Table 5: Comparison between Nanbeige4-3B-Thinking and Qwen series reasoning models.
| Benchmark | Qwen3-4B- 2507 | Qwen3-8B- 2504 | Qwen3-14B- 2504 | Qwen3-30A3- 2507 | Qwen3-32B 2504 | Nanbeige4-3B- 2511 |
| --- | --- | --- | --- | --- | --- | --- |
| Mathematical Reasoning | | | | | | |
| AIME2025 | 81.3 | 67.3 | 70.4 | 85.0 | 72.9 | 85.6 |
| AIME2024 | 83.3 | 76.0 | 79.3 | 89.2 | 81.4 | 90.4 |
| Scientific Reasoning | | | | | | |
| GPQA-Diamond | 67.2 | 62.0 | 64.0 | 73.4 | 68.7 | 82.2 |
| SuperGPQA | 46.7 | 39.1 | 46.8 | 56.8 | 54.1 | 53.2 |
| Tool Use & Coding | | | | | | |
| BFCL-V4 | 44.9 | 42.2 | 45.4 | 48.6 | 47.9 | 53.8 |
| Fullstack Bench | 47.1 | 51.5 | 55.7 | 54.4 | 58.2 | 48.0 |
| Human Preference Alignment | | | | | | |
| ArenaHard-V2 | 40.5 | 26.4 | 39.9 | 60.0 | 48.4 | 60.0 |
| Multi-Challenge | 41.8 | 35.8 | 36.4 | 49.4 | 39.2 | 41.2 |
We evaluate Nanbeige4-3B on a diverse suite of benchmarks to more comprehensively assess its performance. For each benchmark, we conduct multiple evaluation runs and report the average score across these repeated trials (e.g., avg@8 denotes the mean score computed over eight independent repetitions). Our evaluation encompasses the following benchmarks:
- Mathematical Reasoning: AIME 2025, AIME 2024 emergentmind2025aime (reported as avg@8)
- Scientific Reasoning: GPQA-Diamond rein2023gpqagraduatelevelgoogleproofqa, superGPQA pteam2025supergpqascalingllmevaluation (avg@3)
- Tool Use & Coding: BFCL-V4 patil2025bfcl, Fullstack-Bench bytedanceseedfoundationcodeteam2025fullstackbenchevaluatingllms (avg@3)
- Human Preference Alignment: Arena-Hard V2 li2024crowdsourced, Multi-Challenge sirdeshmukh2025multichallengerealisticmultiturnconversation (avg@3)
For all benchmarks, we use a sampling temperature of 0.6 and top-p of 0.95, and we set the maximum generation length to 64k tokens. We compare Nanbeige4-3B against a series of open-source small language models: Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen3-30B-A3B, and Qwen3-32B. We report metrics of other models with recommended hyperparameters in our environment. Table 5 presents the evaluation results.
Experimental results show that Nanbeige4-3B excels at demanding reasoning tasks despite its compact size. It sets new state-of-the-art averages on AIME 2024, AIME 2025 and GPQA-Diamond, outperforming models with up to 10× more parameters, including Qwen3-32B and Qwen3-30B-A3B. Beyond mathematics and science, the model demonstrates strong tool-use proficiency, scoring 53.8 on BFCL-V4—an absolute gain of 5.2 points over Qwen3-30B-A3B. Human-preference alignment is equally notable: Nanbeige4-3B matches the 60.0 point top score on Arena-Hard V2, yielding a 50% relative improvement over Qwen3-4B.
In addition to the benchmark results from our locally deployed evaluations, Nanbeige4-3B also demonstrates outstanding performance in external public evaluations. As shown in Table 1, Nanbeige4-3B-Thinking-2511 ranks among the top models on the WritingBench Leaderboard (November 2025), showcasing writing capabilities across diverse scenarios that are on par with much larger models.
## 4 Conclusion
In this work, we introduce Nanbeige4-3B, a compact yet highly capable 3-billion-parameter language model that redefines what small-scale models can achieve through innovation on data and training paradigm. Trained on a meticulously curated 23 trillion tokens of high-quality data and enhanced with novel post-training techniques—including chain-of-thought introspection and refinement, advanced reinforcement learning, and knowledge distillation—Nanbeige4-3B demonstrates remarkable reasoning and generation capabilities across diverse domains.
Despite its modest size, Nanbeige4-3B outperforms open-source counterparts such as Qwen3-8B and Qwen3-14B on challenging benchmarks, including AIME, SuperGPQA, Arena-Hard V2, and BFCL-V4. Notably, in the latest official WritingBench leaderboard, the Nanbeige4-3B-Thinking-2511 achieves top-tier performance, rivaling that of large-scale models like Deepseek-R1-0528. Nanbeige4-3B thus establishes a new lightweight flagship paradigm, offering high capability, efficiency, and accessibility for both research and real-world deployment.
Looking ahead, we aim to further extend the capabilities of small-scale models to even more complex challenges, such as autonomous software engineering (SWE), deep-research agents, and diverse real-world cross-scenario tool use tasks.
## Appendix A Author List
Authors are listed in alphabetical order by first name. Names marked with an asterisk (*) denote individuals who were previously affiliated with our team. Yang Song is the corresponding author and can be reached at songyang@kanzhun.com.
Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Wei Ruan, Xiaoqi Liu *, Xiaoxue Cheng, Xiyun Xu, Yang Song †, Yanzipeng Gao *, Yiming Jia, Yun Xing, Yuntao Wen, Zekai Wang, Zhenwei An, Zhicong Sun, Zongchao Chen