2601.19834

# Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models **Authors**: Jialong Wu, Xiaoying Zhang, Hongyi Yuan, Xiangcheng Zhang, Tianhao Huang, Changjing He, Chaoyi Deng, Renrui Zhang, Youbin Wu, Mingsheng Long 1]Tsinghua University 2]ByteDance Seed [*]Work done at ByteDance Seed [†]Corresponding authors (January 27, 2026) ## Abstract Humans construct internal models of the world and reason by manipulating the concepts within these models. Recent advances in artificial intelligence (AI), particularly chain-of-thought (CoT) reasoning, approximate such human cognitive abilities, where world models are believed to be embedded within large language models. Expert-level performance in formal and abstract domains such as mathematics and programming has been achieved in current systems, which rely predominantly on verbal reasoning as their primary information-processing pathway. However, they still lag far behind humans in domains like physical and spatial intelligence, which require richer representations and prior knowledge. The emergence of unified multimodal models (UMMs) capable of both verbal and visual generation has therefore sparked interest in more human-like reasoning grounded in complementary multimodal pathways, though a clear consensus on their benefits has not yet been reached. From a world-model perspective, this paper presents the first principled study of when and how visual generation benefits reasoning. Our key position is the visual superiority hypothesis: for certain tasks–particularly those grounded in the physical world–visual generation more naturally serves as world models, whereas purely verbal world models encounter bottlenecks arising from representational limitations or insufficient prior knowledge. Theoretically, we formalize internal world modeling as a core component of deliberate CoT reasoning and analyze distinctions among different forms of world models from both informativeness and knowledge aspects. Empirically, we identify and design tasks that necessitate interleaved visual-verbal CoT reasoning, constructing a new evaluation suite, VisWorld-Eval. Through controlled experiments on a state-of-the-art UMM, we show that interleaved CoT significantly outperforms purely verbal CoT on tasks that favor visual world modeling. Conversely, it offers no clear advantage for tasks that do not require explicit visual modeling. Together, these insights and findings clarify the applicability and potential of multimodal world modeling and reasoning for more powerful, human-like multimodal AI. We publicly release our evaluation suite to facilitate further research. [Project Lead]Jialong Wu at Mingsheng Long at , Xiaoying Zhang at [Project Page] https://thuml.github.io/Reasoning-Visual-World ## 1 Introduction <details> <summary>x1.png Details</summary>  ### Visual Description ## Diagram: World Models in Human Minds and Multimodal AI Reasoning ### Overview The image presents a conceptual framework for world modeling in human minds and multimodal AI systems. It is divided into three main sections: 1. **World Models in Human Minds**: Illustrates how humans construct mental models of the world, including dual representations (verbal/symbolic vs. visual/imagery knowledge). 2. **Reasoning with Verbal World Modeling in Multimodal AI**: Demonstrates AI applications like mathematical reasoning, travel planning, and everyday activity planning. 3. **Reasoning with Visual World Modeling in Multimodal AI**: Focuses on spatial reasoning using visual inputs (e.g., identifying object locations in images). ### Components/Axes #### Section 1: World Models in Human Minds - **Diagram Elements**: - A person with a thought bubble containing a simplified Earth model (labeled "Approximate") and a real Earth (labeled "Feedback"). - Text: "World Model: Mental Model of the World" and "Dual Representations of World Knowledge." - Subcomponents: - **Verbal/Symbolic Knowledge**: Equation `y = ax² + bx + c` and `F = ma` (physics). - **Visual/Imagery Knowledge**: Basketball trajectory diagram with a YouTube play button. - **Labels**: - "Approximate" (arrow from simplified Earth to real Earth). - "Feedback" (arrow from real Earth to thought bubble). - "Dislike in Daily Life" (red text under verbal knowledge). - "Prefer in Daily Life" (green text under visual knowledge). #### Section 2: Reasoning with Verbal World Modeling in Multimodal AI - **Subsections**: 1. **Mathematical Reasoning**: - Question: "If a > 1, then the sum of the real solutions of √(a - √(a + x)) = x is equal to..." - Response: Step-by-step algebraic manipulation (squaring both sides, rearranging terms). 2. **Travel Planning**: - Task: Plan a trip with a $1,700 budget. - State/Observation: "Initial Budget: $1700, Spent: $0. Flight F3573659: $474." - Action: "Plan day 1 transportation. Select Flight F3573659." - Next State: "Spent $474, leaving $1226." 3. **Everyday Activity Planning**: - Goal: Cook tomato and eggs. - State: "Eggs from liquid to partially cooked state." - Action: "Cook the eggs in the pan." - State Change: "Eggs transformed into curds." #### Section 3: Reasoning with Visual World Modeling in Multimodal AI - **Subsections**: 1. **Real-World Spatial Reasoning**: - Question: "When you took the photo in Figure 1, where was the iron refrigerator located relative to you?" - Text: "It’s not visible in that initial view, so I need to change my perspective." - Images: Three photos of a room (fireplace, kitchen, and living area). 2. **Visual Reasoning Process**: - Text: "My initial turn was..." </details> Figure 1: Overview of a world-model perspective on multimodal reasoning. (a) Humans construct mental models of the world, representing information and knowledge through two complementary channels–verbal and visual–to support reasoning, planning, and decision-making. (b) Recent advances in large language models (LLMs) and vision language models (VLMs) largely rely on verbal chain-of-thought reasoning, leveraging primarily verbal and symbolic world knowledge. (c) Unified multimodal models (UMMs) open a new paradigm by using visual generation for visual world modeling, advancing more human-like reasoning on tasks grounded in the physical world. Examples of reasoning with verbal world modeling are adapted from Guo et al. [18], Du et al. [14], Chen et al. [9], Zhang et al. [72]. Humans construct internal mental models of the external world that represent objects and concepts, along with their relationships, structures, and operational mechanisms [11, 16]. These models support reasoning and decision-making by enabling mental simulation, allowing individuals to anticipate the outcome of actions without actually taking them [40]. For example, if a glass of water is spilled on the table, people can rapidly mentally simulate the ensuing events: the water falling downward, spreading across the surface, and potentially dripping onto the floor. Such predictions lead them to quickly move valuable items away or reach for a towel. Beyond physical systems, mental models also extend to any domain where relational structures can be simulated, such as mathematics and logic [31, 32], making them fundamental to how humans understand and interact with all aspects of the world. Cross-disciplinary researchers in philosophy, psychology, cognitive science, and related fields have a long history of developing computational models of human mental models [44]. Among them, artificial intelligence (AI) shares a core ambition of building machines that reason like people. Although debates remain, recent breakthroughs, especially in large language models (LLMs) and chain-of-thought (CoT) reasoning, have made a substantial step towards approximating human reasoning grounded in mental models of the world, often referred to as world models [24, 34] in the AI literature. During chain-of-thought reasoning, LLMs explore, reflect, and backtrack within the structured solution space, guided by world knowledge acquired through large-scale pre-training. These capabilities have already driven progress in diverse domains, including programming [18], mathematics [57, 18], scientific discovery [53], clinical medicine [58], and robotics [42]. Such reasoning capabilities have also been extended to multimodal AI systems, particularly vision language models (VLMs) [28, 6, 19, 70]. These systems typically incorporate visual inputs by aligning visual representations with the embedding space of LLMs, resulting in reasoning that remains primarily constrained to a linguistic space. In contrast, human mental models operate over multiple forms of mental representations. Dual-coding theory [45] suggests that the mind processes information through two complementary codes: verbal and imagery (particularly visual) representations. These pathways can function independently but often collaborate to support reasoning. Indeed, visual imagery has been shown to have advantages over words in memory encoding and retrieval [33]; and individuals with aphantasia, who lack the ability to visualize mental imagery, exhibit worse performance on tasks such as visual search [43]. These evidence from psychology and cognitive science therefore suggest that the absence of a dedicated visual information pathway may explain why current multimodal AI systems excel in formal and abstract domains dominated by verbal world knowledge, yet continue to fall far short of human performance on tasks involving physical and spatial reasoning [49, 8], which fundamentally depend on visual world modeling. Next-generation multimodal AI systems are evolving to be built upon unified multimodal models (UMMs) [54, 63, 62, 13], which seamlessly integrate both verbal and visual generation capabilities. The newly introduced visual generation component offers the potential to explicitly realize visual world modeling, a critical element of multimodal world models in human-like reasoning that current systems largely lack. This naturally makes us ponder: Can current UMMs truly leverage their visual generation capability to enhance reasoning and thereby narrow the performance gap between multimodal AI and humans? A growing body of preliminary research [36, 77, 38, 76, 17] has begun exploring this question from different perspectives. However, the findings so far remain inconclusive. Reported empirical results are mixed, showing no consistent trends that visual generation reliably improves reasoning performance. Moreover, the evaluation tasks used in current studies are designed heuristically, lacking a principled basis for understanding when and how visual generation can meaningfully contribute to multimodal reasoning. In this paper, we present the first principled study of when and how visual generation benefits reasoning from a world-model perspective (see Figure 1), making both theoretical and empirical contributions. Theoretically, we rigorously bridge the concepts of world models and reasoning. (1) World model formulations: We formulate multimodal world models to approximate the underlying multi-observable Markov decision processes (MOMDP) of tasks, and define two fundamental capabilities of world models, namely world reconstruction and world simulation. (2) World model-based reasoning: To realize world models for reasoning, we introduce three reasoning formulations. Two rely solely on verbal CoTs through implicit or verbal world modeling, while the third interleaves verbal and visual CoTs that explicitly incorporate visual generation as a form of visual world modeling. (3) The visual superiority hypothesis: Under this framework, we analyze the distinctions among different world models, highlighting the richer informativeness and complementary prior knowledge afforded by visual world modeling. These insights motivate our central hypothesis that visual world modeling is superior for certain tasks, particularly those grounded in the physical world. Empirically, we validate these insights through a series of controlled experiments. (4) The VisWorld-Eval suite: We identify and design tasks that specifically isolate and demand each atomic world model capability, forming a new evaluation suite to facilitate future research. This suite, VisWorld-Eval, collects seven tasks spanning both synthetic and real-world domains. (5) Empirical evaluation: Experiments with a state-of-the-art UMM [13] on VisWorld-Eval reveal findings consistent with our insights and theoretical analysis. In tasks where verbal world modeling suffers from representational bottlenecks or insufficient prior knowledge, interleaved CoT delivers substantial performance improvements. By contrast, it offers no clear advantages in tasks such as mazes and Sokoban, whose simple states do not require explicit visual world modeling. We further conduct dedicated analyses, including evidence revealing emergent implicit world modeling in the maze task. We hope this work provides early evidence for the central role of multimodal world models in general-purpose AI, in which complementary verbal and visual knowledge emerge from generative modeling across modalities, with the latter being especially valuable for bringing human-like intelligence into the physical world. ## 2 Related Work World models. The field of world models is rapidly evolving, yet remains far from reaching consensus on definitions or methodologies. Although psychology and cognitive science suggest that human mental models rely on compact representations that discard irrelevant details, how to scale approaches capable of learning such abstract representations [48, 26, 34] to arbitrary domains and modalities is still unclear. Consequently, most current techniques preserve complete information of observations, either through reconstructable latent representations [24, 25] or directly at the level of raw data. Prominent examples include modern video generation world models [12, 1, 2, 64] which capture concrete pixel-level dynamics. In contrast, language inherently provides a higher level of abstraction, making it more similar to human mental representations [60, 65, 59, 71, 9]. This motivates the promise of unified multimodal models that generate both languages and visuals as a new direction for building more human-like world models. Unified multimodal models. Multmodal understanding [28, 6, 19] and visual generation [47, 50] have long developed in isolation. Recently, there has been growing interest in integrating these two capabilities into a single unified model. This can be straightforwardly achieved by forwarding the representations of vision language models to an external visual generation module [56, 46]. A more unified approach is to model both language and visual modalities within a single backbone. While language is predominantly modeled through autoregressive next-token prediction, the design space of visual modalities spans a wide spectrum, from discrete tokenization with autoregressive [62, 54, 63] or masked modeling [66, 22], to continuous tokenization with diffusion or flow-based modeling [75, 41, 13]. Among these efforts, BAGEL [13] is one of the most widely adopted open-source models achieving state-of-the-art performance. Despite substantial progress in building unified multimodal models (UMMs), existing evaluations still primarily assess their understanding and generation capabilities separately. One widely recognized advantage of UMMs lies in leveraging reasoning abilities of handling complex instructions to enhance visual generation or editing [74, 21]. Yet when and how visual generation, in turn, enhances reasoning remains insufficiently explored, lacking solid empirical evidence and community consensus. Benchmarking visual generation for reasoning. This paper contributes to a growing line of research on visual generation for reasoning. RealUnify [52] and Uni-MMMU [77] design tasks in which generation is expected to enhance reasoning, but report mixed results without revealing clear trends regarding the benefits of visual generation. ROVER [38] reveals fundamental limitations of current models in generating meaningful visual reasoning steps, often resulting in minimal or even negative gains in final accuracy. In contrast, MIRA [76] conducts a sanity test by providing manually annotated visual cues, thereby bypassing the evaluation of visual world modeling capability. While the aforementioned works evaluate zero-shot performance, ThinkMorph [17] fine-tunes UMMs to reveal emergent reasoning behaviors but restricts each CoT to a single intermediate image, thereby not fully exploiting the potential of interleaved CoT. Our work distinguishes itself through a world-model perspective that enables a principled investigation, allowing us to both demonstrate and systematically explain when visual generation yields positive gains and when it does not. ## 3 A World Model Perspective on Multimodal Reasoning Inspired by the aforementioned connections between human cognition and artificial intelligence, we formalize our world-model perspective on multimodal reasoning (see Figure 2) in this section. ### 3.1 Formulation: Multiple Observations of the World Without loss of generality, the world of a specific task can be formulated as a multi-observable Markov decision process (MOMDP) $\mathcal{M}=(\mathcal{S},\mathcal{A},p,\Phi,\mathcal{O}_{\phi},e_{\phi})$ , where $\mathcal{S}$ denotes the state space, $\mathcal{A}$ the action space, $p$ the transition function, $\Phi$ the parameter space of observation functions, $\mathcal{O}_{\phi}$ the observation space, and $e_{\phi}$ the observation function. Each $s\in\mathcal{S}$ represents the underlying state of the world, which is typically hidden and not directly observable. Instead, it can be perceived through different instantiations of observations (hereafter also referred to as views) [27], given by $o=e_{\phi}(s)\in\mathcal{O}_{\phi}$ , parameterized by $\phi\in\Phi$ . As illustrated in Figure 2 a, such views can span multiple modalities—for example, visual observations corresponding to different camera poses, or verbal descriptions expressed with different emphases or styles. When an action $a\in\mathcal{A}$ is applied to the current state, the world transits according to the dynamics $s^{\prime}\sim p(s^{\prime}|s,a)$ and yields new observations. ### 3.2 Atomic Capabilities of World Models <details> <summary>x2.png Details</summary>  ### Visual Description ## Diagram: Multi-Observational World Modeling and Decision-Making Framework ### Overview The image presents a technical framework for multi-observational world modeling, decision-making processes, and chain-of-thought reasoning. It combines visual-spatial reasoning with formal computational models, using cube-stack examples to illustrate concepts. ### Components/Axes 1. **Top Section: Multi-Observable Markov Decision Process** - **Visual Elements**: - Cube stack with labeled positions: (0,0,0), (1,0,0), (0,1,0), (0,0,1) - L-shaped front view and inverted L-shaped right view - **Textual Elements**: - "Verbal Observations" (textual descriptions of cube positions) - "Visual Observations" (diagrammatic representations) - "Multi-Observable Markov Decision Process" (formal model with state transitions) - **Key Symbols**: - State (s) → Action (a) → Next State (s') - Observations (oφ₁, oφ₂, oφ₃) and modified observations (o'φ₁, o'φ₂, o'φ₃) 2. **Middle Section: Atomic Capabilities of World Models** - **Left Subsection: World Reconstruction** - Input: Top view, Front view, Right view - Output: Coordinate representations (e.g., (0,0,0), (1,0,0)) - **Right Subsection: World Simulation** - Input: Coordinate representations - Output: Modified cube configurations - **Central Element**: "World Model" (black box processing inputs to outputs) 3. **Bottom Section: World Model-Based Chain-of-Thought Formulations** - **Visual Flow**: - Robot interface with cube stack - Three-step process: Reconstruction → Simulation → Iterative Refinement - **Key Text**: - "Given the three views... how can we modify the stack to match the desired back view?" - "Reconstruct the full structure" → "Try put a new cube" → "Wait, retry another choice" ### Detailed Analysis 1. **Multi-Observable Markov Decision Process** - States represent cube configurations with positional coordinates - Actions modify cube positions (e.g., adding/removing cubes) - Observations (oφ) and modified observations (o'φ) show different perspective representations 2. **World Model Architecture** - Takes three orthogonal views (top/front/right) as input - Outputs 3D coordinate representations of cube positions - Processes spatial relationships through formal coordinate systems 3. **Chain-of-Thought Workflow** - Starts with visual observations (top/front/right views) - Uses world model to reconstruct 3D structure - Simulates cube additions/removals - Iterates through multiple attempts to achieve target configuration ### Key Observations 1. **Spatial Reasoning Integration** - Combines 2D visual observations with 3D coordinate representations - Uses formal coordinate systems (x,y,z) for spatial reasoning 2. **Iterative Problem Solving** - Demonstrates trial-and-error process in cube manipulation - Shows feedback loop between simulation and observation 3. **Formal Model Components** - Markov decision process framework for sequential decision making - World model as central processing unit for spatial transformations ### Interpretation This framework demonstrates how AI systems can integrate multiple sensory inputs (visual observations) with formal spatial reasoning (coordinate systems) to perform complex tasks. The cube-stack example illustrates: 1. **Perception**: Converting 2D views into 3D spatial understanding 2. **Reasoning**: Using world models to predict outcomes of actions 3. **Action**: Iteratively modifying the environment based on simulated outcomes The Markov decision process component suggests a probabilistic approach to decision-making under uncertainty, while the chain-of-thought formulation emphasizes the importance of iterative refinement in complex problem-solving tasks. The system appears designed to handle tasks requiring both spatial reasoning and sequential decision-making capabilities. </details> Figure 2: Theoretical formulation of the world model perspective on multimodal reasoning. (a) Observations of the same underlying world state can span multiple modalities, including verbal and visual observations, each reflecting different views or emphases. (b) Two atomic capabilities of world models are defined: world reconstruction, which infers complete structure from partial observations and enables novel view synthesis, and world simulation, which models dynamics to predict future observations. (c) Chain-of-thought reasoning includes internal world modeling, by explicitly maintaining an evolving sequence of observations, generated through either of the atomic world model capabilities. A world model, analogous to human mental models, is then expected to support two fundamental capabilities [34], illustrated in Figure 2 b. The first is called world reconstruction. Humans are remarkably skilled at mentally reconstructing the structure of an environment from only a few partial observations [71], grounded in their prior knowledge of the world. Such mental reconstruction allows them to imagine novel views of the same underlying state, supporting skills such as mental rotation. Formally, the perception component of a world model encodes $n$ observations from limited views into an internal representation: $\hat{s}=\operatorname{enc}(o_{\phi_{1}},\dots,o_{\phi_{n}})\approx s$ . This representation approximates the true state We set aside the debate between compact and comprehensive representations. By treating abstract (e.g., sketches) and high-fidelity observations as different view specifications, this formulation allows the internal representation to flexibly adjust to the level of detail required by the desired views., and can then be decoded to generate an unseen observation: $\hat{o}_{\phi_{n+1}}=\mathrm{dec}(\hat{s},\phi_{n+1})\approx e_{\phi_{n+1}}(s)$ , providing an internal "experience" of navigating the world. In modern generative models, including UMMs, since their latent representations are not explicitly defined, the world reconstruction capability can be realized through end-to-end novel view generation: $$ \displaystyle p_{\theta}(o_{\phi_{n+1}}\mid o_{\phi_{1}},\dots,o_{\phi_{n}}), \tag{1} $$ which implicitly learns the internal representations required to synthesize the new view. The second capability is world simulation. Humans can mentally simulate how the world evolves into the future, supporting reasoning and decision-making, either purely in their minds or with external aids such as a scratchpad. Formally, this corresponds to the prediction component of a world model, which predicts the transition of the current state and action: $\hat{s}^{\prime}\sim\operatorname{pred}(\hat{s},a)$ , providing an internal "experience" of interacting with the world. Similarly, for modern generative models, this capability is more typically realized through predictions of future observations: $$ \displaystyle p_{\theta}(o_{t+1}\mid o_{\leq t},a_{\leq t}). \tag{2} $$ In our new evaluation suite, we deliberately curate tasks that specifically demand each capability, allowing us to independently validate its contribution to multimodal reasoning (see Section 4.1). ### 3.3 Deliberate Reasoning with World Modeling Across Modalities We then formalize how world-modeling capabilities within multimodal models contribute to reasoning. Given a question $Q$ and input images $I$ , the chain-of-thought reasoning process of a multimodal AI system can be expressed as a sequence of intermediate steps (or thoughts) $R=\tau_{1},\tau_{2},\dots,\tau_{H}$ , followed by the answer $A$ . Although this general formulation treats each reasoning step $\tau_{i}$ as an unconstrained, free-form operation, our world model perspective suggests that humans reason by prediction and planning, and each step inherently manipulates the underlying world observations of the problem [59, 10, 72]. We therefore refine the reasoning formulation as $\tau_{i}=(r_{i},o_{i})$ to explicitly incorporate an evolving sequence of observations: $$ \displaystyle R=\left(r_{1},o_{1}\right),\left(r_{2},o_{2}\right),\dots,\left(r_{H},o_{H}\right), \tag{3} $$ where $r_{i}$ We use $i$ to index reasoning steps in order to distinguish them from the true time step $t$ of the underlying MOMDP. The twos are not generally aligned, as we may include branching and backtracking in the reasoning. denotes a logical reasoning step based on the accumulated context, typically expressed in text, and $o_{i}$ denotes the observation generated at that step. Specifically, the input images serve as the initial observation $o_{0}=I$ , and subsequent observations are generated from previous reasoning and observations, by invoking atomic world modeling capabilities: world reconstruction (Eq. (1)) and world simulation (Eq. (2)), where reasoning steps imply actions $a$ and view transformations $\phi$ , as illustrated in Figure 2 c. This formulation is modality-agnostic, allowing observations—and thus world modeling—to arise across various modalities. We focus specifically on verbal and visual observations, motivated by dual-coding theory in human cognition and by the fact that UMMs are equipped to generate both. This yields several concrete CoT instantiations. Specifically, verbal world modeling produces purely verbal CoTs, with $o_{i}$ as verbal descriptions, whereas visual world modeling produces verbal-visual interleaved CoTs, with $o_{i}$ as generated images. In addition, prior work has discovered that language models can implicitly learn world models with emergent internal representations of board-game states without explicit supervision [37]. Motivated by this, we also consider implicit world modeling, in which no explicit observation is generated ( $o_{i}=\emptyset$ ) In practice, strictly distinguishing implicit from verbal world modeling can be difficult, because there are often partial descriptions of the current state in the reasoning part $r_{i}$ . In this work, we treat verbal world modeling as explicitly expressing world states or observations in text, such as coordinates or symbolic matrices.. ### 3.4 The Visual Superiority Hypothesis Contemporary LLMs and VLMs have achieved impressive performance in structured and abstract domains, such as mathematics and programming, largely driven by large-scale language-centric pre-training and verbal chain-of-thought post-training. Although these models have accumulated extensive verbal and symbolic knowledge, their understanding of the visual world remains limited when trained under purely verbal supervision. As a result, they continue to struggle with tasks grounded in basic physical and spatial intuition that even young children naturally master [49, 8]. Visual world modeling is therefore essential for endowing multimodal AI with complementary forms of information and knowledge. (1) In terms of informativeness, while verbal and symbolic representations capture high-level semantic abstractions, they often suffer from ambiguity and representational bottlenecks. In contrast, visual observations are more concrete and information-rich, directly encoding physical properties such as motion and spatial relationships. This provides precise, fine-grained grounding for reasoning about the complex real world, particularly in spatial and physical tasks. (2) In terms of prior knowledge, visual world knowledge is inherently complementary to symbolic knowledge. Humans and animals acquire much of this knowledge (e.g., physical interactions and spatial transformations) through perception, largely independent of language. Consequently, humans naturally represent and communicate such knowledge visually—for example, by sketching an approximate parabolic trajectory without performing explicit calculations. This suggests that different aspects of world knowledge are concentrated in different data modalities, and learning from large-scale generative modeling of visual data can thereby expand the effective knowledge landscape available for multimodal reasoning. We next formalize and justify these insights through theoretical analysis, with formal statements and proofs provided in Appendix 7. Informativeness. For notational convenience, we denote the question $Q$ as $r_{0}$ , the input images as $o_{0}$ , and the final answer as $r_{H+1}$ . Prefixes of a CoT are defined as $R_{i}=(r_{0},o_{0},r_{1},o_{1},\dots,r_{i-1},o_{i-1}),\tilde{R}_{i}=(r_{0},o_{0},r_{1},o_{1},\dots,r_{i-1},o_{i-1},r_{i})$ . We use $\mathbb{H}(\cdot)$ and $\mathbb{I}(\cdot;\cdot)$ to denote Shannon entropy and mutual information, respectively. We first establish that the end-to-end answer error admits an upper bound that naturally decomposes into reasoning and world-modeling errors. **Theorem 1** *Let $p$ denote the distribution over optimal chain-of-thoughts and answers, and let $p_{\theta}$ be a learned reasoning model. Then the following inequality holds: $$ \displaystyle\operatorname{KL}(p(A\mid Q,I)\mid\mid p_{\theta}(A\mid Q,I)) \displaystyle\leq\operatorname{KL}(p(R,A\mid Q,I)\mid\mid p_{\theta}(R,A\mid Q,I)) \displaystyle= \displaystyle\sum_{i=1}^{H+1}\underbrace{\mathbb{E}_{p}\left[\operatorname{KL}(p(r_{i}|R_{i})\mid\mid p_{\theta}(r_{i}|R_{i}))\right]}_{\textnormal{reasoning errors}}+\sum_{i=1}^{H}\underbrace{\mathbb{E}_{p}\left[\operatorname{KL}(p(o_{i}|\tilde{R}_{i})\mid\mid p_{\theta}(o_{i}|\tilde{R}_{i}))\right]}_{\textnormal{world-modeling errors}}. \tag{4} $$* This decomposition reveals a fundamental trade-off between the informativeness of world models for reasoning and the fidelity of the world model itself. In the case of implicit world modeling, where $o_{i}=\emptyset$ , we get rid of the world-modeling error. However, this typically comes at the cost of increased uncertainty and learning difficulty in reasoning, as all state transitions must be implicitly encoded. Empirically, world models that explicitly track the task states, serving as verbal or visual sketchpads, are generally beneficial for reasoning. We dive into the reasoning component of Eq. (4) to elucidate the factors underlying these benefits. **Theorem 2** *Let $s_{i}$ denote the latent states associated with the observations $o_{i}$ . Under appropriate assumptions, the reduction in reasoning uncertainty achieved by explicit world modeling satisfies the following properties: 1. Reasoning uncertainty does not increase: $\mathbb{H}(r_{i}|o_{0},r_{0:i-1})-\mathbb{H}(r_{i}|R_{i})=\mathbb{I}(o_{1:i-1};r_{i}|o_{0},r_{0:i-1})\geq 0.$ 1. The reasoning uncertainty improvement is bounded by both (i) the information that observations provide about the underlying states and (ii) the information that the reasoning step requires about those states: $$ \mathbb{I}(o_{1:i-1};r_{i}|o_{0},r_{0:i-1})\leq\min\left(\mathbb{I}(o_{1:i-1};s_{1:i-1}),\mathbb{I}(r_{i};s_{0:i-1},r_{0:i-1})\right). \tag{5} $$* The uncertainty of the target distribution is closely related to sample efficiency and learning difficulty. Consequently, the upper bound on the improvement of reasoning uncertainty (Eq. (5)) highlights another trade-off in the choice of observation modality for world modeling. The first term indicates that observations should be sufficiently informative about the underlying latent states. In contrast, the second suggests that they need only preserve the task-relevant aspects of the states required to select appropriate reasoning steps. Excessively detailed observations may be unnecessary and even detrimental, increasing world modeling errors. Prior knowledge. Although visual world models are more informative, they are intrinsically more difficult to learn from scratch due to the high dimensionality and complexity of visual observations. Fortunately, modern AI systems are typically large-scale pre-trained, which endows them with strong prior knowledge and enables faster convergence and improved generalization during downstream post-training. As discussed earlier, humans tend to represent different aspects of world knowledge through different modalities. Consequently, for a given downstream task, the distribution shift between its transition distribution and that learned during large-scale Internet pre-training can vary substantially across modalities. The generalization bound in Theorem 6 of Appendix 7.2 suggests that this modality-dependent distribution shift is closely related to the post-training sample efficiency of the corresponding world model. This highlights the importance of acquiring broad prior knowledge across modalities during pre-training, and of leveraging the proper modality whose priors are best aligned with the downstream task. Drawing on the above analysis, we formulate our central hypothesis regarding when and how visual generation benefits reasoning, thereby helping narrow the gap between multimodal AI and human capabilities. The Visual Superiority Hypothesis: In multimodal reasoning tasks grounded in the physical world, visual generation as a world model yields representations that are more informative and knowledge-rich than those produced by verbal world models. ## 4 Experiment Settings Finally, we empirically validate the insights and theoretical analyses presented above through a series of controlled experiments. In this section, we describe the evaluation tasks and model training procedures. ### 4.1 VisWorld-Eval: Task Suite for Reasoning with Visual World Modeling While prior work has primarily designed evaluation tasks heuristically, we principledly evaluate multimodal reasoning across tasks designed to specific world model capabilities. Building on related benchmarks, we identify and curate a total of seven tasks, forming an evaluation suite tailored to assess reasoning with visual world modeling. All tasks are framed as question answering with concise, verifiable answers, and performance is measured by answer accuracy. We refer to this suite as VisWorld-Eval, and summarize it in Figure 3. World simulation. We consider the following tasks that primarily require simulating world dynamics over time: (1) Paper folding: Adapted from SpatialViz-Bench [61], this task presents a sequence of paper folds followed by hole punching, and asks for the distribution of holes after the paper is unfolded. Successfully solving this task requires simulating the unfolding process, relying on prior knowledge of symmetry and spatial transformations that is commonly grounded in visual experience. (2) Multi-hop manipulation: Build upon CLEVR [30], this task features a scene containing objects with various shapes and colors that undergo a sequence of operations, such as addition, removal, or color changes. The final question queries properties of the resulting layouts. Since target objects of operations are often specified via relative spatial relationships, this task places strong demands on state tracking and spatial understanding. (3) Ball tracking: Adapted from RBench-V [20], this task evaluates physical dynamics simulation by requiring the model to infer the trajectory of a ball undergoing ideal specular reflections within a given scene and predicting which numbered hole it will ultimately enter. In addition, we include (4) Maze [29] and (5) Sokoban [55], as these two grid-world tasks are commonly used in prior work of studying visual generation for reasoning [67, 36]. World reconstruction. We also evaluate tasks that emphasize reconstructing underlying world structure from partial observations: (6) Cube 3-view projection: Adapted from SpatialViz-Bench [61], this task provides an isometric view and two orthographic views of a connected cube stack, and asks about an unseen viewpoint. Solving the task requires reconstructing the full 3D structure and mentally rotating or projecting it into the queried view, a process closely aligned with human visual mental representations. (7) Real-world spatial reasoning: We focus on the positional relationship subset of MMSI-Bench [69]. Given multiple views of a realistic scene, these tasks ask about positional relationships among the cameras, objects, and regions. Successfully answering these questions requires constructing a coherent spatial mental model of the scene from limited viewpoints to support accurate spatial reasoning. For each task, we construct SFT data by designing different CoT patterns with implicit, verbal, or visual world modeling, enabling controlled comparative evaluations. Data construction pipeline and examples across tasks are presented in Appendix 8.1. <details> <summary>x3.png Details</summary>  ### Visual Description ## Task Suite for Reasoning with Visual World Modeling ### Overview The image presents a structured task suite divided into two sections: **World Simulation** and **World Reconstruction**. Each section contains multiple tasks with visual diagrams, questions, and answers. The tasks evaluate reasoning skills such as spatial manipulation, problem-solving, and perspective analysis. --- ### Components/Axes #### World Simulation 1. **Paper Folding** - **Question**: "How many cutouts are there in the unfolded paper?" - **Answer**: A: 15 - **Visual**: Diagram of folded paper with creases and cutouts. 2. **Multi-Hop Manipulation** - **Question**: "Starting with the initial arrangement, perform the following: 1. Place a red cylinder to the left of the black cylinder. 2. Swap the colors of the orange cylinder and the black cylinder. After these operations, what is to the left of the orange cylinder?" - **Answer**: A: black sphere, B: white sphere, C: yellow cylinder, D: red cylinder. - **Visual**: Diagram of colored cylinders with positional instructions. 3. **Ball Tracking** - **Question**: "Given a red-point-mass ball that moves at constant speed, reflects perfectly off solid walls, and follows the initial direction indicated by a green arrow, determine which numbered hole at the top it will enter first." - **Answer**: A: 1 - **Visual**: Pool table with a red ball and green directional arrow. 4. **Maze** - **Question**: "Navigate the maze from the red dot to the blue X." - **Answer**: A: (4, 5), (5, 5), (5, 4), ... - **Visual**: Grid-based maze with red start and blue goal. 5. **Sokoban** - **Question**: "Guide the player to push the box onto the goal position." - **Answer**: A: Down, Right, Down, ... - **Visual**: Sokoban grid with boxes, goals, and player. #### World Reconstruction 1. **Cube 3-View Projection** - **Question**: "How many cubes in dark violet can possibly be seen from the back view?" - **Answer**: A: 0, B: 2, C: 3, D: 9 - **Visual**: Front-right, right, and top views of a cube structure. 2. **Real-World Spatial Reasoning** - **Question**: "Which direction is the black door relative to me when I am taking Image 2?" - **Answer**: A: Behind, B: Left, C: Front, D: Right - **Visual**: Two interior room images showing door positions. --- ### Detailed Analysis - **World Simulation Tasks**: - **Paper Folding**: Focuses on spatial reasoning and geometric transformations. - **Multi-Hop Manipulation**: Tests sequential action planning and color/symbol tracking. - **Ball Tracking**: Evaluates physics-based trajectory prediction. - **Maze**: Requires pathfinding and coordinate-based navigation. - **Sokoban**: Combines spatial reasoning with sequential movement constraints. - **World Reconstruction Tasks**: - **Cube 3-View Projection**: Assesses 3D visualization from 2D projections. - **Real-World Spatial Reasoning**: Tests egocentric perspective and environmental navigation. --- ### Key Observations 1. **Structured Format**: Each task includes a question, answer options, and a visual diagram. 2. **Task Diversity**: Tasks span abstract (e.g., paper folding) and real-world scenarios (e.g., maze navigation). 3. **Answer Specificity**: Answers are labeled with letters (A-D) and include numerical or directional responses. 4. **Visual Consistency**: Diagrams align with task descriptions (e.g., maze grid matches coordinate answers). --- ### Interpretation This task suite evaluates **visual world modeling** by requiring subjects to: - Manipulate objects in simulated environments (e.g., cylinder swaps). - Predict outcomes based on physical constraints (e.g., ball trajectories). - Translate 2D representations into 3D understanding (e.g., cube projections). - Navigate real-world spaces using egocentric perspectives (e.g., door direction). The inclusion of both abstract and concrete tasks suggests a focus on **generalizable reasoning skills** applicable to robotics, AI, and human cognition studies. The structured format implies a benchmarking framework for comparing performance across tasks. </details> Figure 3: The VisWorld-Eval suite for assessing multimodal reasoning with visual world modeling. VisWorld-Eval comprises seven tasks spanning both synthetic and real-world domains, each designed to isolate and demand specific atomic world-model capabilities. Table 1: Zero-shot evaluation of advanced VLMs on VisWorld-Eval. We report the average accuracy over five tasks (excluding Maze and Sokoban) and over all seven tasks. | Models | Paper Folding | Multi-Hop Manip. | Ball Tracking | Cube 3-View | MMSI (Pos. Rel.) | Maze | Sokoban | Overall (5 tasks) | Overall (7 tasks) | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Proprietary Models | | | | | | | | | | | Gemini 3 Flash | 25.6 | 75.4 | 55.3 | 52.7 | 41.3 | 73.9 | 99.3 | 50.0 | 60.5 | | Gemini 3 Pro | 27.0 | 74.5 | 44.7 | 53.3 | 49.6 | 33.5 | 90.2 | 49.8 | 53.2 | | Seed 1.8 | 10.6 | 75.2 | 24.4 | 42.5 | 38.8 | 83.9 | 68.3 | 38.3 | 49.1 | | GPT 5.1 | 6.4 | 73.9 | 34.8 | 44.5 | 44.8 | 0.6 | 62.8 | 40.8 | 38.2 | | o3 | 13.5 | 68.1 | 24.7 | 37.7 | 44.4 | 0.0 | 36.0 | 37.6 | 32.0 | | Open-Source Models | | | | | | | | | | | Qwen3-VL-8B-Thinking [5] | 11.0 | 49.3 | 17.8 | 21.2 | 27.7 | 0.0 | 5.8 | 25.4 | 18.9 | | BAGEL-7B-MoT [13] | 11.2 | 31.6 | 19.4 | 26.8 | 27.2 | 0.0 | 0.2 | 23.2 | 16.6 | Evaluation of advanced VLMs. Table 1 reports the zero-shot performance of advanced VLMs on VisWorld-Eval. Overall, these models perform suboptimally, highlighting limitations of current multimodal AI systems. Among them, Gemini 3 Flash and Gemini 3 Pro remarkably outperform the other models; however, their performance remains far from satisfactory on challenging tasks like paper folding, ball tracking, cube 3-view projection, and real-world spatial reasoning. ### 4.2 Unified Multimodal Model Training and Evaluation Evaluation protocol. To investigate the benefits of visual generation in multimodal reasoning, we evaluate post-trained UMMs, rather than the zero-shot performance of base models. To the best of our knowledge, no open-source model has been natively optimized for interleaved verbal-visual generation for reasoning. Even commercial closed-source models currently exhibit fundamental limitations in generating visual intermediate reasoning steps [38, 76]. Focusing on post-trained models, therefore, provides a more meaningful estimate of the upper bound for multimodal reasoning performance, while reducing confounding effects arising from insufficient pre-training due to limited interleaved data availability or quality. Model training. We adopt BAGEL [13], a state-of-the-art open-source unified multimodal model, as our base model. Most experiments are conducted by supervised fine-tuning (SFT) on task-specific datasets, where verbal and visual generation in both chain-of-thought reasoning and final answers are optimized using cross-entropy and flow-matching loss. Specifically, the loss for reasoning with visual world modeling is as follows: $$ \mathcal{L}_{\theta}(Q,I,R,A)=-\sum_{i=1}^{H+1}\sum_{j=1}^{|r_{i}|}\log p_{\theta}\left(r_{i,j}\mid r_{i,<j},R_{i}\right)+\sum_{i=1}^{H}\mathbb{E}_{t,\epsilon}\left\|v_{\theta}(o_{i}^{t},t\mid\tilde{R}_{i})-(\epsilon-o_{i})\right\|_{2}^{2}, \tag{6} $$ where $o_{i}^{t}=to_{i}+(1-t)\epsilon$ are noisy observations. We emphasize that in our formulation, $r_{i}$ refers to a verbal reasoning step, instead of a reward. We also perform reinforcement learning from verifiable rewards (RLVR) following SFT. During RL, only the verbal generation component is optimized by GRPO [18], while visual generation is regularized via the KL-divergence with respect to the SFT-trained reference model: $$ \displaystyle\mathcal{J}_{\theta}(Q,I)=\mathbb{E}_{o,r\sim p_{\theta_{\text{old}}}}\Bigg[ \displaystyle\sum_{i=1}^{H+1}\sum_{j=1}^{|r_{i}|}\Bigg(\min\Big(\frac{p_{\theta}\left(r_{i,j}\mid r_{i,<j},R_{i}\right)}{p_{\theta_{\text{old}}}\left(r_{i,j}\mid r_{i,<j},R_{i}\right)}{A},\ \text{clip}\Big(\frac{p_{\theta}\left(r_{i,j}\mid r_{i,<j},R_{i}\right)}{p_{\theta_{\text{old}}}\left(r_{i,j}\mid r_{i,<j},R_{i}\right)},1-\varepsilon,1+\varepsilon\Big){A}\Big)\Bigg) \displaystyle-\sum_{i=1}^{H}\mathbb{E}_{t,\epsilon}\left\|v_{\theta}(o_{i}^{t},t\mid\tilde{R}_{i})-v_{\theta_{\text{ref}}}(o_{i}^{t},t\mid\tilde{R}_{i})\right\|_{2}^{2}\Bigg]. \tag{7} $$ Full implementation details and hyperparameters are provided in Appendix 8.2. ## 5 Experimental Results In this section, we demonstrate that visual world modeling boosts multimodal reasoning through two atomic capabilities: world simulation (Section 5.1) and world reconstruction (Section 5.2). We also identify tasks in which it is unhelpful (Section 5.3), where implicit or verbal world modeling is sufficient. We conduct analysis in detail. Interestingly, we reveal emergent internal representations in UMMs that support implicit world modeling on simple maze tasks. ### 5.1 Visual World Simulation Boosts Multimodal Reasoning Main results. Figure 4 summarizes the performance of SFT-trained UMMs under different chain-of-thought formulations across all tasks. We observe that interleaved CoT with visual world modeling significantly outperforms its purely verbal counterparts on three world simulation tasks: paper folding, multi-hop manipulation, and ball tracking. These gains are attributed to both the richer expressiveness and stronger prior knowledge afforded by the visual modality. In particular, it is difficult for models to precisely ground object coordinates and perform arithmetic operations without external tools in tasks such as multi-hop manipulation and ball tracking, with the latter being especially challenging. Thus, verbal world modeling is inappropriate and omitted in these tasks. This exacerbates ambiguity and hallucinations in purely verbal reasoning. Similarly, in paper folding, although models can track the states of holes, it remains difficult to completely depict the paper contour during unfolding. Moreover, as showcased in Figure 9 and 16, the spatial transformation involved in paper unfolding critically relies on an understanding of geometric symmetry, which can be more naturally learned from visual data like images and videos. Sample efficiency. To further demonstrate the stronger prior knowledge embedded in the visual modality, we experiment comparing the sample efficiency of verbal and visual world modeling on the paper folding task. As shown in Figure 6, reasoning with visual world modeling exhibits substantially higher sample efficiency, achieving performance comparable to verbal world modeling while using more than $4\times$ less SFT data. ### 5.2 Visual World Reconstruction Boosts Multimodal Reasoning Main results. As shown in Figure 4, multimodal reasoning tasks that rely on world reconstruction capabilities also benefit substantially from visual world modeling. In the cube 3-view task, predicting a novel view of stacked cubes, denoted symbolic character matrices, suffers from limited prior knowledge, whereas visually rotating objects has been a rich experience during pre-training with large-scale Internet videos. For MMSI tasks, fully describing a novel view of a realistic scene using text alone is similarly ill-suited as in the previous subsection, and we also discover hallucinations in pure verbal reasoning, which lacks grounding to visual generation. We do not observe consistent improvements on other positional-relationship subtasks in MMSI-Bench, except camera-object and camera-region, which we attribute to current UMM’s limitations in both spatial understanding during verbal reasoning and generation quality in visual world modeling. Full quantitative results and qualitative examples are provided in Appendix 9. We expect these limitations to be mitigated in future work with stronger base models. <details> <summary>x4.png Details</summary>  ### Visual Description ## Bar Chart: Accuracy of Different World Modeling Approaches Across Tasks ### Overview The chart compares the accuracy of three world modeling approaches (Implicit, Verbal, Visual) across nine cognitive tasks. Each task has three grouped bars representing the three modeling types, with accuracy percentages ranging from 0-80% on the y-axis. ### Components/Axes - **X-axis (Tasks)**: - Paper Folding - Multi-Hop Manip. - Ball Tracking - Cube 3-View Proj. - MMSI (Cam.-Obj.) - MMSI (Cam.-Reg.) - Maze - Sokoban - **Y-axis (Accuracy)**: 0-80% scale - **Legend**: - Pink = Implicit World Modeling - Green = Verbal World Modeling - Blue = Visual World Modeling - **Bar Colors**: - Implicit (pink) consistently shortest bars - Verbal (green) mid-range - Visual (blue) tallest bars in most cases ### Detailed Analysis 1. **Paper Folding**: - Implicit: 21.1% - Verbal: 27.4% - Visual: 39.2% 2. **Multi-Hop Manip.**: - Implicit: 40.0% - Verbal: N/A - Visual: 66.6% 3. **Ball Tracking**: - Implicit: 40.7% - Verbal: N/A - Visual: 57.6% 4. **Cube 3-View Proj.**: - Implicit: 63.7% - Verbal: 60.2% - Visual: 76.8% 5. **MMSI (Cam.-Obj.)**: - Implicit: 46.5% - Verbal: N/A - Visual: 60.9% 6. **MMSI (Cam.-Reg.)**: - Implicit: 37.3% - Verbal: N/A - Visual: 54.4% 7. **Maze**: - Implicit: 77.0% - Verbal: 73.1% - Visual: 70.6% 8. **Sokoban**: - Implicit: 29.6% - Verbal: 36.8% - Visual: 39.3% ### Key Observations - **Visual World Modeling Dominance**: Blue bars (Visual) outperform others in 7/8 tasks, with the largest margin in Multi-Hop Manip. (66.6%). - **Implicit Underperformance**: Pink bars (Implicit) are lowest in 6/8 tasks, with the worst result in Paper Folding (21.1%). - **Maze Exception**: Implicit achieves highest accuracy (77.0%) in Maze, surpassing Verbal (73.1%) and Visual (70.6%). - **Sokoban Parity**: Verbal (36.8%) and Visual (39.3%) show minimal difference, both outperforming Implicit (29.6%). ### Interpretation The data suggests **Visual World Modeling** is generally most effective across diverse cognitive tasks, particularly in complex spatial reasoning (Multi-Hop Manip., Ball Tracking). **Implicit Modeling** struggles in most scenarios but excels in Maze navigation, potentially indicating task-specific advantages (e.g., low-level pattern recognition). The consistent gap between Implicit and other approaches highlights opportunities for improving implicit representation learning. The Maze outlier warrants investigation into whether task characteristics (e.g., rule-based navigation) favor implicit processing. Sokoban's near-parity between Verbal and Visual suggests both may leverage different cognitive mechanisms for puzzle-solving. </details> Figure 4: Performance of SFT-trained UMMs with different world model-based chain-of-thought formulations across seven tasks from VisWorld-Eval. Refer to Table 1 for zero-shot performance of advanced VLMs. Effects of task difficulties. Figure 6 analyzes performance on the cube 3-view projection task across varying sizes of input cube stacks. We observe a consistent advantage of reasoning with visual world modeling over verbal world modeling across all difficulty levels. Notably, for cube stacks of size six—out of the training distribution—visual world modeling still yields approximately a $10\$ performance improvement. World model fidelity. Modern AI models are known to exhibit hallucinations along their reasoning trajectories, even when producing correct final answers [38]. We therefore evaluate the fidelity of world modeling in the cube 3-view projection task by comparing ground-truth views with the intermediate views generated verbally or visually during reasoning. To focus on structural correctness, we compare only the shapes of the views and completely ignore color information. Even under this relaxed evaluation setting, Figure 6 shows that verbal world modeling exhibits dramatically low fidelity, with scores degrading to near zero. Notably, approximately half of the samples require predicting the opposite view of a given input view, a transformation that only involves horizontal mirroring. Visual world modeling, benefiting from stronger prior knowledge of such geometric transformations, captures these patterns effectively and achieves fidelity scores consistently exceeding $50\$ . ### 5.3 Visual World Modeling is Unhelpful for Certain Tasks Main results. (Un)surprisingly, we do not observe notable improvements on grid-world tasks, including maze and Sokoban. In the maze tasks, reasoning with implicit world modeling—without explicitly tracking coordinates—achieves the best performance with a slight advantage. These results are consistent with recent empirical findings [14]. We argue that this is also well explained by our world model perspective. In these tasks, state tracking is relatively simple, typically requiring the maintenance of only one or two two-dimensional coordinates, which can be adequately handled through verbal reasoning alone. Furthermore, in the maze task, we hypothesize that such world modeling can be implicitly encoded in the model’s hidden representations [37], which helps explain the competitive performance of verbal reasoning without explicit coordinate tracking. <details> <summary>x5.png Details</summary>  ### Visual Description ## Diagram: Neural Network Processing Text with Masked Token ### Overview The image depicts a simplified diagram of a neural network processing the text "BAGEL" with a donut icon. The network includes multiple layers (Layer 1 to Layer N), a multi-layer perceptron (MLP), and a coordinate system. A masked token is highlighted in the input text, and a bar chart is connected to the MLP. ### Components/Axes - **Neural Network Layers**: - Labeled as "Layer 1," "Layer 2," ..., "Layer N" in a vertical stack. - The input text "BAGEL" is positioned above the layers, with a donut icon (blue and teal) adjacent to it. - **MLP**: - A green box labeled "MLP" connected to "Layer 2" via a blue arrow. - **Coordinate System**: - "Coordinate: (1,3)" is written in red at the top-right corner. - A bar chart (pink bars) is linked to the MLP, with an arrow pointing from the coordinate to the chart. - **Masked Token**: - The phrase "proceed until I hit a wall, at [masked] ..." is written at the bottom, with "[masked]" in red. ### Detailed Analysis - **Neural Network Structure**: - The input "BAGEL" is processed through a sequence of layers (Layer 1 to Layer N). - The MLP is connected to "Layer 2," suggesting it may perform additional transformations or aggregations. - **Coordinate (1,3)**: - The coordinate (1,3) is positioned in the top-right corner, pointing to the bar chart. This could represent a specific position in the input or output sequence, or a reference to attention weights. - **Bar Chart**: - The bar chart (pink) is connected to the MLP, possibly visualizing attention scores, activation distributions, or other model outputs. - **Masked Token**: - The phrase "proceed until I hit a wall, at [masked] ..." indicates a point in the input sequence where a token is masked (e.g., for training purposes or to simulate uncertainty). ### Key Observations - The MLP is connected to "Layer 2," not the input layer, implying it may process intermediate representations. - The coordinate (1,3) and bar chart suggest a focus on specific positions or distributions within the model. - The masked token highlights a critical point in the input sequence, potentially for debugging or training. ### Interpretation This diagram illustrates how a neural network processes text, with the MLP acting as a secondary processing unit. The coordinate (1,3) and bar chart may indicate attention mechanisms or positional analysis. The masked token suggests the model is designed to handle incomplete or ambiguous inputs, a common technique in natural language processing. The donut icon next to "BAGEL" could symbolize the model's ability to recognize or generate circular/round shapes, though this is speculative without additional context. </details> Figure 5: Probing implicit world models, by training a set of probes, i.e., MLPs which infer the masked point coordinates during reasoning from internal representations. Demystifying implicit world modeling. To validate this hypothesis, we probe the internal representations of models, as illustrated in Figure 5. We consider the same architecture, BAGEL, with three different sets of weights: a randomly initialized model, the pre-trained model, and the model supervised fine-tuned on CoT data in the implicit world modeling format, in which special tokens mask all explicit point coordinates during the reasoning process. For each model, we extract the hidden representations of these special tokens at each layer. We then train multilayer perceptrons (MLPs) on these representations to predict the underlying true point coordinates. Figure 6 reports the prediction accuracy on a validation set. As expected, the randomly initialized model completely fails to internally track point states, achieving only random-guess accuracy on $5\times 5$ mazes. In contrast, the pre-trained model [13] already exhibits emergent representations that are predictive of maze states. Notably, we observe a non-monotonic trend across layers: prediction accuracy increases from lower layers (which capture low-level features) to middle layers, and then decreases toward the final layers, which are likely specialized for next-token prediction. Finally, supervised fine-tuning on domain-specific data, despite providing no explicit coordinate supervision, substantially enhances this internal predictability, achieving near-perfect accuracy. These in-depth results help explain our main experimental findings: as the model already possesses the capability for implicit world modeling, it does not necessarily benefit from explicit verbal world modeling, let alone more complex forms of visual world modeling. <details> <summary>x6.png Details</summary>  ### Visual Description ## Line Chart: Paper Folding Accuracy vs. Training Samples ### Overview The chart illustrates the relationship between accuracy and the number of training samples for different modeling approaches and difficulty levels in a paper folding task. Accuracy is plotted on the y-axis (0–90), and training samples are on the x-axis (0–2500). Four data series are represented: Modeling (blue line), Verbal WM (red line), Normal (gray line), and Hard (triangle markers). ### Components/Axes - **Y-axis**: Accuracy (0–90, increments of 10). - **X-axis**: Number of Training Samples (0–2500, increments of 500). - **Legend**: Located in the top-right corner. Colors and markers: - Blue line: Modeling - Red line: Verbal WM - Gray line: Normal - Triangle markers: Hard ### Detailed Analysis 1. **Modeling (Blue Line)**: - Starts at ~50 accuracy with 500 samples. - Increases steadily to ~70 accuracy at 2500 samples. - Slope: Positive, linear trend. 2. **Verbal WM (Red Line)**: - Starts at ~30 accuracy with 500 samples. - Rises to ~40 accuracy at 2500 samples. - Slope: Positive but less steep than Modeling. 3. **Normal (Gray Line)**: - Flat line at ~30 accuracy across all sample counts. - No variation observed. 4. **Hard (Triangle Markers)**: - Flat line at ~25 accuracy across all sample counts. - No variation observed. ### Key Observations - **Modeling** shows the strongest improvement with increased training samples. - **Verbal WM** demonstrates moderate improvement but lags behind Modeling. - **Normal** and **Hard** difficulty levels remain static, suggesting no dependency on training data volume. ### Interpretation The data indicates that **Modeling** benefits most from additional training samples, achieving a 20-point accuracy gain (50→70). **Verbal WM** also improves but at a slower rate (30→40), implying potential limitations in its learning mechanism. The **Normal** and **Hard** difficulty levels show no improvement, suggesting they may represent fixed constraints (e.g., inherent task difficulty) rather than variables influenced by training. This could imply that certain aspects of the paper folding task (e.g., physical constraints) are unresponsive to iterative learning, while others (e.g., pattern recognition) are highly trainable. The divergence between Modeling and Verbal WM highlights the importance of task-specific modeling strategies. </details> (a) Sample efficiency. <details> <summary>x7.png Details</summary>  ### Visual Description ## Line Chart: Cube 3-View Projection ### Overview The chart illustrates the relationship between "Cube Stack Size" (x-axis) and a "Metric" (y-axis) for two distinct data series: "Visual WM" (blue solid line) and "Verbal WM" (red dashed line). The y-axis ranges from 0 to 100, while the x-axis spans cube stack sizes from 3 to 6. The legend is positioned in the top-right corner, categorizing the data under "Modeling" and "Metric." ### Components/Axes - **Title**: "Cube 3-View Projection" - **X-axis**: "Cube Stack Size" (values: 3, 4, 5, 6) - **Y-axis**: "Metric" (values: 0, 20, 40, 60, 80, 100) - **Legend**: - **Modeling**: - "Visual WM" (blue solid line) - **Metric**: - "Accuracy" (black solid line, not visible in the chart) - "WM Fidelity" (dashed black line, not visible in the chart) - **Verbal WM** (red dashed line, labeled under "Metric" in the legend) ### Detailed Analysis - **Visual WM (Blue Solid Line)**: - Starts at approximately **90** when cube stack size is 3. - Decreases gradually to **~70** at cube stack size 6. - Trend: **Downward slope** with minimal fluctuation. - **Verbal WM (Red Dashed Line)**: - Starts at approximately **50** when cube stack size is 3. - Drops sharply to **~5** at cube stack size 6. - Trend: **Steep downward slope** with a near-linear decline. ### Key Observations 1. **Inverse Relationship**: Both "Visual WM" and "Verbal WM" metrics decrease as cube stack size increases. 2. **Divergence in Decline**: "Verbal WM" exhibits a significantly steeper decline compared to "Visual WM." 3. **Legend Discrepancy**: The legend lists "Accuracy" and "WM Fidelity" under "Metric," but these are not represented in the chart. Only "Visual WM" and "Verbal WM" are plotted. ### Interpretation The chart suggests that increasing cube stack size negatively impacts both "Visual WM" and "Verbal WM" metrics. However, "Verbal WM" is disproportionately affected, dropping from 50 to near 0 as stack size grows. This could imply that verbal working memory is more sensitive to cognitive load or complexity in this context. The legend's inclusion of "Accuracy" and "WM Fidelity" under "Metric" may indicate a broader framework, but their absence in the chart raises questions about data completeness or labeling errors. The spatial grounding of the legend (top-right) and the clear color coding (blue for "Visual WM," red for "Verbal WM") ensure accurate interpretation despite potential ambiguities in the legend's structure. </details> (b) World model fidelity. <details> <summary>x8.png Details</summary>  ### Visual Description ## Line Chart: Maze State Prediction Accuracy ### Overview The chart visualizes the state prediction accuracy of three different initialization methods across 25 layers in a maze environment. Three data series are plotted: "Random Init." (red), "BAGEL PT" (green), and "BAGEL SFT" (blue). The y-axis represents accuracy (0.2–1.0), while the x-axis represents layer indices (0–25). ### Components/Axes - **X-axis (Layer Index)**: Discrete values from 0 to 25, incrementing by 5. - **Y-axis (State Prediction Accuracy)**: Continuous scale from 0.2 to 1.0, with gridlines at 0.1 intervals. - **Legend**: Positioned at the top-right corner, with color-coded labels: - Red: Random Init. - Green: BAGEL PT - Blue: BAGEL SFT ### Detailed Analysis 1. **Random Init. (Red Line)**: - Remains flat at approximately 0.2 across all layers. - No visible variation or trend. 2. **BAGEL PT (Green Line)**: - Starts at ~0.2 (layer 0), rises gradually to ~0.55 (layer 15), then declines to ~0.4 (layer 25). - Peak accuracy observed at layer 15 (~0.55). - Slight fluctuations between layers 10–20 (~0.45–0.55). 3. **BAGEL SFT (Blue Line)**: - Begins at ~0.2 (layer 0), rises sharply to ~0.95 (layer 20), then plateaus with minor fluctuations (~0.9–0.95) until layer 25. - Steep ascent between layers 10–15 (~0.5 to 0.85). - Highest accuracy among all series. ### Key Observations - **BAGEL SFT** consistently outperforms other methods, achieving ~0.95 accuracy by layer 20. - **BAGEL PT** shows a mid-chart peak but underperforms compared to BAGEL SFT. - **Random Init.** remains static, serving as a baseline. - BAGEL PT’s decline after layer 15 suggests potential overfitting or diminishing returns. ### Interpretation The chart demonstrates that **BAGEL SFT** is the most effective initialization method for state prediction in the maze environment, with a significant accuracy advantage over BAGEL PT and Random Init. The sharp rise in BAGEL SFT’s performance (layers 10–20) indicates rapid learning or optimization, while its plateau suggests stabilization. BAGEL PT’s mid-chart peak and subsequent decline may reflect temporary improvements followed by instability or overfitting. Random Init.’s flat line underscores its ineffectiveness as a baseline. These trends highlight the importance of initialization strategies in neural network performance for sequential tasks like maze navigation. </details> (c) Implicit world modeling. Figure 6: Model analysis: (a) Performance of UMMs on the paper-folding task with varying numbers of SFT samples. Reasoning with visual world modeling achieves a $4\times$ improvement in sample efficiency. WM = world modeling. (b) Performance of UMMs on the cube 3-view projection task with increasing sizes of input cube stacks, evaluated using both answer accuracy and world-model fidelity. Visual world modeling demonstrates dramatically better fidelity of view synthesis. (c) Prediction accuracy of masked point coordinates in CoTs using representations extracted from different layers of different UMMs, revealing emergent internal world representations. PT = Pre-trained. <details> <summary>x9.png Details</summary>  ### Visual Description ## Bar Chart: Accuracy of World Modeling Approaches Across Tasks ### Overview The chart compares the accuracy of three world modeling approaches—Implicit, Verbal, and Visual—across three tasks: Paper Folding, Multi-Hop Manipulation, and Cube 3-View Projection. Each approach is evaluated using two models (BAGEL and Qwen-VL) for Implicit and Verbal, while Visual is only evaluated with BAGEL. The data highlights performance differences between models and approaches. ### Components/Axes - **X-axis**: Tasks (Paper Folding, Multi-Hop Manip., Cube 3-View Proj.) - **Y-axis**: Accuracy (0–80%) - **Legend**: - **Implicit World Modeling (BAGEL)**: Solid red - **Implicit World Modeling (Qwen-VL)**: Red with diagonal stripes - **Verbal World Modeling (BAGEL)**: Solid green - **Verbal World Modeling (Qwen-VL)**: Green with diagonal stripes - **Visual World Modeling (BAGEL)**: Solid blue ### Detailed Analysis - **Paper Folding**: - Implicit BAGEL: 21.5% (solid red) - Implicit Qwen-VL: 21.1% (red striped) - Verbal BAGEL: 28.8% (solid green) - Verbal Qwen-VL: 27.4% (green striped) - Visual BAGEL: 39.2% (solid blue) - **Multi-Hop Manip.**: - Implicit BAGEL: 37.5% (solid red) - Implicit Qwen-VL: 40.0% (red striped) - Visual BAGEL: 66.6% (solid blue) - **Cube 3-View Proj.**: - Implicit BAGEL: 60.0% (solid red) - Implicit Qwen-VL: 63.7% (red striped) - Verbal BAGEL: 58.8% (solid green) - Verbal Qwen-VL: 60.2% (green striped) - Visual BAGEL: 76.8% (solid blue) ### Key Observations 1. **Visual World Modeling (BAGEL)** consistently achieves the highest accuracy across all tasks, with the largest margin in Cube 3-View Proj. (76.8%). 2. **Implicit approaches** (BAGEL and Qwen-VL) show lower performance, with Qwen-VL slightly outperforming BAGEL in Multi-Hop Manip. (40.0% vs. 37.5%). 3. **Verbal approaches** (BAGEL and Qwen-VL) perform better than Implicit but lag behind Visual. In Cube 3-View Proj., Verbal Qwen-VL (60.2%) surpasses Verbal BAGEL (58.8%). 4. **Qwen-VL** outperforms BAGEL in Implicit and Verbal categories for Multi-Hop and Cube tasks, suggesting model-specific advantages. ### Interpretation The data demonstrates that **Visual World Modeling (BAGEL)** is the most effective approach, likely due to its ability to integrate spatial and contextual information critical for tasks like Cube 3-View Projection. The **Implicit** and **Verbal** approaches show diminishing returns, with Qwen-VL occasionally outperforming BAGEL in the same category, indicating that model architecture (Qwen-VL) may enhance performance in specific scenarios. The stark difference in Cube 3-View Proj. (76.8% vs. ~60% for others) underscores the importance of visual reasoning in complex spatial tasks. This suggests that future work should prioritize Visual World Modeling for applications requiring high accuracy in spatial reasoning. </details> Figure 7: Performance of SFT-trained VLMs compared with UMMs across three tasks. <details> <summary>x10.png Details</summary>  ### Visual Description ## Line Graphs: Comparative Accuracy of World Modeling Approaches Across Tasks ### Overview The image contains three line graphs comparing the accuracy of different world modeling approaches (BAGEL and Qwen-VL) across three tasks: **Paper Folding**, **Multi-Hop Manipulation**, and **Cube 3-View Projection**. Each graph tracks accuracy (%) over 200 steps, with distinct lines for **Implicit**, **Verbal**, and **Visual World Modeling** for both BAGEL and Qwen-VL. The graphs highlight performance trends, with BAGEL's Visual World Modeling consistently outperforming Qwen-VL's Verbal World Modeling. --- ### Components/Axes - **X-axis**: Steps (0–200), labeled "Steps". - **Y-axis**: Accuracy (%), labeled "Accuracy". - **Legends**: Positioned at the top of the image, with the following entries: - **Implicit World Modeling (BAGEL)**: Pink solid line. - **Implicit World Modeling (Qwen-VL)**: Red dashed line. - **Verbal World Modeling (BAGEL)**: Green solid line. - **Verbal World Modeling (Qwen-VL)**: Green dashed line. - **Visual World Modeling (BAGEL)**: Blue solid line. - **Graphs**: Three separate line graphs, each corresponding to a task. --- ### Detailed Analysis #### Paper Folding - **BAGEL Visual World Modeling (blue)**: Starts at ~40% accuracy, peaks at ~45% by ~50 steps, then stabilizes. - **Qwen-VL Verbal World Modeling (green dashed)**: Begins at ~25%, rises to ~35% by ~100 steps, then declines to ~30% by 200 steps. - **Qwen-VL Implicit World Modeling (red dashed)**: Remains low (~20–25%) throughout, with minor fluctuations. - **BAGEL Implicit World Modeling (pink solid)**: Starts at ~20%, peaks at ~25% by ~100 steps, then stabilizes. #### Multi-Hop Manipulation - **BAGEL Visual World Modeling (blue)**: Starts at ~60%, rises to ~70% by ~50 steps, then fluctuates between ~65–70%. - **Qwen-VL Verbal World Modeling (green dashed)**: Begins at ~40%, peaks at ~50% by ~50 steps, then declines to ~40% by 200 steps. - **Qwen-VL Implicit World Modeling (red dashed)**: Remains low (~20–25%) throughout, with minor fluctuations. - **BAGEL Implicit World Modeling (pink solid)**: Starts at ~40%, peaks at ~50% by ~50 steps, then stabilizes. #### Cube 3-View Projection - **BAGEL Visual World Modeling (blue)**: Starts at ~75%, rises to ~85% by ~50 steps, then stabilizes. - **Qwen-VL Verbal World Modeling (green dashed)**: Begins at ~60%, rises to ~70% by ~50 steps, then fluctuates between ~65–70%. - **Qwen-VL Implicit World Modeling (red dashed)**: Remains low (~60–65%) throughout, with minor fluctuations. - **BAGEL Implicit World Modeling (pink solid)**: Starts at ~60%, peaks at ~70% by ~50 steps, then stabilizes. --- ### Key Observations 1. **BAGEL's Visual World Modeling** consistently outperforms all other approaches across all tasks, achieving the highest accuracy and stability. 2. **Qwen-VL's Verbal World Modeling** shows moderate performance but lags behind BAGEL's Visual in all tasks. 3. **Implicit World Modeling** (both BAGEL and Qwen-VL) underperforms significantly, with minimal improvement over steps. 4. **Qwen-VL's Verbal World Modeling** exhibits higher variability compared to BAGEL's models, with sharper declines in later steps. --- ### Interpretation The data suggests that **BAGEL's Visual World Modeling** is the most effective approach for the tested tasks, likely due to its ability to handle visual-spatial reasoning more effectively. Qwen-VL's Verbal World Modeling, while better </details> Figure 8: Performance of RLVR-trained VLMs and UMMs with different world-model-based CoT formulations across three tasks. ### 5.4 Comparison with VLMs: Do UMMs Compromise Verbal Reasoning Capabilities? One may argue that UMMs are typically trained with a stronger emphasis on visual generation [13], which could compromise verbal reasoning capabilities, and bias comparisons in favor of visual world modeling. To address this concern, we compare with a pure VLM baseline, Qwen2.5-VL-7B-Instruct [6], which shares the same Qwen 2.5 LLM base model, with BAGEL. We fine-tune Qwen2.5-VL on the same verbal CoT datasets used in the previous subsections and evaluate it on three representative tasks: paper folding, cube 3-view projection, and multi-hop manipulation. Results. As shown in Figure 7, the SFT performance of Qwen2.5-VL with implicit and verbal world modeling is comparable to that of BAGEL, without exhibiting significant advantages. It still lags behind BAGEL in settings that leverage visual world modeling. These results indicate that our findings arise from the inherent advantages of visual world modeling rather than from compromised verbal reasoning capabilities in UMMs. ### 5.5 RL Enhances Various CoTs, Yet Does Not Close the Gap Reinforcement learning from verifiable rewards (RLVR) has been a major driver of recent progress in reasoning models equipped with verbal chain-of-thoughts, achieving strong performance across domains such as mathematics [18]. While Figure 4 shows a clear advantage of reasoning with visual world modeling after SFT, RLVR may further incentivize emergent reasoning behaviors that improve verbal CoTs. We thus conduct comparative RLVR experiments across different world model–based CoT formulations on three representative tasks. Results. Figure 8 presents the learning curves under RLVR for different models. We observe consistent improvements during RLVR for different CoT formulations. However, the performance gap persists. We also find that VLMs and UMMs generally perform similarly with verbal CoTs. These results suggest that the superiority arises from inherent advantages of the world modeling approach, rather than insufficient post-training. Notably, RL enhances reasoning with visual world modeling, even though only the verbal generation components of interleaved CoTs are directly optimized. We envision that the full potential of interleaved CoTs will be further released with the development of RL algorithms tailored for verbal-visual interleaved generation. ## 6 Discussions By bridging concepts from human cognition and artificial intelligence, we revisit the mechanisms underlying human reasoning and the central role that world models play. This provides a new perspective on the use of visual generation for reasoning in multimodal AI, highlighting its potential to serve as visual world models that complements the verbal world models embedded in LLMs, thereby enabling more human-like reasoning on scenarios grounded in the physical world. For the first time, this perspective is studied in a principled manner, through theoretical formulations that bridge world models and reasoning, as well as through empirical evaluations whose results are well explained by and strongly support the proposed insights. We hope this work helps address longstanding questions about the synergistic effects between generation and reasoning, and more broadly contributes to the development of more human-like AI that thinks and acts with multimodal world models. <details> <summary>x11.png Details</summary>  ### Visual Description ## Screenshot: Real-World Spatial Reasoning and Paper Folding Tasks ### Overview The image contains two distinct reasoning tasks: 1. **Real-World Spatial Reasoning**: Determining the direction of a potted plant relative to a camera based on overlapping images. 2. **Paper Folding**: Analyzing a folded paper with cutouts to reconstruct the original pattern. Both sections include questions, model outputs (reasoning steps), and visual aids (images/diagrams). --- ### Components/Axes #### Real-World Spatial Reasoning - **Question**: *"The pictures are taken continuously from a first-person perspective, with two pictures overlapping at the frame. At the moment of the last picture, in which direction is the potted plant relative to you? Options: A: Front right, B: Back left, C: Back right, D: Front left."* - **Model Output**: A detailed textual explanation of spatial reasoning, including: - Camera movement analysis (45-degree turns). - Identification of objects (framed pictures, lamp, teapot cabinet, curtains, chair). - Final conclusion: *"Therefore, the correct option is: A: Front right."* - **Images**: - **Image 1**: Potted plant on a table next to a window with curtains and a chair. - **Image 2**: Wall with framed pictures, teapot cabinet, and lamp. - **Image 3**: Framed picture, lamp, teapot cabinet, and chair. #### Paper Folding - **Question**: *"Analyze the image showing a folded paper with cutouts. Mentally reverse the folding process to reconstruct the final unfolded paper. How many cutouts are there in the unfolded paper?"* - **Model Output**: Step-by-step reasoning: - Unfolding process (diagonal folds, mirror images of holes). - Identification of stationary vs. moving flaps. - Final count: *"Finally we can calculate that the number of cutoffs is 2."* - **Diagram**: - Folded paper with a square and triangular cutout. - Visual representation of folding steps and hole positions. --- ### Detailed Analysis #### Real-World Spatial Reasoning 1. **Model Reasoning**: - The camera moves from the first image to the second, rotating 45 degrees left. - The potted plant shifts from the "front left" in the first image to the "front right" in the second due to the camera's rotation. - Final confirmation: The plant is in the "front right" direction. 2. **Image Details**: - **Image 1**: Plant on a table, window with curtains, chair. - **Image 2**: Wall with framed pictures, teapot cabinet, lamp. - **Image 3**: Framed picture, lamp, teapot cabinet, chair. #### Paper Folding 1. **Model Reasoning**: - The folded paper has a diagonal fold (top-left corner folded down). - The square cutout is in the stationary part of the paper (bottom-left section). - The triangular cutout is in the moving flap (top-left quadrant). - Unfolding reveals two cutouts: one in the bottom-left and one in the top-left. 2. **Diagram Details**: - Folded paper shows a square and triangular cutout. - Visual cues indicate fold lines and hole positions. --- ### Key Observations 1. **Spatial Reasoning**: - The model correctly identifies the camera's movement and object positions using relative spatial cues (e.g., "front right" vs. "front left"). - Objects like the teapot cabinet and lamp help anchor the spatial context. 2. **Paper Folding**: - The model distinguishes between stationary and moving flaps during unfolding. - The final count of 2 cutouts aligns with the diagram's visual representation. --- ### Interpretation 1. **Spatial Reasoning**: - Demonstrates the model's ability to infer object positions through camera movement and environmental context. - Highlights the importance of relative positioning (e.g., 45-degree turns) in first-person perspective tasks. 2. **Paper Folding**: - Illustrates the model's capacity to reverse-engineer folding processes by analyzing mirror images and flap dynamics. - The final answer (2 cutouts) is validated by the diagram's structure and the model's step-by-step logic. **Conclusion**: Both tasks showcase the model's proficiency in spatial reasoning and geometric problem-solving, leveraging visual cues and logical deduction to arrive at accurate conclusions. </details> Figure 9: Showcases of interleaved verbal-visual chain-of-thought reasoning, generated by post-trained UMMs, where visual generation serves as world models. <image> denotes a placeholder indicating the position of a generated image. Limitations and future work. This work primarily focuses on spatial and physical reasoning tasks, where multimodal AI systems exhibit a pronounced performance gap relative to humans. Many other tasks proposed in the related literature can also be interpreted through our world model perspective. For example, a prominent class of benchmarks involves visual jigsaw tasks [52, 17, 38, 77], in which input image patches are cropped, masked, or shuffled. Such tasks essentially probe a form of world reconstruction capability, as corrupted images and videos are commonly treated as specific views within the world model literature [3, 7, 4]. Another active area of interest lies in STEM reasoning. Recent work [51] leverages visual generation for mathematical diagram editing, such as constructing auxiliary geometric lines. This closely resembles how humans use visual sketchpads to support math understanding and reasoning, constructing visual world models of a symbolic system. However, as symbolic representations in mathematics are largely complete, and mathematical reasoning has been extensively optimized in modern LLMs, it remains unclear whether multimodal interleaved CoT can fundamentally break through the performance limit, warranting further investigation. We do not apply reinforcement learning to the visual generation components of verbal–visual interleaved CoTs [39]. Prior work has shown that world models themselves can be improved through RLVR [65]. As discussed in Section 5.5, developing RL algorithms specifically tailored to interleaved verbal–visual generation may further improve world-model fidelity during reasoning and incentivize the emergence of stronger and intriguing world-modeling capabilities. The analysis of emergent representations for implicit world modeling in Figure 6 is intriguing but preliminary. We hope this result will rekindle interest in probing approaches [37] for interpreting the latent representations learned by different models. In particular, we are interested in comparing the internal representations of VLMs and UMMs, as the latter may capture complementary aspects of world knowledge through training for multimodal generation. Artificial intelligence is increasingly being embodied in the physical world [23]. Our work, particularly the visual superiority hypothesis, suggests that learning visual world models is therefore essential for embodied intelligence. Visual world modeling enables embodied agents to better understand their environments, from imagining occluded regions to interpreting user intentions from an egocentric perspective, thereby supporting more reliable and natural everyday services. It also facilitates planning and decision-making by allowing agents to mentally simulate the precise outcomes of potential actions, leading to more effective interaction with the world. Rather than relying on loosely coupled modules [15] or performing only single-step reasoning [73], we envision a future direction in which flexible multimodal world modeling and reasoning, empowered by interleave verbal-visual generation within a unified model, form core components of physical and embodied AI. ## Acknowledgements We would like to thank Yanwei Li, Rui Yang, Ziyu Zhu, and Feng Cheng for their assistance in constructing some preliminary training and test data. We also appreciate Xinchen Zhang, Jianhua Zhu, Yifan Du, Yuezhou Ma, Xingzhuo Guo, Ningya Feng, Shangchen Miao, and many colleagues for their valuable discussion. ## References - Agarwal et al. [2025] Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025. - Alonso et al. [2024] Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos J Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in atari. In NeurIPS, 2024. - Assran et al. [2023] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In CVPR, 2023. - Assran et al. [2025] Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025. - Bai et al. [2025a] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025a. - Bai et al. [2025b] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923, 2025b. - Bardes et al. [2024] Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual representations from video. arXiv preprint arXiv:2404.08471, 2024. - Cai et al. [2025] Zhongang Cai, Yubo Wang, Qingping Sun, Ruisi Wang, Chenyang Gu, Wanqi Yin, Zhiqian Lin, Zhitao Yang, Chen Wei, Xuanke Shi, et al. Has gpt-5 achieved spatial intelligence? an empirical study. arXiv preprint arXiv:2508.13142, 2025. - Chen et al. [2025] Delong Chen, Theo Moutakanni, Willy Chung, Yejin Bang, Ziwei Ji, Allen Bolourchi, and Pascale Fung. Planning with reasoning using vision language world model. arXiv preprint arXiv:2509.02722, 2025. - Copet et al. [2025] Jade Copet, Quentin Carbonneaux, Gal Cohen, Jonas Gehring, Jacob Kahn, Jannik Kossen, Felix Kreuk, Emily McMilin, Michel Meyer, Yuxiang Wei, et al. Cwm: An open-weights llm for research on code generation with world models. arXiv preprint arXiv:2510.02387, 2025. - Craik [1967] Kenneth James Williams Craik. The nature of explanation, volume 445. CUP Archive, 1967. - DeepMind [2025] Google DeepMind. Genie 3: A new frontier for world models. 2025. - Deng et al. [2025] Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025. - Du et al. [2025] Yifan Du, Kun Zhou, Yingqian Min, Yue Ling, Wayne Xin Zhao, and Youbin Wu. Revisiting the necessity of lengthy chain-of-thought in vision-centric reasoning generalization. arXiv preprint arXiv:2511.22586, 2025. - Feng et al. [2025] Yunhai Feng, Jiaming Han, Zhuoran Yang, Xiangyu Yue, Sergey Levine, and Jianlan Luo. Reflective planning: Vision-language models for multi-stage long-horizon robotic manipulation. arXiv preprint arXiv:2502.16707, 2025. - Forrester [1971] Jay W Forrester. Counterintuitive behavior of social systems. Theory and decision, 2(2):109–140, 1971. - Gu et al. [2025] Jiawei Gu, Yunzhuo Hao, Huichen Will Wang, Linjie Li, Michael Qizhe Shieh, Yejin Choi, Ranjay Krishna, and Yu Cheng. Thinkmorph: Emergent properties in multimodal interleaved chain-of-thought reasoning. arXiv preprint arXiv:2510.27492, 2025. - Guo et al. [2025a] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 2025a. - Guo et al. [2025b] Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1.5-vl technical report. arXiv preprint arXiv:2505.07062, 2025b. - Guo et al. [2025c] Meng-Hao Guo, Xuanyu Chu, Qianrui Yang, Zhe-Han Mo, Yiqing Shen, Pei-lin Li, Xinjie Lin, Jinnian Zhang, Xin-Sheng Chen, Yi Zhang, et al. Rbench-v: A primary assessment for visual reasoning models with multi-modal outputs. arXiv preprint arXiv:2505.16770, 2025c. - Guo et al. [2025d] Ziyu Guo, Renrui Zhang, Hongyu Li, Manyuan Zhang, Xinyan Chen, Sifan Wang, Yan Feng, Peng Pei, and Pheng-Ann Heng. Thinking-while-generating: Interleaving textual reasoning throughout visual generation. arXiv preprint arXiv:2511.16671, 2025d. - Guo et al. [2025e] Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Rui Huang, Haoquan Zhang, Manyuan Zhang, Jiaming Liu, Shanghang Zhang, Peng Gao, et al. Can we generate images with cot? let’s verify and reinforce image generation step by step. In CVPR, 2025e. - Gupta et al. [2021] Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature Communications, 2021. - Ha and Schmidhuber [2018] David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2(3), 2018. - Hafner et al. [2025] Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, 2025. - Hansen et al. [2022] Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. In ICML, 2022. - Huh et al. [2024] Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. In ICML, 2024. - Hurst et al. [2024] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. - Ivanitskiy et al. [2023] Michael Igorevich Ivanitskiy, Rusheb Shah, Alex F. Spies, Tilman Räuker, Dan Valentine, Can Rager, Lucia Quirke, Chris Mathwin, Guillaume Corlouer, Cecilia Diniz Behn, and Samy Wu Fung. A configurable library for generating and manipulating maze datasets. arXiv preprint arXiv:2309.10498, 2023. - Johnson et al. [2017] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, 2017. - Johnson-Laird [1983] PN Johnson-Laird. Mental models: Towards a cognitive science of language, inference, and consciousness. Harvard University Press, 1983. - Lakoff and Núñez [2000] George Lakoff and Rafael Núñez. Where mathematics comes from, volume 6. New York: Basic Books, 2000. - Landy and Goldstone [2007] David Landy and Robert L Goldstone. How abstract is symbolic thought? Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(4):720, 2007. - LeCun [2022] Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1):1–62, 2022. - Li et al. [2025a] Ang Li, Charles Wang, Deqing Fu, Kaiyu Yue, Zikui Cai, Wang Bill Zhu, Ollie Liu, Peng Guo, Willie Neiswanger, Furong Huang, et al. Zebra-cot: A dataset for interleaved vision language reasoning. arXiv preprint arXiv:2507.16746, 2025a. - Li et al. [2025b] Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought. In ICML, 2025b. - Li et al. [2023] Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Emergent world representations: Exploring a sequence model trained on a synthetic task. In ICLR, 2023. - Liang et al. [2025] Yongyuan Liang, Wei Chow, Feng Li, Ziqiao Ma, Xiyao Wang, Jiageng Mao, Jiuhai Chen, Jiatao Gu, Yue Wang, and Furong Huang. Rover: Benchmarking reciprocal cross-modal reasoning for omnimodal generation. arXiv preprint arXiv:2511.01163, 2025. - Liu et al. [2025] Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. In NeurIPS, 2025. - Loomis et al. [1991] JM Loomis, RL Klatzky, RG Golledge, and JG CicineIli. Mental models, psychology of. Psychology, 14:56–89, 1991. - Ma et al. [2025] Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, et al. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. In CVPR, 2025. - Mon-Williams et al. [2025] Ruaridh Mon-Williams, Gen Li, Ran Long, Wenqian Du, and Christopher G Lucas. Embodied large language models enable robots to complete complex tasks in unpredictable environments. Nature Machine Intelligence, 2025. - Monzel and Reuter [2024] Merlin Monzel and Martin Reuter. Where’s wanda? the influence of visual imagery vividness on visual search speed measured by means of hidden object pictures. Attention, Perception, & Psychophysics, 86(1):22–27, 2024. - Norman [2014] Donald A Norman. Some observations on mental models. In Mental models, pages 7–14. Psychology Press, 2014. - Paivio [1990] Allan Paivio. Mental representations: A dual coding approach. Oxford university press, 1990. - Pan et al. [2025] Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al. Transfer between modalities with metaqueries. arXiv preprint arXiv:2504.06256, 2025. - Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. - Schrittwieser et al. [2020] Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 2020. - Schulze Buschoff et al. [2025] Luca M Schulze Buschoff, Elif Akata, Matthias Bethge, and Eric Schulz. Visual cognition in multimodal large language models. Nature Machine Intelligence, 2025. - Seedream et al. [2025] Team Seedream, Yunpeng Chen, Yu Gao, Lixue Gong, Meng Guo, Qiushan Guo, Zhiyao Guo, Xiaoxia Hou, Weilin Huang, Yixuan Huang, et al. Seedream 4.0: Toward next-generation multimodal image generation. arXiv preprint arXiv:2509.20427, 2025. - Shi et al. [2025a] Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, et al. Mathcanvas: Intrinsic visual chain-of-thought for multimodal mathematical reasoning. arXiv preprint arXiv:2510.14958, 2025a. - Shi et al. [2025b] Yang Shi, Yuhao Dong, Yue Ding, Yuran Wang, Xuanyu Zhu, Sheng Zhou, Wenting Liu, Haochen Tian, Rundong Wang, Huanqian Wang, et al. Realunify: Do unified models truly benefit from unification? a comprehensive benchmark. arXiv preprint arXiv:2509.24897, 2025b. - Swanson et al. [2025] Kyle Swanson, Wesley Wu, Nash L Bulaong, John E Pak, and James Zou. The virtual lab of ai agents designs new sars-cov-2 nanobodies. Nature, 2025. - Team [2024] Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024. - Tong et al. [2025a] Jingqi Tong, Jixin Tang, Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, et al. Game-rl: Synthesizing multimodal verifiable game data to boost vlms’ general reasoning. arXiv preprint arXiv:2505.13886, 2025a. - Tong et al. [2025b] Shengbang Tong, David Fan, Jiachen Li, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. In ICCV, 2025b. - Trinh et al. [2024] Trieu H Trinh, Yuhuai Wu, Quoc V Le, He He, and Thang Luong. Solving olympiad geometry without human demonstrations. Nature, 2024. - Tu et al. [2025] Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Yong Cheng, et al. Towards conversational diagnostic artificial intelligence. Nature, 2025. - Wang et al. [2025a] Kangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao, Linjie Li, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, et al. Vagen: Reinforcing world model reasoning for multi-turn vlm agents. In NeurIPS, 2025a. - Wang et al. [2024a] Ruoyao Wang, Graham Todd, Ziang Xiao, Xingdi Yuan, Marc-Alexandre Côté, Peter Clark, and Peter Jansen. Can language models serve as text-based world simulators? In ACL, 2024a. - Wang et al. [2025b] Siting Wang, Luoyang Sun, Cheng Deng, Kun Shao, Minnan Pei, Zheng Tian, Haifeng Zhang, and Jun Wang. Spatialviz-bench: Automatically generated spatial visualization reasoning tasks for mllms. arXiv preprint arXiv:2507.07610, 2025b. - Wang et al. [2024b] Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024b. - Wu et al. [2025a] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In CVPR, 2025a. - Wu et al. [2024] Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, and Mingsheng Long. ivideogpt: Interactive videogpts are scalable world models. In NeurIPS, 2024. - Wu et al. [2025b] Jialong Wu, Shaofeng Yin, Ningya Feng, and Mingsheng Long. Rlvr-world: Training world models with reinforcement learning. In NeurIPS, 2025b. - Xie et al. [2025] Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In ICLR, 2025. - Xu et al. [2025] Yi Xu, Chengzu Li, Han Zhou, Xingchen Wan, Caiqi Zhang, Anna Korhonen, and Ivan Vulić. Visual planning: Let’s think only with images. arXiv preprint arXiv:2505.11409, 2025. - Yang et al. [2025a] Rui Yang, Ziyu Zhu, Yanwei Li, Jingjia Huang, Shen Yan, Siyuan Zhou, Zhe Liu, Xiangtai Li, Shuangye Li, Wenqian Wang, et al. Visual spatial tuning. arXiv preprint arXiv:2511.05491, 2025a. - Yang et al. [2025b] Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence. arXiv preprint arXiv:2505.23764, 2025b. - Yao et al. [2025] Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Chi Chen, Haoyu Li, Weilin Zhao, et al. Efficient gpt-4v level multimodal large language model for deployment on edge devices. Nature Communications, 2025. - Yin et al. [2025] Baiqiao Yin, Qineng Wang, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, et al. Spatial mental modeling from limited views. In Structural Priors for Vision Workshop at ICCV’25, 2025. - Zhang et al. [2025] Kai Zhang, Xiangchao Chen, Bo Liu, Tianci Xue, Zeyi Liao, Zhihan Liu, Xiyao Wang, Yuting Ning, Zhaorun Chen, Xiaohan Fu, et al. Agent learning via early experience. arXiv preprint arXiv:2510.08558, 2025. - Zhao et al. [2025a] Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of-thought reasoning for vision-language-action models. In CVPR, 2025a. - Zhao et al. [2025b] Xiangyu Zhao, Peiyuan Zhang, Kexian Tang, Xiaorong Zhu, Hao Li, Wenhao Chai, Zicheng Zhang, Renqiu Xia, Guangtao Zhai, Junchi Yan, et al. Envisioning beyond the pixels: Benchmarking reasoning-informed visual editing. In NeurIPS, 2025b. - Zhou et al. [2025a] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. In ICLR, 2025a. - Zhou et al. [2025b] Yiyang Zhou, Haoqin Tu, Zijun Wang, Zeyu Wang, Niklas Muennighoff, Fan Nie, Yejin Choi, James Zou, Chaorui Deng, Shen Yan, et al. When visualizing is the first step to reasoning: Mira, a benchmark for visual chain-of-thought. arXiv preprint arXiv:2511.02779, 2025b. - Zou et al. [2025] Kai Zou, Ziqi Huang, Yuhao Dong, Shulin Tian, Dian Zheng, Hongbo Liu, Jingwen He, Bin Liu, Yu Qiao, and Ziwei Liu. Uni-mmmu: A massive multi-discipline multimodal unified benchmark. arXiv preprint arXiv:2510.13759, 2025. ## 7 Theorectical Analysis ### 7.1 Informativeness In this section, we provide the rigorous version of our world model-based chain-of-thought formulations, and proofs for Theorem 1 and Theorem 2. Formal problem setup and assumptions. Given a question $Q$ and input images $I$ , multimodal reasoning generates a chain-of-thought process $R$ , followed by a final answer $A$ . We explicitly formulate the reasoning process as an interleaving of logic reasoning steps and observations of the underlying MOMDP defined in Section 3.1: $R=(r_{1},o_{1}),(r_{2},o_{2}),\dots,(r_{H},o_{H})$ where $H$ denotes the (fixed) CoT length. For notation convenience, we denote the input image(s) as the initial observation $o_{0}$ . We assume that each MOMDP observation function admits a two-stage decomposition: $e_{\phi}(s)=g_{\phi_{m}}\left(f_{\phi_{s}}(s)\right),\Phi=\Phi_{s}\times\Phi_{m}$ , where the inner modality-agnostic mapping $f_{\phi_{s}}$ (parameterized by $\phi_{s}\in\Phi_{s}$ ) extracts a slice of the underlying state $s$ , retaining only partial state information, and the outer modality-specific mapping $g_{\phi_{m}}$ (parameterized by $\phi_{m}\in\Phi_{m}$ ) renders the extracted slice into a particular observation modality. Under this decomposition, we assume that reasoning across different modalities of observations shares a common underlying oracle reasoning process: $$ p(Q,\bar{s}_{0},r_{1},\bar{s}_{1},\dots,r_{H},\bar{s}_{H},A)=p(Q)\left[\prod_{i=1}^{H}p(r_{i}|\bar{s}_{0:{i-1}},r_{1:i-1},Q)p(\bar{s}_{i}|\bar{s}_{0:i-1},r_{1:i},Q)\right]p(A|r_{1:H},\bar{s}_{0:H},Q), $$ where $\bar{s}_{i}=(s_{i},{\phi_{s}}_{i})\in\mathcal{S}\times\Phi_{s}$ denotes a modality-agnostic sliced state. Each logic step $r_{i}$ is assumed to reason on sufficient sliced state information: $p(r_{i}\mid\bar{s}_{0:i-1},r_{1:i-1},Q)=p\!\left(r_{i}\mid f_{{\phi_{s}}_{0}}(s_{0}),\dots,f_{{\phi_{s}}_{i-1}}(s_{i-1}),r_{1:i-1},Q\right),$ and produces actionable outcomes that either (i) transit a previous world state $s_{j<i}$ via an implicit action $a_{i}$ : $\bar{s}_{i}=(s_{i},{\phi_{s}}_{j}),s_{i}\sim p(s_{j},a_{i})$ or (ii) query the same underlying world state with a new slice ${\phi_{s}}_{i}$ , yielding $\bar{s}_{i}=(s_{j},{\phi_{s}}_{i})$ The oracle reasoning process is then rendered into a specific modality via $o_{i}=g_{\phi_{m}}\!\left(f_{{\phi_{s}}_{i}}(s_{i})\right).$ Unless otherwise specified, we abuse notation and use $s_{i}$ to denote $\bar{s}_{i}=(s_{i},{\phi_{s}}_{i})$ in the remainder of our analysis. Given the above oracle CoT generation process, we learn a model $p_{\theta}$ whose joint distribution over CoTs and answers factorizes into a reasoning component and a world-modeling component: $$ \displaystyle p_{\theta}(R,A|Q,I)=p_{\theta}(r_{1},o_{1},r_{2},o_{2},\dots,r_{H},o_{H},r_{H+1}|r_{0},o_{0})=\prod_{i=1}^{H+1}p_{\theta}(r_{i}|R_{i})\prod_{i=1}^{H}p_{\theta}(o_{i}|\tilde{R}_{i}), \tag{8} $$ where we denote the question as $r_{0}$ , the initial observation (input image(s)) as $o_{0}$ , and the final answer as $r_{H+1}$ . The CoT prefixes are defined as $R_{i}=(r_{0},o_{0},r_{1},o_{1},\dots,r_{i-1},o_{i-1}),\tilde{R}_{i}=(r_{0},o_{0},r_{1},o_{1},\dots,r_{i-1},o_{i-1},r_{i}).$ Proofs. We provide proofs of Theorem 1 and Theorem 2 below. **Theorem 3 (Restatement of Theorem1)** *For any observation modality $m$ , the following inequality holds: $$ \displaystyle\operatorname{KL}(p(A\mid Q,I)\mid\mid p_{\theta}(A\mid Q,I)) \displaystyle\leq\operatorname{KL}(p(R,A\mid Q,I)\mid\mid p_{\theta}(R,A\mid Q,I)) \displaystyle= \displaystyle\sum_{i=1}^{H+1}\underbrace{\mathbb{E}_{p}\left[\operatorname{KL}(p(r_{i}|R_{i})\mid\mid p_{\theta}(r_{i}|R_{i}))\right]}_{\textnormal{reasoning errors}}+\sum_{i=1}^{H}\underbrace{\mathbb{E}_{p}\left[\operatorname{KL}(p(o_{i}|\tilde{R}_{i})\mid\mid p_{\theta}(o_{i}|\tilde{R}_{i}))\right]}_{\textnormal{world modeling errors}}. \tag{9} $$* * Proof* The first inequality follows from the data processing inequality: marginalizing out $R$ cannot increase the KL divergence. For the equality, we apply the chain rule for KL divergence together with the CoT factorization in Eq. (8). In particular, substituting the factorizations of $p(R,A\mid Q,I)$ and $p_{\theta}(R,A\mid Q,I)$ into $\operatorname{KL}(p(R,A\mid Q,I)\,\|\,p_{\theta}(R,A\mid Q,I))$ leads to the stated decomposition. ∎ **Theorem 4 (Restatement of Theorem2)** *For any observation modality $m$ , the reduction in reasoning uncertainty achieved by explicit world modeling satisfies: 1. Reasoning uncertainty does not increase: $\mathbb{H}(r_{i}\mid o_{0},r_{0:i-1})-\mathbb{H}(r_{i}|R_{i})=\mathbb{I}(o_{1:i-1};r_{i}\mid o_{0},r_{0:i-1})\geq 0.$ 1. Uncertainty reduction is upper-bounded by both (i) the information that observations provide about the underlying states and (ii) the information that the reasoning step requires about those states: $$ \mathbb{I}(o_{1:i-1};r_{i}\mid o_{0},r_{0:i-1})\leq\min\left(\mathbb{I}(o_{1:i-1};s_{1:i-1}),\mathbb{I}(r_{i};s_{0:i-1},r_{0:i-1})\right). \tag{10} $$* * Proof* The first property follows the definition and the non-negativity of mutual information. For the second property, denote the conditioning context as $C=(o_{0},r_{0:i-1})$ . Using the properties of ternary mutual information: $\mathbb{I}(X;Y;Z)=\mathbb{I}(X;Y)-\mathbb{I}(X;Y\mid Z).$ we obtain $$ \displaystyle\mathbb{I}(o_{1:i-1};r_{i}\mid C) \displaystyle=\mathbb{I}(o_{1:i-1};r_{i}\mid C)-\mathbb{I}(o_{1:i-1};r_{i}\mid s_{1:i-1},C)=\mathbb{I}(s_{1:i-1};o_{1:i-1};r_{i}\mid C) \displaystyle=\mathbb{I}(o_{1:i-1};s_{1:i-1}\mid C)-\mathbb{I}(o_{1:i-1};s_{1:i-1}\mid r_{i},C)\leq\mathbb{I}(o_{1:i-1};s_{1:i-1}\mid C), \tag{11} $$ where $\mathbb{I}(o_{1:i-1};r_{i}|s_{1:i-1},C)=0$ follows from the conditional independence $r_{i}\perp o_{1:i-1}\mid s_{1:i-1}$ . Further, due to $o$ as the deterministic function of $s$ , we have: | | $\displaystyle\mathbb{I}(o_{1:i-1};s_{1:i-1}\mid C)$ | $\displaystyle=\mathbb{H}(o_{1:i-1}\mid C)-\mathbb{H}(o_{1:i-1}\mid s_{1:i-1},C)$ | | | --- | --- | --- | --- | where $\mathbb{H}(o_{1:i-1}\mid s_{1:i-1})=\mathbb{H}(o_{1:i-1}\mid s_{1:i-1},C)=0.$ Symmetrically, we have: | | $\displaystyle\mathbb{I}(o_{1:i-1};r_{i}|C)$ | $\displaystyle=\mathbb{I}(s_{1:i-1};o_{1:i-1};r_{i}\mid C)\leq\mathbb{I}(s_{1:i-1};r_{i}|C)=\mathbb{H}(r_{i}|C)-\mathbb{H}(r_{i}|s_{1:i-1},C)$ | | | --- | --- | --- | --- | where $\mathbb{H}(r_{i}|s_{0:i-1},r_{0:i-1})\leq\mathbb{H}(r_{i}|s_{1:i-1},o_{0},r_{0:i-1})$ due to data processing inequality. Combining the two upper bounds proves Eq. (10). ∎ **Corollary 1** *If observations are fully informative about the underlying states, i.e., $\mathbb{H}(s_{i}\mid o_{i})=0$ for all $i$ , and the state transition dynamics are deterministic, then explicit world modeling provides no reduction in reasoning uncertainty: $\mathbb{I}(o_{1:i-1};r_{i}\mid o_{0},r_{0:i-1})=0.$* * Proof* By Eq. (11), we have | | $\displaystyle\mathbb{I}(o_{1:i-1};r_{i}\mid o_{0},r_{0:i-1})\leq\mathbb{I}(o_{1:i-1};s_{1:i-1}\mid o_{0},r_{0:i-1})\leq\mathbb{H}(s_{1:i-1}\mid o_{0},r_{0:i-1}).$ | | | --- | --- | --- | Under the assumption $\mathbb{H}(s_{0}\mid o_{0})=0$ , the initial observation $o_{0}$ uniquely determines $s_{0}$ . Moreover, under deterministic state transitions, the trajectory $s_{1:i-1}$ is uniquely determined by $(s_{0},r_{1:i-1})$ . Hence, $$ \mathbb{H}(s_{1:i-1}\mid o_{0},r_{1:i-1})=\mathbb{H}(s_{1:i-1}\mid s_{0},r_{1:i-1})=0. $$ Therefore, $\mathbb{I}(o_{1:i-1};r_{i}\mid o_{0},r_{1:i-1})=0,$ which proves the corollary. ∎ Remarks. Corollary 1 shows that in deterministic and fully observable environments, given sufficient data and model capacity, explicit world modeling provides no additional benefit. This theoretical result is consistent with our empirical findings on the simple maze task. ### 7.2 Prior Knowledge In this section, we first derive a generalization bound for transfer learning under distribution shift, and relate it to our perspective on prior knowledge in multimodal reasoning. #### 7.2.1 General Transfer Learning Analysis Problem setup. A standard transfer learning setup involves a pre-training data distribution $P$ and the fine-tuning data distribution $Q$ over samples $(x,y)\in\mathcal{X}\times\mathcal{Y}$ , and a loss function $\ell_{\theta}(x,y)\in[0,1].$ Define the population risks $\mathcal{L}_{D}(\theta):=\mathbb{E}_{(x,y)\sim D}[\ell_{\theta}(x,y)],D\in\{P,Q\}$ , and the population minimizers $\theta_{D}^{\star}\in\arg\min_{\theta\in\Theta}\mathcal{L}_{D}(\theta),D\in\{P,Q\}$ We assume we can obtain $\theta_{P}^{*}$ as the pre-trained model given sufficient data and optimization. For a radius $r>0$ , we then define the fine-tuning constraint set (local neighborhood around the pre-trained model) $$ \Theta_{r}:=\{\theta\in\Theta:\ \|\theta-\theta_{P}^{*}\|\leq r\}. $$ Given $n$ i.i.d. samples from $Q$ : $S=\left\{(x_{i},y_{i})_{i=1}^{n}\right\},(x_{i},y_{i})\sim Q$ , the fine-tuned model $\theta_{Q}$ minimize empirical risk over $\Theta_{r}$ , $\widehat{\mathcal{L}}_{Q}(\theta):=\frac{1}{n}\sum_{i=1}^{n}[\ell_{\theta}(x_{i},y_{i})].$ Our analysis focus on the excess risk on $Q$ : $\mathcal{E}_{Q}(\theta_{Q}):=\mathcal{L}_{Q}(\theta_{Q})-\mathcal{L}_{Q}(\theta_{Q}^{\star}).$ From distribution shift to parameter drift. We first derive how the distribution shift relates to the shift of the population minimizer. **Lemma 1 (Uniform Loss Shift under Total Variation)** *For any subset $\mathcal{S}\subseteq\Theta$ , $$ \sup_{\theta\in\mathcal{S}}\big|\mathcal{L}_{Q}(\theta)-\mathcal{L}_{P}(\theta)\big|\leq\mathrm{TV}(P,Q). $$* * Proof* Fix any $\theta\in\mathcal{S}$ and define $f_{\theta}(h,a,o^{\prime}):=\ell_{\theta}(h,a,o^{\prime})\in[0,1]$ . By the definition of total variation and the standard inequality for bounded functions, $\big|\mathbb{E}_{Q}[f_{\theta}]-\mathbb{E}_{P}[f_{\theta}]\big|\leq\mathrm{TV}(P,Q).$ Taking the supremum over $\theta\in\mathcal{S}$ yields the claim. ∎ **Lemma 2 (Risk Proximity ofθQ⋆\theta_{Q}^{\star}underPP)** *$$ \mathcal{L}_{P}(\theta_{Q}^{\star})\leq\mathcal{L}_{P}(\theta_{P}^{\star})+2\mathrm{TV}(P,Q). \tag{12} $$* * Proof* By Lemma 1, $\mathcal{L}_{P}(\theta_{Q}^{\star})\leq\mathcal{L}_{Q}(\theta_{Q}^{\star})+\mathrm{TV}(P,Q).$ By optimality of $\theta_{Q}^{\star}$ on $Q$ , $\mathcal{L}_{Q}(\theta_{Q}^{\star})\leq\mathcal{L}_{Q}(\theta_{P}^{\star}).$ Applying Lemma 1 again, $\mathcal{L}_{Q}(\theta_{P}^{\star})\leq\mathcal{L}_{P}(\theta_{P}^{\star})+\mathrm{TV}(P,Q).$ Chaining the three inequalities proves (12). ∎ **Assumption 1 (Local Quadratic Growth / Sharpness ofℒP\mathcal{L}_{P})** *There exists $\mu>0$ such that for all $\theta$ in a neighborhood containing $\theta_{Q}^{\star}$ , $$ \mathcal{L}_{P}(\theta)\geq\mathcal{L}_{P}(\theta_{P}^{\star})+\frac{\mu}{2}\|\theta-\theta_{P}^{\star}\|^{2}. $$* **Lemma 3 (Parameter Drift Controlled byTV​(P,Q)\mathrm{TV}(P,Q))** *Under Assumption 1, $$ \|\theta_{Q}^{\star}-\theta_{P}^{\star}\|\leq\sqrt{\frac{4}{\mu}\,\mathrm{TV}(P,Q)}. \tag{13} $$* * Proof* By Assumption 1 with $\theta=\theta_{Q}^{\star}$ , $\mathcal{L}_{P}(\theta_{Q}^{\star})\geq\mathcal{L}_{P}(\theta_{P}^{\star})+\frac{\mu}{2}\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|^{2}.$ Rearranging, $\frac{\mu}{2}\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|^{2}\leq\mathcal{L}_{P}(\theta_{Q}^{\star})-\mathcal{L}_{P}(\theta_{P}^{\star}).$ Applying Lemma 2 yields $\frac{\mu}{2}\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|^{2}\leq 2\mathrm{TV}(P,Q),$ and hence (13). ∎ Control of the bias term. Recall the fine-tuning bias induced by restricting to $\Theta_{r}$ : $\varepsilon_{\mathrm{bias}}(r):=\inf_{\theta\in\Theta_{r}}\mathcal{L}_{Q}(\theta)-\mathcal{L}_{Q}(\theta_{Q}^{\star}).$ **Assumption 2 (ℒQ\mathcal{L}_{Q}is Locally Lipschitz)** *There exists $L_{Q}>0$ such that for all $\theta,\theta^{\prime}\in\Theta_{r}$ , $$ |\mathcal{L}_{Q}(\theta)-\mathcal{L}_{Q}(\theta^{\prime})|\leq L_{Q}\|\theta-\theta^{\prime}\|. $$* **Theorem 5 (Bias Bound via Distribution Shift)** *Under Assumption 1 and Assumption 2, $$ \varepsilon_{\mathrm{bias}}(r)\leq L_{Q}\left(\sqrt{\frac{4}{\mu}\,\mathrm{TV}(P,Q)}-r\right)_{+}, \tag{14} $$ where $(x)_{+}:=\max\{x,0\}$ . In particular, if $r\geq\sqrt{\frac{4}{\mu}\,\mathrm{TV}(P,Q)},$ then $\varepsilon_{\mathrm{bias}}(r)=0$ .* * Proof* If $r\geq\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|$ , then $\theta_{Q}^{\star}\in\Theta_{r}$ and thus $\inf_{\theta\in\Theta_{r}}\mathcal{L}_{Q}(\theta)\leq\mathcal{L}_{Q}(\theta_{Q}^{\star})$ , implying $\varepsilon_{\mathrm{bias}}(r)=0$ . Now consider $r<\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|$ . Let $\theta_{r}$ be the projection of $\theta_{Q}^{\star}$ onto the closed ball $\Theta_{r}$ , i.e., $\theta_{r}:=\theta_{P}+r\cdot\frac{\theta_{Q}^{\star}-\theta_{P}^{\star}}{\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|}$ . Then $\theta_{r}\in\Theta_{r}$ and $\|\theta_{r}-\theta_{Q}^{\star}\|=\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|-r.$ Therefore, $$ \varepsilon_{\mathrm{bias}}(r)=\inf_{\theta\in\Theta_{r}}\mathcal{L}_{Q}(\theta)-\mathcal{L}_{Q}(\theta_{Q}^{\star})\leq\mathcal{L}_{Q}(\theta_{r})-\mathcal{L}_{Q}(\theta_{Q}^{\star})\leq L_{Q}\|\theta_{r}-\theta_{Q}^{\star}\|=L_{Q}(\|\theta_{Q}^{\star}-\theta_{P}^{\star}\|-r). $$ Using Lemma 3 to bound $\|\theta_{Q}^{\star}-\theta_{P}\|$ completes the proof of (14). ∎ Fine-tuning excess risk bound. We then arrive at the final result: **Theorem 6 (Fine-tuning Excess Risk Bound with Shift-Controlled Bias)** *Assume Assumptions 1 and 2 and uniform convergence over $\Theta_{r}$ holds: with probability at least $1-\delta$ over samples $S$ , $$ \sup_{\theta\in\Theta_{r}}\big|\mathcal{L}_{Q}(\theta)-\widehat{\mathcal{L}}_{Q}(\theta)\big|\leq\varepsilon_{\mathrm{gen}}=O\!\left(\sqrt{\frac{\operatorname{Rad}_{Q,n}(\Theta_{r})+\log(1/\delta)}{n}}\right), $$ where $\operatorname{Rad}_{Q,n}(\Theta_{r})$ is the Rademacher complexity of of the function class $\{\ell_{\theta},\theta\in\Theta_{r}\}$ with respect to $Q$ for sample size $n$ . Then with probability at least $1-\delta$ , $$ \mathcal{E}_{Q}(\theta_{Q})\leq 2\varepsilon_{\mathrm{gen}}+L_{Q}\left(\sqrt{\frac{4}{\mu}\,\mathrm{TV}(P,Q)}-r\right)_{+}. \tag{15} $$* * Proof* Decompose the excess risk as $\mathcal{E}_{Q}(\theta_{Q})=\Big(\mathcal{L}_{Q}(\theta_{Q})-\inf_{\theta\in\Theta_{r}}\mathcal{L}_{Q}(\theta)\Big)+\varepsilon_{\mathrm{bias}}(r).$ The first term is bounded by a standard ERM argument using uniform convergence: $\mathcal{L}_{Q}(\theta_{Q})-\inf_{\theta\in\Theta_{r}}\mathcal{L}_{Q}(\theta)\leq 2\varepsilon_{\mathrm{gen}}.$ The second term is bounded by Lemma 5. Combining the two bounds yields (15). ∎ #### 7.2.2 Remarks on Multimodal Reasoning Theorem 6 reveals a trade-off between modality complexity and distribution shift. This general transfer learning analysis can be instantiated in our setting of learning world models and reasoning policies. Specifically, training pairs $(x,y)$ can be instantiated as $((o_{0:i},r_{0:{i+1}}),o_{i+1})$ for world modeling and $((o_{0:i},r_{0:{i}}),r_{i})$ for reasoning, respectively. Crucially, the distribution shift between large-scale pre-training data and downstream tasks may differ substantially across modalities. For example, there are abundant visual demonstrations of paper folding on the Internet, whereas detailed verbal descriptions of folding dynamics are comparatively scarce. This suggests that downstream tasks should be formulated under the most appropriate observation modality for world modeling and reasoning—i.e., the modality that best aligns with pre-training data—in order to achieve stronger generalization at inference time and higher sample efficiency during post-training. ## 8 Experiment Details ### 8.1 VisWorld-Eval and Training Data In this section, we elaborate on the construction of training and test data for each task in VisWorld-Eval. Paper folding. This task involves folding a paper grid with varying grid sizes (3–8) and folding steps (1–4). After folding, holes of different shapes—circles, triangles, stars, diamonds, and squares—are punched into the paper. The model is then asked to predict the distribution of holes after the paper is completely unfolded, including queries such as the total number of holes, the number of holes of a specific shape, or the difference in counts between shapes. All test prompts are constructed at the highest difficulty level (grid size 8 with 4 folding steps). For SFT, we generate chain-of-thoughts using rule-based templates that follow a fixed procedure: unfold the paper step-by-step and then count the resulting holes by shape. These CoTs are then rewritten with Gemini 2.5 Pro to improve clarity and logical coherence. Under visual world modeling, we interleave reasoning steps with images of partially unfolded paper states. Under verbal world modeling, we represent intermediate states using two matrices encoding grid coverage status and hole shape at each position. Under implicit world modeling, we directly skip the explicit tracking of states from original CoTs. Multi-hop manipulation. This task begins with an initial arrangement of several geometric objects (cubes, spheres, and cylinders) in various colors, rendered by Blender https://www.blender.org/. A sequence of text-based instructions is then provided, describing operations such as changing or swapping objects’ color or shape, adding new objects, or removing existing ones. To ensure these commands can be interpreted unambiguously in a 3D space, the instructions consistently use relative spatial references, with each object uniquely identified by its combined color and shape attributes—for example: "Place a purple cylinder between the black sphere and the yellow cube." The model is asked to infer the resulting spatial layout. Queries may include the total number of objects of a specific shape, the directional relationship between two objects, or which object lies in a given direction relative to a reference object. Test prompts are constructed by varying both the number of initial objects (between 3 and 6) and the frequency (between 1 and 5) of different operation types. For SFT, chain-of-thought reasoning is generated using rule-based templates that simulate the stepwise execution of instructions before answering the final query, and these CoTs are subsequently refined with Gemini 2.5 Pro. Ball tracking. This task features a red point-mass ball that moves at constant speed, reflects elastically off solid walls, and travels in the initial direction indicated by a green arrow. The model is asked to predict which numbered hole at the top of the image the ball will enter first. We generate input images with randomized resolution, initial ball position and direction, and a random number of holes (4–8). For test prompts, we select cases in which the ball trajectory reflects off at least one wall before entering a hole. For SFT, CoTs are generated by Seed 1.6, which is asked to explain the ball dynamics between adjacent frames. Sokoban. Sokoban is a classic grid-based puzzle game. We generate instances with grid sizes ranging from 6 to 10, containing a single box and a target position. Test prompts are sampled from the same distribution as the training data. To construct CoTs, we use a search algorithm to compute an optimal solution path. To avoid excessively long trajectories, we render only key intermediate steps, including: (i) the player moving toward the box, (ii) pushing the box in a direction, and (iii) changing the pushing direction. To encourage reflective behavior, we additionally augment trajectories with randomized detours that involve walking into walls, reflecting, and backtracking to rejoin the optimal path. CoTs are generated by Seed 1.6, which explains the dynamics between adjacent frames. For visual world modeling, the rendered intermediate steps are interleaved with verbal CoTs. For pure verbal world modeling, these intermediate renderings are removed. For implicit world modeling, we additionally mask all explicit coordinates during CoTs with special tokens [masked]. Maze. Maze is a classic grid-based puzzle task. We generate both training and test samples with a fixed grid size of $5\times 5$ . To construct CoTs, we use rule-based templates followed by rewriting for improved naturalness. Under visual world modeling, rendered intermediate steps through points and lines are interleaved with verbal CoTs. The settings for verbal and implicit world modeling follow the same protocol as in Sokoban, with masking special tokens as <point>[masked]</point>. Cube 3-view projection. This task considers stacks of colored cubes arranged on grids of varying sizes (3–5), with two cube colors. The input consists of one isometric view (either front-left or front-right) and two orthographic views of the stack. The question asks for the number of cubes of a specified color visible from another orthogonal view. Both the questions and answer choices account for ambiguity caused by occlusions, leading to uncertainty in the cube count. All test prompts are constructed using uniformly random grid sizes between 3 and 5. We generate CoTs using rule-based templates: the model first constructs the queried view, marks potentially occluded cubes using a third (auxiliary) color, and then counts cubes by color. These CoTs are subsequently rewritten by Gemini 2.5 Pro for improved naturalness. Under visual world modeling, we interleave reasoning steps with an image of the queried view. Under verbal world modeling, we represent intermediate views using character matrices, where different colors are encoded by different symbols. Real-world spatial reasoning. For this real-world task, we directly adopt test samples from MMSI-Bench, focusing on camera–object and camera–region positional relationship questions. We construct training prompts following a pipeline similar to Yang et al. [68]. To obtain training CoTs, we run a visual-CoT model, which uses an SFT-trained BAGEL model for novel view synthesis as a tool. The resulting visual CoTs are subsequently filtered and rewritten by Gemini 2.5 Pro. We summarize the training and test sample counts for each task in VisWorld-Eval, along with the corresponding original or referenced benchmarks, in Table 2. Table 2: Overview of VisWorld-Eval and corresponding training data: features, statistics, and references. | Task | Capability | Domain | Training Samples | Test Samples | Source/Reference | | --- | --- | --- | --- | --- | --- | | Paper folding | Simulation | Synthetic | 2,357 | 480 | SpatialViz [61] | | Multi-hop manipulation | Simulation | Synthetic | 2,000 | 480 | ZebraCoT [35], CLEVR [30] | | Ball tracking | Simulation | Synthetic | 2,254 | 1,024 | RBench-V [20] | | Maze | Simulation | Synthetic | 8,448 | 480 | maze-dataset [29] | | Sokoban | Simulation | Synthetic | 7,715 | 480 | GameRL [55] | | Cube 3-view projection | Reconstruction | Synthetic | 2,500 | 480 | SpatialViz [61] | | Real-world spatial reasoning | Reconstruction | Real-world | 10,661 | 522 | MMSI-Bench [69] | Examples of training CoTs are presented in Figure 10, 11, 12, 13, and 14. ### 8.2 Model Training We perform supervised fine-tuning (SFT) of BAGEL based on its official repository https://github.com/ByteDance-Seed/Bagel, using 8 GPUs, and conduct reinforcement learning from verifiable rewards (RLVR) using verl https://github.com/volcengine/verl on 64 GPUs. Hyperparameters for SFT and RLVR are reported in Table 4 and Table 4, respectively. Table 3: Hyperparameters for supervised fine-tuning UMMs. | Hyperparameter | Value | | --- | --- | | Learning rate | $3\times 10^{-5}$ | | LR Schedule | Constant | | Optimizer | AdamW | | Loss weight (CE:MSE) | 1:10 | | Warm-up steps | 200 | | Training steps | 4000 | | Gen. resolution | (256, 1024) for paper folding, cube 3-view | | (240, 1024) for multi-hop manipulation | | | (256, 512) otherwise | | | Und. resolution | (224, 980) | | Sequence length per rank | 32K | | Num. ranks | 8 | Table 4: Hyperparameters for reinforcement learning UMMs. | Hyperparameter | Value | | --- | --- | | Learning rate | $1\times 10^{-5}$ | | Batch size | 128 | | GRPO mini batch size | 32 | | Group size | 16 | | KL loss coefficient for visual gen. | 0.1 | | KL loss coefficient for verbal gen. | 0.0 | <details> <summary>x12.png Details</summary>  ### Visual Description ## Paper Folding Diagram: Unfolding Process and Cutout Analysis ### Overview The image presents a technical problem involving paper folding and cutout patterns. It includes three diagrams showing the paper's folding stages, code-like structures representing coordinate transformations, and a final answer calculation. The problem asks to determine the number of `triangle_left` cutouts after fully unfolding a folded paper with specific cut patterns. ### Components/Axes 1. **Diagrams**: - **Top Diagram**: Shows the folded paper with red dashed lines indicating fold lines and white triangles representing cutouts. - **Middle Diagram**: Illustrates the paper after reversing the second fold (vertical reflection). - **Bottom Diagram**: Shows the fully unfolded paper with mirrored cutouts. 2. **Code Blocks**: - **First Code Block**: Represents initial folded state with coordinates and shapes: ```python [[[0, -1, -1], [1, -1, -1], [1, -1, -1]], [['triangle_right', '', ''], ['triangle_left', '', '']]] ``` - **Second Code Block**: After first reflection (vertical fold reversal): ```python [[[0, 0, -1], [1, 1, -1], [1, 1, -1]], [['triangle_right', 'triangle_left', ''], ['triangle_left', 'triangle_right', '']]] ``` - **Third Code Block**: Final unfolded state: ```python [[[0, 0, 0], [1, 1, 1], [1, 1, 1]], [['triangle_right', 'triangle_left', 'triangle_right'], ['triangle_left', 'triangle_right', 'triangle_left']]] ``` 3. **Textual Elements**: - Problem statement: "How many triangle_left cutouts are there in the unfolded paper?" - Step-by-step explanation of reflection transformations. - Final answer: "3 triangle_left cutouts." ### Detailed Analysis 1. **Folding Process**: - **First Fold (Vertical)**: Right side folded over left. Coordinates transform via reflection across the vertical axis between columns 0 and 1. - `triangle_right` at (1,0) becomes `triangle_left` at (1,1). - `triangle_left` at (2,0) becomes `triangle_right` at (2,1). - **Second Fold (Horizontal)**: Right third folded over middle third. Coordinates transform via reflection across the horizontal axis between rows 1 and 2. - Shapes in middle column (originally single-layer) become double-layered, creating mirrored shapes. 2. **Final Unfolded State**: - The bottom diagram shows 6 total cutouts (3 `triangle_left`, 3 `triangle_right`). - Code block confirms 3 `triangle_left` instances in the final array. ### Key Observations 1. **Mirroring Logic**: - Each fold creates symmetrical transformations. Vertical folds mirror left/right, horizontal folds mirror top/bottom. - Double-layered cuts (middle column) produce two mirrored shapes per original cut. 2. **Coordinate System**: - Uses 2D grid with [row, column, depth] coordinates. - Depth (-1/1) indicates layer position (folded/unfolded). 3. **Shape Transformation**: - `triangle_right` ↔ `triangle_left` mirroring occurs during reflections. - Final pattern shows alternating shapes in adjacent columns. ### Interpretation The problem demonstrates spatial reasoning through iterative reflection transformations. Each fold operation: 1. Creates symmetrical duplicates of cutouts 2. Merges layers through reflection 3. Preserves original cut positions while mirroring shapes The final count of 3 `triangle_left` cutouts emerges from: - Original left column cuts (unchanged) - Mirrored cuts from middle column (2 instances) - Mirrored cuts from right column (1 instance) This pattern validation through coordinate transformations and shape mirroring confirms the solution's correctness. The technical approach combines geometric reflection principles with array-based state tracking, providing a systematic method for solving spatial folding problems. </details> Figure 10: Examples of chain-of-thought SFT data for the paper folding task, under visual world modeling (left) and verbal world modeling (right). <details> <summary>x13.png Details</summary>  ### Visual Description ## Screenshot: Ball Tracking and Multi-Hop Manipulation Tasks ### Overview The image contains two technical problem-solving scenarios: 1. **Ball Tracking**: A physics-based problem involving a red ball's trajectory and reflection rules. 2. **Multi-Hop Manipulation**: A spatial reasoning task involving object transformations and positional changes. Both sections include questions, step-by-step reasoning, and final answers. --- ### Components/Axes #### Ball Tracking Section - **Diagram**: Rectangular area with three horizontal holes (labeled 1, 2, 3) at the top. - **Objects**: - Red ball with a green arrow indicating initial motion. - Solid black boundaries representing walls. - **Rules**: - Ideal reflection (perpendicular velocity reverses, parallel component unchanged). - Ball treated as a point mass. #### Multi-Hop Manipulation Section - **Initial Scene**: - Objects: Blue cylinder, gray sphere, gray cuboid, red cylinder, yellow cuboid. - Perspective: Oblique front view (closer objects = "front"). - **Operations**: 1. Change yellow cuboid's direction to a rose cylinder. 2. Place gray cylinder behind the rose cylinder. 3. Place gray sphere to the left of the rose cylinder. --- ### Detailed Analysis #### Ball Tracking 1. **Initial Setup**: - Red ball starts at the bottom-left corner, moving upward-right (green arrow). - Holes 1, 2, 3 are horizontal at the top. 2. **Motion Analysis**: - Ball moves downward until hitting the bottom wall (first reflection). - Vertical velocity reverses; horizontal velocity remains unchanged. - Path after reflection is symmetric to the incoming path. 3. **Final Destination**: - After reflection, the ball's path intersects **hole 2**. - Final answer: `\boxed{2}`. #### Multi-Hop Manipulation 1. **Operation 1**: - Yellow cuboid's direction changed to a rose cylinder. 2. **Operation 2**: - Gray cylinder placed behind the rose cylinder. 3. **Operation 3**: - Gray sphere placed to the left of the rose cylinder. 4. **Final Question**: - "What is the object to the right of the blue cylinder?" - Answer: **A. gray cylinder** (positioned to the right of the blue cylinder in the final scene). --- ### Key Observations - **Ball Tracking**: - The ball's trajectory is governed by reflection rules, leading to hole 2. - No other holes are intersected due to the symmetry of the path. - **Multi-Hop Manipulation**: - Object transformations alter spatial relationships. - Final object placement confirms the gray cylinder is to the right of the blue cylinder. --- ### Interpretation - **Ball Tracking**: Demonstrates ideal reflection physics, where the ball's path is predictable based on initial velocity and wall interactions. - **Multi-Hop Manipulation**: Highlights spatial reasoning, requiring sequential object transformations and positional updates. - **Notable Patterns**: - The ball's reflection rules ensure a deterministic outcome (hole 2). - Object manipulations create a clear spatial hierarchy in the final scene. This structured approach ensures all textual and diagrammatic elements are extracted and contextualized for technical documentation. </details> Figure 11: Examples of chain-of-thought SFT data for the ball tracking and multi-hop manipulation task. <details> <summary>x14.png Details</summary>  ### Visual Description ## Screenshot: AI Problem-Solving Dialogue ### Overview The image shows a technical dialogue between a user and an AI assistant solving two distinct problems: 1. **Maze Navigation**: Finding a path from a red dot (start) to a blue X (goal) in a grid. 2. **Sokoban Puzzle**: Pushing a box to a goal position in a grid with walls and obstacles. Both responses include step-by-step reasoning, positional coordinates, and code-like sequences representing movements. --- ### Components/Axes #### Maze Section - **Question**: - Task: Navigate a maze from a red dot (start) to a blue X (goal). - Input: Grid with walls (blocked paths), start (red dot), and goal (blue X). - Output: List of waypoints in format `:x/y/<point>x y/<point>y...`. - **Response**: - Step-by-step pathfinding using coordinate analysis (e.g., `point=165 83`, `point=246 83`). - Code-like sequences (e.g., `bbx[up, up, right]`) to represent movements. - Visualizations of maze progress (e.g., `image` tags for path diagrams). #### Sokoban Section - **Question**: - Task: Push a box (marked `X`) to a goal (marked `O`) in a grid with walls (`#`) and floor tiles (`.`). - Constraints: Player can only push the box, not pull it. - Output: Sequence of movements in format `bbx[direction1, direction2, ...]`. - **Response**: - Step-by-step analysis of player and box positions (e.g., `(3,1)`, `(1,2)`). - Code-like sequences (e.g., `bbx[up, up, right]`) to represent pushes. - Visualizations of grid states (e.g., `image` tags for grid snapshots). --- ### Detailed Analysis #### Maze Solution 1. **Start**: Red dot at `(165, 83)`. 2. **Path**: - Move right to `(246, 83)` (blocked above, left, and below). - Move down to `(246, 246)` (blocked left and right). - Move down to `(246, 328)` (blocked left). - Move left to `(83, 328)` (blocked left and below). - Move down to `(83, 410)` (goal reached). 3. **Final Path**: `bbx[right, right, down, down, left, down]`. #### Sokoban Solution 1. **Initial State**: - Player at `(3,1)`, box at `(1,2)`, goal at `(1,4)`. 2. **Steps**: - **Step 1**: Move left to `(2,1)`, then up to `(1,1)`. - **Step 2**: Push box right twice to `(1,4)` (goal). 3. **Final Sequence**: `bbx[up, up, right]`. --- ### Key Observations 1. **Maze**: - The AI uses coordinate tracking to navigate blocked paths. - The final path avoids dead ends by prioritizing open corridors. 2. **Sokoban**: - The AI calculates box movements by analyzing player proximity and grid constraints. - The solution requires precise sequencing to avoid blocking the box. 3. **Code Sequences**: - Both responses use `bbx[]` syntax to encode movements, suggesting a domain-specific language for pathfinding. --- ### Interpretation - **Problem-Solving Methodology**: The AI breaks down complex spatial problems into discrete steps, using positional coordinates and grid constraints to guide decisions. - **Code Representation**: The `bbx[]` syntax likely represents a "bounded box" or "bounded movement" framework, abstracting directional actions for automation. - **Visual Aids**: The `image` tags imply dynamic visualization of progress, though the actual diagrams are not fully visible in the screenshot. - **Limitations**: The responses assume perfect grid knowledge (e.g., no hidden walls), which may not reflect real-world uncertainty. This dialogue demonstrates an AI’s ability to decompose spatial reasoning tasks into algorithmic steps, leveraging coordinate systems and constrained movement logic. </details> Figure 12: Examples of chain-of-thought SFT data for the maze and sokoban task. <details> <summary>x15.png Details</summary>  ### Visual Description ## Screenshot: Cube 3-View Projection Problem ### Overview The image depicts a step-by-step solution to a spatial reasoning problem involving a 3D cube stack. It includes isometric and 2D projections of cube arrangements, with a focus on determining the number of red cubes visible from the left perspective. The solution involves iterative reasoning, uncertainty handling, and logical deduction. ### Components/Axes - **Title**: "Cube 3-View Projection" (top-left). - **Views**: - **Front-Right View**: A 3x3 grid with red (`O`) and lightcyan (`X`) cubes. - **Top View**: A 3x3 grid with red (`O`) and lightcyan (`X`) cubes. - **Right View**: A 3x3 grid with red (`O`) and lightcyan (`X`) cubes. - **Legend**: - `O`: Red cube. - `X`: Lightcyan cube. - `*`: Uncertain cube (color unknown). - **Diagrams**: - Initial left view construction (partial). - Corrected left view accounting for hidden cubes. - Final answer diagram. ### Detailed Analysis 1. **Initial Left View Construction**: - A 4x4 grid with `O`, `X`, and `*` symbols. - Confirmed red cubes: 2 (`O`). - Uncertain positions: 2 (`*`). 2. **Corrected Left View**: - Expanded to 4x4 grid with `O`, `X`, and `*` symbols. - Confirmed red cubes: 2 (`O`). - Uncertain positions: 2 (`*`). 3. **Final Answer**: - Red cubes visible from the left range from **2 to 4**, inclusive. - Uncertain cubes (`*`) could be red or lightcyan, leading to maximum visibility of 4 red cubes. ### Key Observations - The problem requires synthesizing 3D spatial relationships from 2D projections. - Uncertainty in hidden cubes (`*`) introduces a range of possible outcomes. - The solution iteratively refines the left view by accounting for occluded cubes. ### Interpretation The data demonstrates a method for resolving spatial ambiguity in 3D projections. By marking hidden cubes as uncertain (`*`), the solution accounts for potential variations in visibility. </details> Figure 13: Examples of chain-of-thought SFT data for the cube 3-view projection task, under visual world modeling (left) and verbal world modeling (right). <details> <summary>x16.png Details</summary>  ### Visual Description ```markdown ## Screenshot: Spatial Reasoning Task with Image-Based Responses ### Overview The image depicts a structured reasoning task involving spatial analysis of objects in a room. It includes two questions with multiple-choice options, followed by detailed responses that combine textual reasoning and references to generated images. The responses use XML-like tags (` </details> Figure 14: Examples of chain-of-thought SFT data for the real-world spatial reasoning task. The Qwen-VL baselines are trained using LLaMA-Factory https://github.com/hiyouga/LLaMA-Factory for supervised fine-tuning (SFT) and verl for reinforcement learning from verifiable rewards (RLVR). ### 8.3 Analytic Experiments Sample efficiency. For Figure 6, we randomly subsample either 500 or 1000 training examples. The resulting models are evaluated under two settings: (i) a hard setting with the maximum difficulty (grid size 8 and 4 folding steps, default in VisWorld-Eval), and (ii) an in-distribution setting (denoted as Normal in the figure) with randomly sampled grid sizes (3–8) and folding steps (1–4). Task difficulties and world model fidelity. For Figure 6, we generate test samples with varying cube-stack sizes (3–6), where size 6 is out-of-distribution relative to the training data. To assess world-model fidelity, we compare the generated views with the ground-truth views: for verbal world modeling, we use string pattern matching; for visual world modeling, we use Gemini 3 Pro to compare images. Since accurately inferring colors becomes particularly challenging at larger stack sizes, we evaluate only the shapes of the views and ignore color information. We also find that overall accuracy can be bottlenecked by verbal subskills (e.g., counting holes) after SFT, thus we report the accuracy of RL-trained models in Figure 6. In contrast, RL can distract verbal world modeling capabilities, leading to invalid formats of generated symbolic matrices, thus we report world-model fidelity of SFT-trained models. Implicit world modeling. For Figure 6, we supervised fine-tune (SFT) BAGEL on CoTs with implicit world modeling, in which all explicit point coordinates are replaced by the placeholder token sequence <point>masked<point>. After training, we extract the hidden representations at the position of the token masked from each transformer layer. We then split the extracted representations from different CoTs into training and validation sets with an 8:2 ratio and train a two-layer MLP (hidden size 4096) to predict the ground-truth point coordinates. Since all samples are $5\times 5$ mazes, we formulate coordinate prediction as two 5-way classification tasks (for $x$ and $y$ , respectively). We compute classification accuracy for each coordinate and report the average of the two. ## 9 Extended Experimental Results ### 9.1 Full Results on MMSI-Bench We report all scores on positional relationship tasks of MMSI-Bench in Table 5. Table 5: Full results of SFT-trained UMMs on MMSI-Bench positional relationship tasks. | | MMSI-Bench (Positional Relationship) | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | | Models | Cam.-Cam. | Obj.–Obj. | Reg.–Reg. | Cam.–Obj. | Obj.–Reg. | Cam.–Reg. | Overall | | Implicit WM | 33.1 | 31.2 | 31.8 | 46.5 | 29.1 | 37.3 | 34.8 | | Visual WM | 29.6 | 29.5 | 31.6 | 60.9 | 25.8 | 54.4 | 38.4 | ### 9.2 Additional Qualitative Evaluation We provide additional qualitative evaluation of trained UMMs’ reasoning, particularly failure cases. Real-world spatial reasoning. As shown in Figure 15 a, reasoning with implicit world modeling is prone to hallucinations. In contrast, visual generation (Figure 15 b) yields more faithful world models, but still suffers from insufficient quality, including blurring and corrupted details. Moreover, we find that current VLMs and UMMs continue to exhibit limited understanding of positions and directions across different viewpoints. We expect that stronger base models and better-curated post-training data will enable more effective use of visual world models for spatial reasoning in future work. Paper folding. As illustrated in Figure 16, verbal reasoning about geometric symmetry is prone to hallucinations, leading to inaccurate verbal world modeling. In contrast, visual world models, benefiting from stronger prior knowledge, generate correct intermediate unfolding steps even in the presence of erroneous verbal reasoning. Cube 3-view projection. As shown in Figure 17, visual world models are able to approximately generate novel views of cube stacks even in the challenging out-of-distribution setting with an unseen stack size of 6, indicating strong prior knowledge of spatial transformations. Nevertheless, overall task performance remains limited by subtle shape-generation errors (Figure 17 b,d) and inaccurate color inference (Figure 17 c). We expect these issues to be alleviated through improved post-training and stronger base models. <details> <summary>x17.png Details</summary>  ### Visual Description ## Real-World Spatial Reasoning Document Analysis ### Overview The document presents two spatial reasoning scenarios (labeled a and b) with paired images, questions, multiple-choice options, and model-generated reasoning processes. Each scenario tests implicit and visual world modeling capabilities through step-by-step logical deductions. ### Components/Axes **Section (a):** - **Images:** 1. Black chair in front of white wall with painting 2. Bed with black headboard against white wall - **Question:** "In which direction is the black chair relative to you in the last image?" - **Options:** A: Front right, B: Back left, C: Front left, D: Back right - **Model Output (Implicit World Modeling):** - Analyzes spatial relationships between images - Simulates camera movements and perspective changes - Concludes chair is "back" relative to camera position **Section (b):** - **Images:** 1. White door with glass panels 2. TV area with yellow wall and blue/green wall art - **Question:** "When entering through the white door, which direction reaches the TV area?" - **Options:** A: Straight, B: Left, C: Cannot determine, D: Right - **Model Output (Visual World Modeling):** - Uses image analysis to map room layout - Simulates camera movements (left/right/backward) - Determines TV area is "left" relative to entry point ### Detailed Analysis **Section (a) Reasoning Flow:** 1. Analyzes initial images to build mental room map 2. Identifies hallway continuity between images 3. Simulates 180-degree perspective shift 4. Combines findings to conclude chair position **Section (b) Reasoning Flow:** 1. Maps room layout from initial images 2. Simulates leftward camera movement 3. Confirms TV area position through spatial relationship 4. Final determination: "go left" ### Key Observations - Both scenarios require perspective transformation analysis - Implicit modeling focuses on object relationships - Visual modeling emphasizes environmental mapping - Final answers: (a) B (Back left), (b) B (Go left) ### Interpretation The document demonstrates: 1. **Spatial Reasoning Complexity:** Requires understanding perspective shifts and environmental relationships 2. **Modeling Approaches:** - Implicit modeling uses object-centric reasoning - Visual modeling employs environmental mapping 3. **Decision Process:** Combines image analysis with simulated camera movements 4. **Accuracy:** Both scenarios show correct final answers through logical deduction chains The structured approach reveals how spatial reasoning models process visual information through: - Perspective transformation - Environmental mapping - Object relationship analysis - Stepwise logical deduction </details> Figure 15: Showcases of reasoning generateed by post-trained UMMs in the real-world spatial reasoning task. We highlight hallucinations or incorrect reasoning steps in red. <details> <summary>x18.png Details</summary>  ### Visual Description ## Screenshot: Paper Folding Analysis Document ### Overview The image is a screenshot of a technical document analyzing a folded paper with cutouts. It includes diagrams illustrating the folding process, textual explanations of the model's reasoning, and code snippets for simulating the unfolding process. The document compares two model outputs: "Visual World Modeling" and "Verbal World Modeling," each with step-by-step explanations and diagrams. ### Components/Axes - **Diagrams**: - **Step 1**: A square paper with a vertical cutout (labeled `triangle_left` and `triangle_right`). - **Step 2**: A folded paper with mirrored cutouts. - **Step 3**: A partially unfolded paper with a horizontal crease. - **Step 4**: A fully unfolded paper with mirrored cutouts. - **Textual Explanations**: - Descriptions of folding steps, symmetry principles, and mirroring effects. - Code snippets simulating the unfolding process using Python `print` statements. - **Code Snippets**: - Variables like `triangle_down`, `triangle_right`, and `triangle_left` correspond to diagram labels. - Print statements output labels (e.g., `['triangle_left', 'triangle_right']`). ### Detailed Analysis #### Visual World Modeling (Left Side) 1. **Step 1**: The model identifies the original square and triangle cutouts. - Diagram: A square with a vertical cutout labeled `triangle_left` and `triangle_right`. - Text: "The original square and triangle simply remain in place." 2. **Step 2**: The model reverses the fourth and final fold (vertical fold). - Diagram: A folded paper with mirrored cutouts. - Text: "The left side of the paper was folded over the right." 3. **Step 3**: The model unfolds the third fold (horizontal fold). - Diagram: A partially unfolded paper with a horizontal crease. - Text: "The top half was folded down over the bottom half." 4. **Step 4**: The model combines reflections to reconstruct the original pattern. - Diagram: A fully unfolded paper with mirrored cutouts. - Text: "The square cutout is on the stationary left side." #### Verbal World Modeling (Right Side) 1. **Step 1**: The model identifies the original square and triangle cutouts. - Diagram: A square with a vertical cutout labeled `triangle_left` and `triangle_right`. - Text: "The square cutout is on the stationary left side." 2. **Step 2**: The model reverses the fourth and final fold (vertical fold). - Diagram: A folded paper with mirrored cutouts. - Text: "The left side of the paper was folded over the right." 3. **Step 3**: The model unfolds the third fold (horizontal fold). - Diagram: A partially unfolded paper with a horizontal crease. - Text: "The top half was folded down over the bottom half." 4. **Step 4**: The model combines reflections to reconstruct the original pattern. - Diagram: A fully unfolded paper with mirrored cutouts. - Text: "The square cutout is on the stationary left side." ### Key Observations - **Diagram Labels**: - `triangle_left` and `triangle_right` are consistently labeled across diagrams. - Numbers (e.g., `1`, `-1`) indicate mirrored or reflected positions. - **Code Logic**: - The code simulates unfolding by printing labels (e.g., `['triangle_left', 'triangle_right']`). - Variables like `triangle_down` and `triangle_right` map to diagram elements. - **Symmetry Principle**: - The model uses mirroring to reverse folds, ensuring cutouts are repositioned correctly. ### Interpretation The document demonstrates a method to reverse-engineer folded paper with cutouts by systematically unfolding each step. The "Visual World Modeling" approach uses diagrams to visualize the process, while the "Verbal World Modeling" approach translates this into code. The code snippets simulate the unfolding process by printing labels that correspond to the diagrams, ensuring the final pattern matches the original design. This method is critical for applications in computational geometry, origami design, and automated pattern reconstruction. **Note**: No numerical data or trends are present; the focus is on textual and diagrammatic reasoning. </details> Figure 16: Showcases of reasoning generated by post-trained UMMs in the paper folding task. We highlight hallucinations or incorrect reasoning steps in red, but also mark correctly generated visual unfolding intermediate steps with green borders. <details> <summary>x19.png Details</summary>  ### Visual Description ```markdown ## Cube 3-View Projection Puzzles ### Overview The image contains six cube projection puzzles labeled (a) to (f). Each puzzle presents a 3D cube stack with colored cubes (yellow, blue, purple, darkviolet, green) and asks how many cubes of a specific color are visible from a particular perspective (front, top, right). Each puzzle includes: - Isometric 3D cube views (front, top, left/right) - A question about visible cube counts - A model output explaining reasoning - A ground-truth answer --- ### Components/Axes #### Labels and Views 1. **Cube Views**: - **Front**: Front-left or front-right perspective - **Top**: Overhead view - **Left/Right**: Side view from left to right 2. **Color Legends**: - Colors (yellow, blue, purple, darkviolet, green) correspond to cube counts in model outputs. 3. **Question Structure**: - Format: "How many [color] cubes can possibly be seen from [view]?" - Options: A–D (e.g., "A. 2. B. All three...") #### Model Output Format - ` </details> Figure 17: Showcases of reasoning generated by post-trained UMMs in the paper folding task. We mark correct and incorrect generated cube views with green and red borders, respectively. For incorrect generations, the corresponding ground-truth views are provided for reference (note that these are shown only for readers and are never provided to the models during reasoning).