2604.24625v1

# Meta-CoT: Enhancing Granularity and Generalization in Image Editing ## Abstract Unified multi-modal understanding/generative models have shown improved image editing performance by incorporating fine-grained understanding into their Chain-of-Thought (CoT) process. However, a critical question remains underexplored: what forms of CoT and training strategy can jointly enhance both the understanding granularity and generalization? To address this, we propose Meta-CoT, a paradigm that performs a two-level decomposition of any single-image editing operation with two key properties: (1) Decomposability. We observe that any editing intention can be represented as a triplet — (task, target, required understanding ability). Inspired by this, Meta-CoT decomposes both the editing task and the target, generating task-specific CoT and traversing editing operations on all targets. This decomposition enhances the model’s understanding granularity of editing operations and guides it to learn each element of the triplet during training, substantially improving the editing capability. (2) Generalizability. In the second decomposition level, we further break down editing tasks into five fundamental meta-tasks. We find that training on these five meta-tasks, together with the other two elements of the triplet, is sufficient to achieve strong generalization across diverse, unseen editing tasks. To further align the model’s editing behavior with its CoT reasoning, we introduce the CoT-Editing Consistency Reward, which encourages more accurate and effective utilization of CoT information during editing. Experiments demonstrate that our method achieves an overall 15.8% improvement across 21 editing tasks, and generalizes effectively to unseen editing tasks when trained on only a small set of meta-tasks. Our code, benchmark, and model are released at here. footnotetext: Equal contribution. footnotetext: Corresponding author. <details> <summary>2604.24625v1/x1.png Details</summary>  ### Visual Description ## Diagram: Task Editing Workflow with Camera Motion and Object Adjustment ### Overview The diagram illustrates a multi-step process for editing images involving camera motion and object manipulation. It combines textual instructions with visual elements (e.g., a robot icon) to represent task decomposition, object adjustments, and data integration strategies. --- ### Components/Axes 1. **Task Summary** (Top Section): - **Labels**: "Step 1: Task Summary", "Determine task type" - **Text**: "The editing instruction is: Zoom out the camera view. This is a Camera Motion task." - **Color**: Light gray background. 2. **Task Thinking** (Middle Section): - **Labels**: "Step 2: Task Thinking", "Generate specific thinking processes for different tasks" - **Subcomponents**: - **New Subjects Appearing**: Red text, lists two stools with specific attributes. - **Old Subjects Disappearing**: Green text, describes size reduction and repositioning. - **Color**: White background with red/green text highlights. 3. **Target Editing Mode** (Bottom Section): - **Labels**: "Step 3: Target Editing Mode Traversal" - **Subcomponents**: - **Object Adjustments**: Green text listing edits for "backpack," "black rectangular object," "glass vase," etc. - **Spatial Modifications**: Instructions to reduce size, move upward/backward, or extend visible areas. - **Color**: White background with green text. 4. **Mix Train with Understanding Data** (Right Panel): - **Labels**: "Mix Train with Understanding Data" - **Subcomponents**: - **Captioning**: Light blue background. - **Reasoning**: Dark blue background. - **Color**: Vertical gradient from light to dark blue. 5. **Visual Elements**: - **Robot Icon**: Positioned at the bottom center, labeled "Thinking..." with a speech bubble containing "Zoom out the camera view view." - **Image Preview**: Shows a room with a countertop, cabinets, and floor. --- ### Detailed Analysis - **Task Summary**: Explicitly identifies the task as "Camera Motion" with a focus on zooming out the camera view. - **Task Thinking**: Introduces new objects (stools) and describes changes to existing objects (size reduction, repositioning). - **Target Editing Mode**: Provides granular instructions for adjusting object sizes and positions, emphasizing proportional scaling and spatial alignment. - **Mix Train Section**: Suggests combining captioning and reasoning data to enhance understanding. --- ### Key Observations 1. **Color Coding**: - Red highlights new objects (stools). - Green highlights disappearing objects and their adjustments. - Blue gradients represent data integration strategies. 2. **Robot Icon**: Symbolizes AI-driven task analysis, positioned to "think" about the camera motion instruction. 3. **Spatial Relationships**: - The robot is centered below the textual steps, linking abstract instructions to visual representation. - The image preview aligns with the described countertop and floor adjustments. --- ### Interpretation This diagram outlines a structured workflow for image editing, emphasizing: 1. **Task Decomposition**: Breaking down camera motion and object adjustments into discrete steps. 2. **Proportional Editing**: Ensuring objects maintain relative sizes and positions during edits. 3. **Data Fusion**: Combining captioning (descriptive data) and reasoning (logical adjustments) to improve AI understanding of spatial relationships. The robot icon serves as a metaphor for AI cognition, bridging textual instructions with visual execution. The use of color coding and spatial grounding ensures clarity in distinguishing task phases and object interactions. </details> Figure 1: Overview of Triplet Decomposition. Triplet Decomposition enables fine-grained reasoning over both the task and the target through three steps: (1) Task Summary, (2) Task Thinking, and (3) Target-wise Editing Traversal. This decomposition allows the model to optimize reasoning and editing along two elements: task and target. During training, we jointly incorporate diverse visual understanding tasks to capture all components of the (task, target, understanding capability) triplet, as illustrated in the bottom-right. ## 1 Introduction Recent studies show that Chain-of-Thought (CoT) can effectively enhance editing performance in unified generation/understanding models [11, 8]. By explicitly reasoning the editing process, the model activates its understanding ability and produces more accurate edits. We argue that an effective image editing CoT paradigm should possess two properties: (1) Stimulation of the model’s understanding ability. (2) Generalization across diverse editing tasks. Existing works mainly focus on the first property, such as introducing spatial localization cues or other explicit understanding information into the CoT [11]. However, such approaches often exhibit limited generality since CoTs tailored to specific understanding forms may not adapt well to broader editing tasks. For example, localization-based CoT performs poorly on tasks like style transfer or viewpoint transformation. Therefore, this paper aims to investigate a critical question: what forms of CoT and training strategy can simultaneously enhance understanding granularity and generalization capability during image editing? To address this, we first observe that any single-image editing operation can be defined by a triplet: (Task, Target, Required Understanding Ability). For example, the editing instruction “Change the number of puppies to three” involves the task type “Quantity Modification”, editing targets “puppies”, and requires the model to possess understanding abilities in localization and counting. Building on this, as shown in Figure 1, we propose the first decomposition level in the Meta-CoT, Triplet Decomposition, which guides fine-grained reasoning over both the editing task and the editing target. This paradigm comprises three steps: (1) Task Summary, (2) Task Thinking, and (3) Target-wise Editing Traversal. This design offers strong decomposability, enabling the model to optimize over task and target, thereby not only learning to comprehend diverse editing operations but also mastering how to apply distinct editing strategies to different entities. To capture the third element, required understanding ability, of the triplet, we incorporate data from a variety of visual understanding tasks during training, ensuring the model fully learns across all three elements. Table 1: Any single-image editing operation can be represented as a triplet of Task (first column), Editing Target (third column), and Required Understanding Capability (fourth column). Each operation can be further decomposed into combinations of Meta-Tasks. We summarize five generic meta-tasks (top) and list their possible compositional combinations for different editing tasks (second column). | Task Addition Deletion | Meta-Task Addition Deletion | Target Any Any | Understanding Localization Localization | | --- | --- | --- | --- | | Replacement | Replacement | Any | Localization | | Camera Motion | Camera Motion | Any | Spatial Understanding | | Position Change | Position Change | Any | Spatial Understanding | | Style Transfer | Replacement | Style | Style Understanding | | Tone Adjustment | Replacement | Tone / Lighting | Tone Understanding | | Text Editing | Addition, Deletion, Replacement | Text | Image-text Understanding / OCR | | Shape Modification | Replacement | Shape | Shape Understanding | | Structural Change | Addition, Deletion, Replacement, Camera Movement | Artificial Structure | Structural Understanding | | Color Change | Replacement | Color | Color Understanding | | Quantity Change | Addition, Deletion | Any | Counting | | Specified Quantity Change | Any | Any | Counting, Localization | | Human Attribute Editing | Addition, Deletion, Replacement | Human | Human Attribute Understanding | | Material Change | Replacement | Material | Material Understanding | | Motion Change | Replacement | Motion | Motion Understanding | | Logical Reasoning | Addition, Deletion, Replacement | Shape, Symbol, Text | Symbolic / Mathematical Reasoning | | Causal Reasoning | Addition, Deletion, Replacement | Any | Causal Reasoning | | Spatial Composition | Addition, Deletion, Position Change, Camera Movement | Any | Spatial Reasoning | | Temporal Reasoning | Addition, Deletion, Replacement | Any | Temporal Reasoning | | Multi-Instruction Editing | Any | Any | Multi-instruction Understanding | Furthermore, to achieve generalization, we analyze single-image editing tasks and identify a set of primitive, universal operations, termed “meta-tasks”. As shown in Table 1 and Figure 2, analogous to bases in a vector space, these meta-tasks (e.g., add, remove, replace) form a minimal set capable of spanning the entire editing operation space. We then propose the second decomposition level in Meta-CoT: Meta-task Decomposition. Specifically, we redefine the triplet element “Task” as one or more meta-tasks, and adapt the CoT paradigm by replacing “Task Summary” with “Meta-task Summary”. For example, “Quantity Change” corresponds to the meta-tasks add and remove. The model learns to decompose instructions into simple meta-tasks, forming a reasoning chain of fundamental operations. This paradigm yields strong generalizability: by training on only a small number of meta-tasks (as few as five), the model can generalize to all other editing tasks via compositional reasoning. Thus, during training, we no longer need to cover every task individually; all remaining tasks can be expressed as combinations of these meta-tasks. We also incorporate diverse visual understanding tasks into the training data to ensure mastery of all triplet elements. <details> <summary>2604.24625v1/x2.png Details</summary>  ### Visual Description ## Diagram: Task Decomposition for Image Editing ### Overview The image illustrates a structured workflow for decomposing user instructions into specific image editing tasks. It demonstrates how textual prompts are translated into actionable steps for modifying visual content, with examples spanning arithmetic correction, aging simulation, and interior visualization. ### Components/Axes - **Columns**: 1. **Input Image**: Original visual content (e.g., chalkboard, person, car). 2. **Instruction**: User-provided textual prompt (e.g., "Please complete the answers to the questions"). 3. **Meta-Task Decomposition**: Breakdown of the instruction into editing steps (e.g., "Replace the question mark in the image with '2'"). 4. **Edited Image**: Result of applying the decomposition steps. - **Rows**: Three examples, each demonstrating a distinct task: - Arithmetic correction (1+1=? → 1+1=2). - Aging simulation (young man → elderly man with wrinkles and white hair). - Interior visualization (car exterior → car interior with steering wheel, seats, dashboard). ### Detailed Analysis #### Example 1: Arithmetic Correction - **Input Image**: Chalkboard with "1+1=?". - **Instruction**: "Please complete the answers to the questions". - **Decomposition**: "Replace the question mark in the image with '2'". - **Edited Image**: Chalkboard with "1+1=2". #### Example 2: Aging Simulation - **Input Image**: Young man with black hair. - **Instruction**: "What will he look like when he gets old?". - **Decomposition**: - Replace black hair with white hair. - Add wrinkles and age spots to the face. - Add wrinkles to the hands. - **Edited Image**: Elderly man with white hair, facial wrinkles, and aged hands. #### Example 3: Interior Visualization - **Input Image**: Exterior of a black SUV. - **Instruction**: "What does the interior of this car look like?". - **Decomposition**: - Move the viewpoint inside. - Add components: steering wheel, seats, dashboard display. - **Edited Image**: Car interior with visible steering wheel, seats, and dashboard. ### Key Observations 1. **Actionable Verbs**: Decomposition steps use imperative verbs (e.g., "Replace," "Add," "Move") to specify edits. 2. **Granularity**: Tasks are broken into fine-grained actions (e.g., adding wrinkles to specific body parts). 3. **Consistency**: Edited images directly reflect the decomposition steps without ambiguity. 4. **Versatility**: The framework applies to diverse domains (math, biology, automotive design). ### Interpretation This diagram exemplifies a systematic approach to image editing guided by natural language instructions. By decomposing high-level prompts into discrete, executable steps, the workflow bridges the gap between abstract user intent and precise visual manipulation. The examples highlight its adaptability across tasks, from simple arithmetic to complex scene reconstruction. The structured decomposition ensures reproducibility and clarity, critical for automated image editing systems or human-AI collaboration tools. </details> Figure 2: Examples of editing intent decomposed into meta-tasks. We further observe the inconsistency between the CoT reasoning and the final edited result in existing models. To address this, we propose the CoT–Editing Consistency Reward. Specifically, we employ a VLM (e.g., Qwen2.5-VL [2]) to assess the consistency between the CoT and the edited image from both the task and object perspectives, and provide corresponding rewards. Using this reward, we perform Flow-GRPO [37] and effectively improve the alignment between the CoT reasoning and editing outcomes. To validate our approach, we construct a benchmark covering 21 distinct editing tasks, including Gedit-Bench [38], RiseBench [84], ComplexEdit [73], and five additional benchmarks we developed to fill missing task types. Compared to existing benchmarks, our benchmark offers broader task coverage, enabling a more thorough evaluation. The benchmark data is distributionally independent from training data, ensuring fair evaluation. We also conduct experiments on ImgEdit [75] to demonstrate the effectiveness of our method. We conduct our experiments based on Bagel [8], a mainstream open-source unified model that supports CoT–based editing, making it well-suited for our comparative study. After applying Meta-CoT via SFT and the CoT–Editing Consistency Reward via GRPO optimization, our method achieves a 15.7% overall performance improvement across all 21 tasks compared to the base model. Moreover, experiments demonstrate that training on only a few meta-tasks is sufficient to achieve strong generalization to other editing tasks, yielding performance comparable to that achieved by training on the full set of task types. The contributions of this paper can be summarized as: - We propose Triplet Decomposition, which breaks down editing tasks into task, target, and understanding ability, enhancing the granularity during the reasoning process. - We introduce Meta-task Decomposition, which reduces tasks to fundamental meta-tasks, enabling strong generalization through the meta-task-based training strategy. - We design a CoT-Editing Consistency Reward to mitigate mismatches between CoT reasoning and editing results. - We construct a comprehensive image editing benchmark which covers a broader range of editing task types and empirically validate the effectiveness of our approach. ## 2 Related Work ### 2.1 CoT in Understanding/Generation Models Chain-of-Thought (CoT) reasoning, originally introduced to elicit step-by-step logical inference in large language models, has recently been extended to vision-language models (VLMs) to enhance multimodal reasoning [82, 59, 58, 20, 40]. Early efforts integrated visual information into textual CoT [82, 40], while subsequent work developed native multimodal CoT by interleaving textual reasoning with visual tokens [14, 63, 19, 50, 12, 29]. CoT has also been applied to image generation [21, 15, 10, 23] and editing [8, 11]. In image editing, several works have incorporated CoT to improve instruction following and spatial reasoning. For instance, Bagel [8] and GoT [11] generate intermediate reasoning steps before editing images, demonstrating improved consistency between visual understanding and generation. However, current CoT paradigms for editing are either too generic to stimulate the model’s understanding capacity [8] or too specialized to adapt across diverse tasks [11], motivating us to develop a paradigm that enhances both the understanding granularity and generalization in image editing. ### 2.2 Unified Understanding and Generation Models Recent research has increasingly focused on unifying multimodal understanding [1, 2, 79, 78] and generation [47, 80, 86, 30, 31, 42, 43, 68] within single models for joint understanding and synthesis [53, 54, 61, 66, 4, 33, 67, 62, 8, 22, 69, 7, 16, 36, 35, 87, 72, 71, 17]. GPT‑4o [45] exemplifies this by fusing visual analysis and generation, outperforming earlier unified models. Current unified models can be organized into three categories: (1) Autoregressive approaches that apply next-token prediction to jointly generate textual and visual tokens [61, 6, 39, 48, 54, 57]; (2) Additional Diffusion frameworks that couple a pre-trained LLM backbone with an external diffusion module, where the language model derives semantic conditions to guide the diffusion process [9, 64, 46, 56]; and (3) Unified Integrated Transformer models that natively combine both LLM and diffusion mechanisms within a shared transformer architecture [8, 32, 41, 52, 85]. We build our method upon Bagel [8], a unified multimodal transformer with intrinsic CoT-based editing capabilities, making it a suitable foundation for CoT paradigms exploration in image editing. ## 3 Method ### 3.1 Theoretical Definition Triplet Decomposition. Let $T$ denote the original CoT space, and let $T_1$ , $T_2$ , and $T_3$ denote the task, target, and required understanding capability. Then the triplet space is $S_triplet=T_1× T_2× T_3$ . We further let $H=\log|space|$ denote space complexity (i.e., entropy). We prove: $H(T_1,T_2,T_3)=\log|S_triplet|<\log|T|=H(T)$ , indicating triplet decomposition lowers the editing complexity. Next, we use the mutual information per entropy $G=\frac{I(T;X_tgt)}{H(T)}$ [55] to denote the understanding granularity in CoT, where $X_tgt$ is the target image. We prove: $G(T_1,T_2,T_3)>G(T)$ , indicating Meta-CoT achieves higher understanding granularity than classical CoT (noted as $T$ ). See supplementary material for detailed derivation. Meta-task Decomposition. $B=\{t_1,…,t_n\}$ is a set of meta-tasks. We say $B$ forms a basis of the task space $T$ if $$ ∀ T∈T, ∃ t_i_1,…,t_i_{k}∈B s.t. T=t_i_1∘ t_i_2∘⋯∘ t_i_{k}. $$ ### 3.2 Triplet Decomposition As shown in Table 1, any single-image editing operation can be decomposed into a triplet comprising three fundamental elements: the editing task, the editing target, and the understanding capability required to accomplish the operation. Building on this insight, we propose the first decomposition level, Triplet Decomposition (illustrated in Figure 1), in Meta-CoT. This paradigm decomposes an editing instruction into task and target, guiding the model to explicitly learn to understand different editing tasks and master editing methods for various targets during training. To accommodate the diverse understanding capabilities required for different editing tasks, we incorporate a diverse set of visual understanding tasks during training, ensuring the comprehensive mastery of all three elements of the defined triplet. As shown in Figure 1, our Triplet Decomposition unfolds in three steps. (1) Task Summary. The model infers the task type inductively from the instruction. (2) Task Thinking. The model generates a task-specific reasoning process based on the task type. For instance, for style transfer, it analyzes the visual attributes of the target style; for camera motion, it identifies object appearance or disappearance; and for logical reasoning-based editing, it deduces the implicit operations suggested by the instruction (see the supplementary materials for more examples). (3) Target Editing Mode Traversal. The model traverses all targets in the image and reasons about whether and how each should be edited. This step ensures spatial and semantic consistency and provides fine-grained, interpretable editing guidance. ### 3.3 Meta-task Decomposition Furthermore, as shown in Table 1, we identify a set of generic operations within single-image editing, referred to as “meta-tasks”. Meta-tasks serve as a set of bases in the single-image editing operation space, capable of combining and generalizing to produce various complex editing operations. In the ideal case, training on these meta-tasks enables the model to handle other, more complex editing tasks. Building on this insight, we propose the second decomposition level, Meta-task Decomposition, in Meta-CoT. In practice, we define five distinct meta-tasks (listed in Table 1). Accordingly, our triplet evolves into (meta-task, target, required understanding capability). Next, we replace the first step of Meta-CoT, Task Summary, with Meta-task Summary, decomposing the instruction into a combination of basic meta-tasks. For example, the style transfer task can be decomposed into a “replacement” operation on the style attribute of the editing target. During training, as noted in Section 3.2, we supervise all triplet elements by training on data of the five meta-tasks and incorporating diverse visual understanding tasks to achieve comprehensive mastery. ### 3.4 CoT-Editing Consistency Reward In our experiments, we observed that in certain editing scenarios, particularly when the editing instruction does not explicitly specify the operation, the model may fail to follow the reasoning outlined in the CoT, even if the correct editing operation is inferred. This misalignment between CoT reasoning and execution often results in incorrect editing. To address this, we introduce the CoT–Editing Consistency (CEC) Reward. Specifically, we design a consistency metric from both the task and target perspectives, where a VLM (Qwen2.5-VL [2]) evaluates whether the generated edit aligns with the CoT reasoning in terms of both operation and target, producing a score from 0 to 10. Before training, we conduct a correlation validation for the CEC Reward. Specifically, we use the Meta-CoT SFT model to generate 500 editing samples with CoT. Four annotators then score the CoT–editing consistency. For samples with scores ranging $<3$ , we average all scores. For those with a range $≥ 3$ , we average the three closest scores. We then iteratively adjust the VLM’s initial prompt, computing the Pearson correlation $r$ and mean absolute error $ε_MAE$ against human annotations on the 500 samples, until $r≥ 0.8$ and $ε_MAE≤ 2.5$ [27, 74, 25]. We provide the details for the CEC Reward in the supplementary material. <details> <summary>2604.24625v1/x3.png Details</summary>  ### Visual Description ## Diagram: Machine Learning Model Architecture for Image Generation ### Overview This diagram illustrates a two-stage machine learning model architecture for image generation, featuring components like Meta-CoT, MLLM, Und. Expert, Gen. Expert, and encoders. The flow progresses from an Input Image through text tokens, encoders, and experts to produce a Target Image. Key elements include trainable/frozen components, random noise, and a CEC Reward metric. ### Components/Axes 1. **Legend**: - 🔥 **Trainable**: Components actively trained during specific stages. - ❄️ **Frozen**: Components locked after training completion. - **CEC Reward = 9.0**: Performance metric (likely accuracy or efficiency). 2. **Key Blocks**: - **Meta-CoT**: Contains 8 text token slots (blue squares). - **MLLM**: Frozen in Stage II (top block with snowflake). - **Target Image**: Output of the model (green squares). - **Und. Expert**: Trainable in Stage I, frozen in Stage II (blue block with fire). - **Gen. Expert**: Trainable in Stage I, frozen in Stage II (green block with fire). - **Text Token**: Input instruction (blue square). - **Und Encoder**: Processes input image (blue block). - **Gen Encoder**: Generates random noise (green block with striped squares). 3. **Flow Arrows**: - Red arrows indicate progression between stages (Stage I → Stage II). - Dashed red lines outline the entire architecture. ### Detailed Analysis - **Stage I**: - Input Image → Und Encoder → Und. Expert (trainable). - Text Token → Meta-CoT (trainable). - Gen Encoder → Random Noise → Gen. Expert (trainable). - CEC Reward = 9.0 achieved during training. - **Stage II**: - MLLM and Und. Expert are frozen. - Gen. Expert remains frozen after Stage I. - Target Image is generated via frozen components. ### Key Observations 1. **Freezing Strategy**: Critical components (MLLM, Und. Expert, Gen. Expert) are frozen after Stage I, suggesting a focus on stability in later stages. 2. **Noise Integration**: Random Noise is injected via Gen Encoder, indicating a generative adversarial or diffusion-based approach. 3. **Text-Image Alignment**: Meta-CoT processes text tokens, linking instructions to image generation. 4. **Performance Metric**: CEC Reward = 9.0 (high value, but scale unspecified). ### Interpretation This architecture prioritizes modular training, freezing components after initial stages to prevent overfitting or catastrophic forgetting. The CEC Reward of 9.0 suggests strong performance, though the metric's definition (e.g., accuracy, FID score) is unclear. The use of random noise and text tokens implies a hybrid model combining generative and instruction-following capabilities. The freezing strategy may enable efficient fine-tuning or deployment in resource-constrained environments. </details> Figure 3: Training pipeline (self attention omitted). (1) Stage 1: SFT on both reasoning and editing. (2) Stage 2: GRPO on editing. We adopt Flow-GRPO [37, 51] to optimize the model with the CEC Reward. Since the CEC Reward measures semantic alignment, we only focus the optimization on early denoising timesteps, where semantic fidelity is most critical, and omit updates on later timesteps. Empirically, we find that reducing optimization on later timesteps also alleviates potential noise artifacts introduced by Flow-GRPO. ### 3.5 Training Pipeline As shown in Figure 3, our training includes two stages. In the SFT stage, both the understanding expert and generation expert are tuned to train CoT reasoning and image editing, with the image understanding encoder also updated. In the subsequent RL stage, we freeze the image understanding encoder and train only the generation expert. This is motivated by two observations: (1) after SFT, the model already achieves highly accurate CoTs that should be preserved; (2) training both modules during RL causes unstable optimization and degrades the reasoning ability learned in SFT. <details> <summary>2604.24625v1/x4.png Details</summary>  ### Visual Description ##Flowchart: CoT-Editing Consistency Process with Gemini ### Overview This flowchart illustrates a technical workflow for analyzing and editing camera motion tasks using Gemini and Qwen2.5 models. It demonstrates how visual changes (e.g., object additions/removals) are parsed, contextualized, and summarized through iterative prompting and consistency checks. ### Components/Axes 1. **Left Panel (Input/Instruction)**: - Two sequential images: - *Top Image*: Kitchen counter with backpack and glass - *Bottom Image*: Same scene with two added stools - Instructions: - "Zoom out the view" - "Check CoT-Editing Consistency with Gemini" - Task labels: - "Task: Camera Movement" 2. **Middle Panel (Processing Flow)**: - Three interconnected boxes: - *Top Box*: "In-context Prompting" (Qwen2.5-VL logo) - *Middle Box*: "In-context Prompting" (Qwen2.5 logo) - *Bottom Box*: "In-context Prompting" (Qwen2.5 logo) - Arrows indicate sequential processing flow 3. **Right Panel (Output/Analysis)**: - *Meta-CoT Section*: - **(Meta-)Task Summary**: "This is a Camera Motion task" - **Task Thinking**: - New subjects: "Two stools: Add two stools under the countertop..." - Old subjects: "No old subjects disappear completely, but all..." - **Subject Editing Mode Traversal**: - Backpack: "Reduce its size proportionally..." - Black rectangular object: "Reduce its size and..." - Glass vase: "Reduce its size proportionally and..." ### Detailed Analysis 1. **Input Sequence**: - Initial state: Backpack and glass on countertop - Modified state: Two stools added under countertop - Both states labeled as "Camera Movement" tasks 2. **Processing Flow**: - Visual changes trigger Qwen2.5 model prompting - Three-stage prompting process: 1. Initial context establishment (Qwen2.5-VL) 2. Intermediate analysis (Qwen2.5) 3. Final contextualization (Qwen2.5) 3. **Output Analysis**: - Task classification: Camera motion task - Subject changes: - Addition: Two stools (new subjects) - Persistence: Original objects remain visible - Editing constraints: - Proportional size reduction for all objects - Spatial relationships maintained ### Key Observations 1. **Consistency Framework**: - Gemini checks ensure CoT-Editing alignment - Qwen2.5 handles incremental contextual updates 2. **Object Relationships**: - New objects (stools) don't fully occlude original subjects - All objects undergo proportional size reduction 3. **Temporal Dynamics**: - Two distinct states processed sequentially - Editing mode traversal maintains object hierarchy ### Interpretation This workflow demonstrates a hybrid approach combining Gemini's consistency verification with Qwen2.5's contextual prompting for dynamic scene editing. The process maintains spatial relationships while handling object additions/removals through proportional scaling. The dual-image comparison suggests a focus on relative motion analysis rather than absolute positioning. The "Subject Editing Mode Traversal" indicates a prioritization of object hierarchy preservation during edits, with the backpack (likely foreground object) receiving explicit size reduction instructions. The persistent visibility of original objects despite new additions implies a transparency-aware editing paradigm. </details> Figure 4: Meta-CoT Data Construction. This pipeline processes the source image, target image, and instruction to the Meta-CoT. ### 3.6 Meta-CoT Data Creation Pipeline As shown in Figure 4, for each editing sample, we first determine its editing task type with Qwen2.5 [70], based on carefully designed task definition rules and the instruction, followed by a consistency check between the predicted task type and instruction with Gemini-2.5-Flash. Next, we input the source image, target image, instruction, and task type into Qwen2.5-VL [2], which, guided by a carefully designed prompt, generates the (Meta-)Task Summary, Task Thinking, and Target Editing Mode Traversal. This process also includes an evaluation to verify the alignment between the generated Meta-CoT and the actual editing process. ## 4 Experiment Table 2: Comparison of Overall Scores on the 21-task benchmark. All metrics are evaluated using GPT-4.1. Train Editing Only denotes the setting trained with the same parameters and editing data as our method, but without Meta-CoT. | Bagel(w/o think) Bagel(w think) Train Editing Only | 6.172 6.686 6.743 | 6.668 6.469 6.537 | 6.024 5.969 6.052 | 4.182 3.997 4.155 | 4.933 3.771 4.257 | 6.760 6.156 6.571 | 7.244 7.338 7.363 | 6.319 6.272 6.296 | 6.774 6.430 6.687 | 5.170 2.124 2.651 | 6.323 6.511 6.547 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | SFT(Meta-CoT) | 7.173 | 7.201 | 6.493 | 4.470 | 4.760 | 6.965 | 7.640 | 7.833 | 6.931 | 3.271 | 7.236 | | gray!20 Meta-CoT + RL (Ours) | 7.251 | 7.323 | 6.636 | 4.574 | 4.956 | 7.035 | 7.762 | 8.129 | 7.065 | 3.328 | 7.318 | | Method / Task | Causal | Logical | Spatial Reasoning | Temporal | Camera | Structure | Position | Quantity | Specified Quantity | Multi- Instruction | Average | | Bagel(w/o think) | 4.628 | 2.994 | 4.706 | 3.982 | 6.364 | 6.492 | 5.838 | 5.581 | 5.177 | 6.799 | 5.673 | | Bagel(w think) | 5.561 | 3.146 | 4.718 | 5.477 | 5.572 | 5.188 | 4.549 | 4.530 | 4.909 | 6.066 | 5.307 | | Train Editing Only | 5.710 | 3.217 | 4.683 | 5.663 | 5.964 | 5.629 | 5.177 | 4.957 | 5.093 | 6.349 | 5.538 | | SFT(Meta-CoT) | 6.647 | 3.526 | 4.858 | 6.233 | 6.876 | 7.309 | 5.862 | 6.468 | 6.019 | 6.936 | 6.224 | | gray!20 Meta-CoT + RL (Ours) | 6.953 | 4.014 | 5.046 | 6.376 | 7.100 | 7.557 | 6.185 | 6.712 | 6.322 | 7.077 | 6.415 | Table 3: System comparison on ImgEdit. All metrics are evaluated by GPT-4.1. Overall denotes average score across all tasks. | Method Closed-source Models FLUX.1 Kontext [Pro] [26] | Add 4.25 | Adjust 4.15 | Extract 2.35 | Replace 4.56 | Remove 3.57 | Background 4.26 | Style 4.57 | Hybrid 3.68 | Action 4.63 | Overall $↑$ 4.00 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | GPT Image 1 [High] [44] | 4.61 | 4.33 | 2.90 | 4.35 | 3.66 | 4.57 | 4.93 | 3.96 | 4.89 | 4.20 | | Multi-round Editing Models | | | | | | | | | | | | IEAP [18] | 2.34 | 3.08 | 2.03 | 3.62 | 3.45 | 1.81 | 3.19 | 2.68 | 2.76 | 2.77 | | Diffusion Only Models | | | | | | | | | | | | MagicBrush [77] | 2.84 | 1.58 | 1.51 | 1.97 | 1.58 | 1.75 | 2.38 | 1.62 | 1.22 | 1.90 | | Instruct-Pix2Pix [3] | 2.45 | 1.83 | 1.44 | 2.01 | 1.50 | 1.44 | 3.55 | 1.20 | 1.46 | 1.88 | | AnyEdit [76] | 3.18 | 2.95 | 1.88 | 2.47 | 2.23 | 2.24 | 2.85 | 1.56 | 2.65 | 2.45 | | UltraEdit [83] | 3.44 | 2.81 | 2.13 | 2.96 | 1.45 | 2.83 | 3.76 | 1.91 | 2.98 | 2.70 | | OmniGen [65] | 3.47 | 3.04 | 1.71 | 2.94 | 2.43 | 3.21 | 4.19 | 2.24 | 3.38 | 2.96 | | ICEdit [81] | 3.58 | 3.39 | 1.73 | 3.15 | 2.93 | 3.08 | 3.84 | 2.04 | 3.68 | 3.05 | | Step1X-Edit [38] | 3.88 | 3.14 | 1.76 | 3.40 | 2.41 | 3.16 | 4.63 | 2.64 | 2.52 | 3.06 | | Qwen-Image [60] | 4.38 | 4.16 | 3.43 | 4.66 | 4.14 | 4.38 | 4.81 | 3.82 | 4.69 | 4.27 | | Unified Models | | | | | | | | | | | | GoT [11] | 3.61 | 2.94 | 1.35 | 2.78 | 2.57 | 2.29 | 3.51 | 1.75 | 2.66 | 2.61 | | Ming-UniVision [22] | 3.55 | 3.14 | 1.52 | 3.25 | 3.29 | 2.77 | 3.99 | 2.74 | 3.91 | 3.06 | | BAGEL(w/o think) [8] | 3.56 | 3.31 | 1.70 | 3.38 | 2.62 | 3.24 | 4.49 | 2.38 | 4.17 | 3.20 | | UniWorld-V1 [34] | 3.82 | 3.64 | 2.27 | 3.47 | 3.24 | 2.99 | 4.21 | 2.96 | 2.74 | 3.26 | | BAGEL(think) [8] | 3.65 | 3.53 | 2.03 | 3.60 | 3.03 | 3.45 | 4.43 | 2.59 | 4.22 | 3.39 | | OmniGen2 [62] | 3.57 | 3.06 | 1.77 | 3.74 | 3.20 | 3.57 | 4.81 | 2.52 | 4.68 | 3.44 | | BLIP3o-NEXT [5] | 4.00 | 3.78 | 2.39 | 4.05 | 2.61 | 4.30 | 4.64 | 2.67 | 4.13 | 3.62 | | gray!20 Meta-CoT+RL(Ours) | 3.87 | 3.91 | 2.40 | 4.22 | 3.74 | 3.98 | 4.80 | 3.26 | 4.33 | 3.83 | | gray!20 $Δ$ Over Base Model | +6.0% | +10.8% | +18.2% | +17.2% | +23.4% | +15.4% | +8.4% | +25.9% | +2.6% | +13.0% | ### 4.1 Implementation Details Benchmark and Metrics. To comprehensively evaluate our model’s performance across diverse editing tasks, we construct a benchmark comprising 21 editing tasks. Among them, 11 categories are inherited from and fully overlap with GEdit-Bench [38], and 4 logic-related categories are fully sourced from RiseBench [84]. The multi-instruction editing task is fully drawn from ComplexEdit [73]. We additionally introduce 5 new task categories (each with 100 samples) built from data entirely independent of the training set. Following GEdit-Bench, we adopt the Overall Score from VIEScore [25], which jointly measures instruction following, subject consistency, naturalness, and artifacts (ranging from 0 to 10) as our evaluation metric. Following [60, 38, 34, 8], all metrics are evaluated using GPT-4.1. We also evaluate our method on ImgEdit [75], which encompasses nine representative editing tasks covering diverse editing categories, with a total of 734 real-world test cases. The evaluation metrics include instruction adherence, image editing quality, and detail preservation, each scored from 1 to 5, with all scores assessed by GPT-4.1. Training Details During the SFT stage, we train 10k steps on 48 GPUs using a 1.5M image–instruction–CoT dataset built from open-source data [49, 24]. Image-instruction pairs are created by (1) instructions generation with Gemini-2.5-Flash under our defined edit taxonomy, (2) image editing with [26, 60, 44], and (3) filtering with both VLM (Gemini-2.5-Flash, GPT-4.1) and human evaluation. The creation of Meta-CoTs follows Section 3.6. The joint 100k understanding data source from LLaVA-OV [28] and Mammoth-VL [13]. During the RL stage, we train for 500 steps on an additional 20K editing dataset using 32 GPUs. More details are provided in the supplementary material. Table 4: Comparison of the four components that form the Overall Score in VIEScore across the 21-task benchmark. | Bagel(w/o think) | 6.76 | 7.73 | 7.01 | 7.94 | | --- | --- | --- | --- | --- | | Bagel(w think) | 6.30 | 8.44 | 6.98 | 7.71 | | Train Editing Only | 6.61 | 8.22 | 7.18 | 8.06 | | SFT(Meta-CoT) | 7.23 | 8.53 | 7.26 | 8.25 | | gray!20SFT + RL (Ours) | 7.44 | 8.53 | 7.31 | 8.34 | ### 4.2 Quantitative Evaluation As shown in Table 2 and Table 3, our method achieves notable improvements in the overall editing score, which considers instruction following, consistency, and visual quality. Compared to Bagel (no-think), it achieves +13.1% on the 21-task benchmark and +19.7% on ImgEdit. Relative to Bagel (think), gains reach 20.1% and 13.0%, respectively. To isolate the contribution of the Meta-CoT paradigm, we also train a variant using identical data and optimized with the same parameters, excluding the training of Meta-CoT. As shown in Table 2, our method outperforms this setting by 15.8%, validating the effectiveness of Meta-CoT. The RL stage also further enhances alignment and stability. Table 4 provides a breakdown across the four dimensions of VIEScore, Instruction Following, Subject Consistency, Naturalness, and Artifacts, where our method consistently outperforms the baselines, with the largest improvement in Instruction Following. This suggests that Meta-CoT reasoning enhances semantic understanding of both editing operations and targets, leading to more instruction-faithful edits. At the per-task level (as shown in Table 2), our method improves performance on all tasks except text editing. We observe that the reasoning process tends to hinder text editing, likely because the extensive textual reasoning interferes with identifying the correct text to modify. Developing mechanisms to preserve accurate text perception during reasoning remains a promising direction for future work. Table 5: Ablation study on (1) the number of meta-tasks defined and tasks trained, and (2) the Task Thinking in Meta-CoT. (n meta) denotes defining n meta-tasks and training only on them. (5 meta, full-task) indicates defining 5 meta-tasks, training on full tasks, and decomposing each task’s data into meta-tasks. | Method | Ins. | Con. | Nat. | Art. | | --- | --- | --- | --- | --- | | Train Editing Only | 6.61 | 8.22 | 7.18 | 8.06 | | SFT(3 meta) | 6.75 | 8.33 | 7.15 | 7.86 | | SFT(4 meta) | 6.93 | 8.44 | 7.17 | 7.94 | | SFT(5 meta) | 7.09 | 8.48 | 7.20 | 8.10 | | SFT(6 meta) | 7.13 | 8.51 | 7.22 | 8.07 | | SFT(5 meta, full-task) | 7.20 | 8.49 | 7.23 | 8.12 | | SFT(w/o task think) | 6.98 | 8.35 | 7.19 | 8.07 | | gray!20SFT(Meta-CoT) | 7.23 | 8.53 | 7.26 | 8.25 | | gray!20SFT + RL(Ours) | 7.44 | 8.53 | 7.31 | 8.34 | Table 6: Ablation study on the amount of visual understanding data mixed during training. w/o und. denotes training without mixing visual understanding data. CoT measures the completeness and accuracy of Task Thinking and Target Editing Traversal. | SFT(w/o und.) SFT(1k und.) gray!20SFT(Meta-CoT) | 6.74 6.92 7.23 | 8.28 8.31 8.53 | 7.18 7.20 7.26 | 8.11 8.16 8.25 | 7.56 7.81 8.89 | | --- | --- | --- | --- | --- | --- | <details> <summary>2604.24625v1/x5.png Details</summary>  ### Visual Description ## Image Editing Method Comparison: Bagel vs. Train Edit Only vs. Ours ### Overview The image presents a side-by-side comparison of three image editing methods ("Bagel," "Train Edit Only," and "Ours") across 12 distinct editing tasks. Each task includes: 1. An input image 2. Three edited versions (one per method) 3. A textual description of the editing task and results ### Components/Axes - **Categories**: 12 editing tasks organized in two columns (left and right) - **Methods**: - Bagel (blue) - Train Edit Only (orange) - Ours (green) - **Textual Descriptions**: Accompanying each task, explaining: - Editing goal - Method-specific results - Visual characteristics of edits ### Detailed Analysis #### Left Column Tasks 1. **Material** - Task: Build horse using red bricks - Bagel: Horse with red brick texture but distorted proportions - Train Edit Only: Horse with red brick pattern but unnatural coloration - Ours: Realistic horse with accurate red brick texture and natural proportions 2. **Replace** - Task: Replace guitar with Coca-Cola can - Bagel: Can appears but with incorrect proportions - Train Edit Only: Can partially visible with color bleeding - Ours: Perfectly integrated can with realistic lighting and shadows 3. **Action** - Task: Change person's movement to look forward - Bagel: Person facing forward but with motion blur artifacts - Train Edit Only: Person facing forward but with unnatural pose - Ours: Natural forward-facing pose with proper motion dynamics 4. **Color** - Task: Change domes to blue shades - Bagel: Uniform blue color without gradient - Train Edit Only: Partial blue coloring with color bleeding - Ours: Realistic blue gradient with proper shading 5. **Quantity** - Task: Add 3 clouds to sky - Bagel: 3 clouds but with unrealistic shapes - Train Edit Only: 3 clouds but with incorrect placement - Ours: 3 natural-looking clouds with proper atmospheric perspective 6. **Logical** - Task: Replace question mark with correct answer - Bagel: Incorrect mathematical symbol - Train Edit Only: Partially correct symbol - Ours: Accurate mathematical symbol with proper context #### Right Column Tasks 1. **Style** - Task: Convert to folded-paper origami style - Bagel: Basic origami pattern but with flat appearance - Train Edit Only: Origami style but with unnatural folds - Ours: Realistic origami with proper shadowing and depth 2. **Complex Instruction** - Task: Replace glove with golden artifact - Bagel: Basic golden object but with incorrect proportions - Train Edit Only: Golden object but with color inconsistencies - Ours: Realistic golden artifact with proper reflections and lighting 3. **Background Change** - Task: Replace background with white - Bagel: White background but with edge artifacts - Train Edit Only: White background but with partial transparency - Ours: Clean white background with proper edge handling 4. **Extract** - Task: Extract human figure with accessories - Bagel: Figure extracted but with background remnants - Train Edit Only: Figure extracted but with color artifacts - Ours: Clean extraction with proper edge handling 5. **Temporal** - Task: Predict appearance after 3 months - Bagel: Basic aging effects but with unrealistic details - Train Edit Only: Aging effects but with inconsistent textures - Ours: Realistic aging with proper texture evolution 6. **Spatial** - Task: Rotate top shoe 180 degrees - Bagel: Partial rotation with distortion - Train Edit Only: Rotation but with color bleeding - Ours: Perfect 180-degree rotation with proper perspective 7. **Viewpoint** - Task: Change camera view to side - Bagel: Side view but with perspective errors - Train Edit Only: Side view but with unnatural proportions - Ours: Accurate side view with proper perspective 8. **Structure** - Task: Open shoji screens to reveal view - Bagel: Partial opening with unrealistic shadows - Train Edit Only: Opening but with color inconsistencies - Ours: Realistic opening with proper lighting and reflections ### Key Observations 1. **Ours Method Superiority**: Consistently produces more realistic results across all tasks 2. **Complex Task Handling**: "Ours" excels in complex edits requiring multiple transformations 3. **Artifact Reduction**: "Ours" shows fewer artifacts compared to other methods 4. **Instruction Following**: "Ours" demonstrates better adherence to detailed instructions ### Interpretation This comparison demonstrates the effectiveness of the "Ours" method in handling diverse image editing tasks. The method shows superior performance in: - Maintaining realistic textures and lighting - Accurately following complex instructions - Reducing editing artifacts - Preserving spatial relationships The results suggest that "Ours" likely employs more advanced semantic understanding and context-aware editing capabilities compared to the other methods. The consistent performance across different task types (material changes, object replacement, perspective adjustments) indicates a robust editing framework capable of handling both simple and complex image transformations. </details> Figure 5: Qualitative results across diverse editing tasks, including conventional editing, reasoning-based editing, and multi-instruction editing (Zoom in to view). We present a partial visualization of the Meta-CoT reasoning process. Meta-CoT can decompose instructions, categorize them into specific tasks, generate reasoning based on the task characteristics, and accurately determine whether each target should be edited or not, ultimately achieving better editing results. Please see the supplementary materials for additional task examples. ### 4.3 Abaltion Study We further investigate three critical questions related to our method and present the results in Table 5 and Table 6. Can the meta-task training paradigm enable generalization to unseen editing tasks, and how many meta-tasks are needed? As shown in Table 5, we replace the first step of the CoT (“Task Summary”) with Meta-task Summary and conduct training under five distinct settings, each corresponding to different definitions of meta-tasks and the number of training task types. Starting from the basic 3-meta-tasks setting (add, delete, replace), we gradually increase the number of meta-tasks. The results show that, first, the model trained only on the five meta-tasks already achieves performance comparable to the full-data model on the 21-task benchmark and significantly outperforms the train-edit-only version. This demonstrates the strong generalization capability of the meta-task training strategy: training on a small set of meta-tasks while learning task decomposition suffices to generalize to unseen tasks. In other words, mastering universal meta-tasks and task decomposition reasoning enhances the model’s ability to generalize. Second, results show that our defined five meta-tasks strike a good balance between generalization and performance: defining fewer meta-tasks leads to a significant drop in instruction-following, while defining more meta-tasks provides little additional improvement across all tasks. Does the Task Thinking in Meta-CoT benefit the editing process? As shown in Table 5, we compare our method with a variant that removes the Task Thinking (the second step of Meta-CoT). Results show a significant drop in instruction-following performance, confirming that reasoning based on task characteristics is crucial for the editing. How does joint training with understanding data affect editing performance? In Table 6, we compare two reduced settings: (a) removing all understanding data and (b) using only 1K samples, against the default 100K. In addition to the four VIEScore components, GPT-4.1 also evaluates CoT quality in terms of the completeness and accuracy of Task Thinking and Target Editing Traversal in Meta-CoT. Results show that both reduced settings cause significant drops in editing performance, particularly in instruction following, as limited understanding data weakens the model’s comprehension of editing instructions. This is further corroborated by the notable decline in CoT quality, underscoring the necessity of balancing all three triplet elements during training and demonstrating that higher-quality Meta-CoT reasoning leads to better editing results. ### 4.4 Qualitative Evaluation As shown in Figure 5, we present comparisons across diverse editing tasks, including conventional, reasoning-based editing, and multi-instruction editing. Our method significantly improves instruction following, logical reasoning, and multi-instruction understanding compared with baseline methods. This demonstrates that our approach more effectively activates and leverages the model’s inherent understanding capability during the editing process. ## 5 Conclusion In this paper, we have investigated the problem of how to simultaneously enhance the understanding granularity and generalization capability of Chain-of-Thought (CoT)-guided image editing. To address this, we have presented Meta-CoT, which first employs the Triplet Decomposition to stimulate the model’s reasoning ability from both task and target perspectives. Furthermore, we have proposed the Meta-task Decomposition, which endows Meta-CoT with strong generalization capability across diverse editing scenarios. To align the CoT reasoning with the editing behavior, we have introduced the CoT-Editing Consistency Reward. Extensive experiments on our proposed 21-task benchmark and ImgEdit have demonstrated that our method not only significantly improves editing performance but also exhibits strong generalization to unseen editing tasks. #### Acknowledgments. This work was supported part by the Guangdong Natural Science Funds for Distinguished Young Scholar (No. 2025B1515020012). ## References - [1] J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou (2023) Qwen-vl: a versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966. Cited by: §2.2. - [2] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin (2025) Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: §1, §2.2, §3.4, §3.6. - [3] T. Brooks, A. Holynski, and A. A. Efros (2023) Instructpix2pix: learning to follow image editing instructions. In CVPR, pp. 18392–18402. Cited by: Table 3. - [4] J. Chen, Z. Xu, X. Pan, Y. Hu, C. Qin, T. Goldstein, L. Huang, T. Zhou, S. Xie, S. Savarese, et al. (2025) Blip3-o: a family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint arXiv:2505.09568. Cited by: §2.2. - [5] J. Chen, L. Xue, Z. Xu, X. Pan, S. Yang, C. Qin, A. Yan, H. Zhou, Z. Chen, L. Huang, et al. (2025) BLIP3o-next: next frontier of native image generation. arXiv preprint arXiv:2510.15857. Cited by: Table 3. - [6] X. Chen, C. Wu, Z. Wu, Y. Ma, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, C. Ruan, and P. Luo (2025) Janus-pro: unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811. Cited by: §2.2. - [7] W. Dai, Z. Zhao, Y. Zhong, Y. Cheng, J. Zhang, L. Wang, S. Zhang, Y. Lin, R. He, F. Song, et al. (2026) ChatUMM: robust context tracking for conversational interleaved generation. arXiv preprint arXiv:2602.06442. Cited by: §2.2. - [8] C. Deng, D. Zhu, K. Li, C. Gou, F. Li, Z. Wang, S. Zhong, W. Yu, X. Nie, Z. Song, et al. (2025) Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683. Cited by: §1, §1, §2.1, §2.2, §4.1, Table 3, Table 3. - [9] R. Dong, C. Han, Y. Peng, Z. Qi, Z. Ge, J. Yang, L. Zhao, J. Sun, H. Zhou, H. Wei, et al. (2024) Dreamllm: synergistic multimodal comprehension and creation. In ICLR, Cited by: §2.2. - [10] C. Duan, R. Fang, Y. Wang, K. Wang, L. Huang, X. Zeng, H. Li, and X. Liu (2025) Got-r1: unleashing reasoning capability of mllm for visual generation with reinforcement learning. arXiv preprint arXiv:2505.17022. Cited by: §2.1. - [11] R. Fang, C. Duan, K. Wang, L. Huang, H. Li, S. Yan, H. Tian, X. Zeng, R. Zhao, J. Dai, et al. (2025) Got: unleashing reasoning capability of multimodal large language model for visual generation and editing. arXiv preprint arXiv:2503.10639. Cited by: §1, §1, §2.1, Table 3. - [12] X. Fu, M. Liu, Z. Yang, J. Corring, Y. Lu, J. Yang, D. Roth, D. Florencio, and C. Zhang (2025) Refocus: visual editing as a chain of thought for structured image understanding. arXiv preprint arXiv:2501.05452. Cited by: §2.1. - [13] J. Guo, T. Zheng, Y. Bai, B. Li, Y. Wang, K. Zhu, Y. Li, G. Neubig, W. Chen, and X. Yue (2024) Mammoth-vl: eliciting multimodal reasoning with instruction tuning at scale. arXiv preprint arXiv:2412.05237. Cited by: §4.1. - [14] T. Gupta and A. Kembhavi (2023) Visual programming: compositional visual reasoning without training. In CVPR, pp. 14953–14962. Cited by: §2.1. - [15] F. Han, Y. Jiao, S. Chen, J. Xu, J. Chen, and Y. Jiang (2025) ControlThinker: unveiling latent semantics for controllable image generation through visual reasoning. arXiv preprint arXiv:2506.03596. Cited by: §2.1. - [16] R. He, Y. Cheng, T. Hang, Z. Li, Y. Xu, Z. Yin, S. Zhang, W. Dai, P. Du, A. Ma, et al. (2026) Re-align: structured reasoning-guided alignment for in-context image generation and editing. arXiv preprint arXiv:2601.05124. Cited by: §2.2. - [17] R. He, K. Ma, L. Huang, S. Huang, J. Gao, X. Wei, J. Dai, J. Han, and S. Liu (2025) Freeedit: mask-free reference-based image editing with multi-modal instruction. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §2.2. - [18] Y. Hu, S. Liu, Z. Tan, X. Yang, and X. Wang (2025) Image editing as programs with diffusion models. arXiv preprint arXiv:2506.04158. Cited by: Table 3. - [19] Y. Hu, W. Shi, X. Fu, D. Roth, M. Ostendorf, L. Zettlemoyer, N. A. Smith, and R. Krishna (2024) Visual sketchpad: sketching as a visual chain of thought for multimodal language models. NeurIPS 37, pp. 139348–139379. Cited by: §2.1. - [20] J. Huang, S. S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han (2022) Large language models can self-improve. arXiv preprint arXiv:2210.11610. Cited by: §2.1. - [21] W. Huang, S. Chen, Z. Xie, S. Cao, S. Tang, Y. Shen, Q. Yin, W. Hu, X. Wang, Y. Tang, et al. (2025) Interleaving reasoning for better text-to-image generation. arXiv preprint arXiv:2509.06945. Cited by: §2.1. - [22] Z. Huang, D. Zheng, C. Zou, R. Liu, X. Wang, K. Ji, W. Chai, J. Sun, L. Wang, Y. Lv, et al. (2025) Ming-univision: joint image understanding and generation with a unified continuous tokenizer. arXiv preprint arXiv:2510.06590. Cited by: §2.2, Table 3. - [23] D. Jiang, Z. Guo, R. Zhang, Z. Zong, H. Li, L. Zhuo, S. Yan, P. Heng, and H. Li (2025) T2i-r1: reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703. Cited by: §2.1. - [24] I. Krasin, T. Duerig, N. Alldrin, A. Veit, S. Abu-El-Haija, S. Belongie, D. Cai, Z. Feng, V. Ferrari, V. Gomes, A. Gupta, C. Sun, G. Chechik, K. Murphy, D. Narayanan, S. Shetty, Y. Song, J. Tighe, A. Vedaldi, S. Vijayanarasimhan, and O. Vinyals (2017) OpenImages: a public dataset for large-scale multi-label and multi-class image classification. Dataset available fromnhttps://storage.googleapis.com/openimages/web/index.html. Note: https://storage.googleapis.com/openimages/web/factsfigures.html Cited by: §4.1. - [25] M. Ku, D. Jiang, C. Wei, X. Yue, and W. Chen (2023) Viescore: towards explainable metrics for conditional image synthesis evaluation. arXiv preprint arXiv:2312.14867. Cited by: §3.4, §4.1. - [26] B. F. Labs, S. Batifol, A. Blattmann, F. Boesel, S. Consul, C. Diagne, T. Dockhorn, J. English, Z. English, P. Esser, et al. (2025) FLUX. 1 kontext: flow matching for in-context image generation and editing in latent space. arXiv preprint arXiv:2506.15742. Cited by: §4.1, Table 3. - [27] S. Lee, S. Kim, S. Park, G. Kim, and M. Seo (2024) Prometheus-vision: vision-language model as a judge for fine-grained evaluation. In Findings of the association for computational linguistics ACL 2024, pp. 11286–11315. Cited by: §3.4. - [28] B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y. Li, Z. Liu, et al. (2024) Llava-onevision: easy visual task transfer. arXiv preprint arXiv:2408.03326. Cited by: §4.1. - [29] C. Li, W. Wu, H. Zhang, Y. Xia, S. Mao, L. Dong, I. Vulić, and F. Wei (2025) Imagine while reasoning in space: multimodal visualization-of-thought. arXiv preprint arXiv:2501.07542. Cited by: §2.1. - [30] L. Li, G. Wang, Z. Zhang, Y. Li, X. Li, Q. Dou, J. Gu, T. Xue, and Y. Shan (2025) Tooncomposer: streamlining cartoon production with generative post-keyframing. arXiv preprint arXiv:2508.10881. Cited by: §2.2. - [31] L. Li, Z. Zhang, Y. Li, J. Xu, W. Hu, X. Li, W. Cheng, J. Gu, T. Xue, and Y. Shan (2025) Nvcomposer: boosting generative novel view synthesis with multiple sparse and unposed images. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 777–787. Cited by: §2.2. - [32] W. Liang, L. Yu, L. Luo, S. Iyer, N. Dong, C. Zhou, G. Ghosh, M. Lewis, W. Yih, L. Zettlemoyer, et al. (2024) Mixture-of-transformers: a sparse and scalable architecture for multi-modal foundation models. arXiv preprint arXiv:2411.04996. Cited by: §2.2. - [33] C. Liao, L. Liu, X. Wang, Z. Luo, X. Zhang, W. Zhao, J. Wu, L. Li, Z. Tian, and W. Huang (2025) Mogao: an omni foundation model for interleaved multi-modal generation. arXiv preprint arXiv:2505.05472. Cited by: §2.2. - [34] B. Lin, Z. Li, X. Cheng, Y. Niu, Y. Ye, X. He, S. Yuan, W. Yu, S. Wang, Y. Ge, et al. (2025) UniWorld-v1: high-resolution semantic encoders for unified visual understanding and generation. arXiv preprint arXiv:2506.03147. Cited by: §4.1, Table 3. - [35] Y. Lin, Z. Lin, K. Lin, J. Bai, P. Pan, C. Li, H. Chen, Z. Wang, X. Ding, W. Li, et al. (2025) JarvisArt: liberating human artistic creativity via an intelligent photo retouching agent. arXiv preprint arXiv:2506.17612. Cited by: §2.2. - [36] Y. Lin, L. Wang, K. Lin, Z. Lin, K. Gong, W. Li, B. Lin, Z. Li, S. Zhang, Y. Peng, et al. (2025) JarvisEvo: towards a self-evolving photo editing agent with synergistic editor-evaluator optimization. arXiv preprint arXiv:2511.23002. Cited by: §2.2. - [37] J. Liu, G. Liu, J. Liang, Y. Li, J. Liu, X. Wang, P. Wan, D. Zhang, and W. Ouyang (2025) Flow-grpo: training flow matching models via online rl. arXiv preprint arXiv:2505.05470. Cited by: §1, §3.4. - [38] S. Liu, Y. Han, P. Xing, F. Yin, R. Wang, W. Cheng, J. Liao, Y. Wang, H. Fu, C. Han, et al. (2025) Step1x-edit: a practical framework for general image editing. arXiv preprint arXiv:2504.17761. Cited by: §1, §4.1, Table 3. - [39] J. Lu, C. Clark, S. Lee, Z. Zhang, S. Khosla, R. Marten, D. Hoiem, and A. Kembhavi (2024) Unified-io 2: scaling autoregressive multimodal models with vision language audio and action. In CVPR, pp. 26439–26455. Cited by: §2.2. - [40] P. Lu, S. Mishra, T. Xia, L. Qiu, K. Chang, S. Zhu, O. Tafjord, P. Clark, and A. Kalyan (2022) Learn to explain: multimodal reasoning via thought chains for science question answering. NeurIPS 35, pp. 2507–2521. Cited by: §2.1. - [41] Y. Ma, X. Liu, X. Chen, W. Liu, C. Wu, Z. Wu, Z. Pan, Z. Xie, H. Zhang, X. Yu, L. Zhao, Y. Wang, J. Liu, and C. Ruan (2024) JanusFlow: harmonizing autoregression and rectified flow for unified multimodal understanding and generation. arXiv preprint arXiv:2411.07975. Cited by: §2.2. - [42] Y. Ma, Y. He, X. Cun, X. Wang, S. Chen, X. Li, and Q. Chen (2024) Follow your pose: pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, pp. 4117–4125. Cited by: §2.2. - [43] Y. Ma, Z. Wang, T. Ren, M. Zheng, H. Liu, J. Guo, M. Fong, Y. Xue, Z. Zhao, K. Schindler, et al. (2026) FastVMT: eliminating redundancy in video motion transfer. arXiv preprint arXiv:2602.05551. Cited by: §2.2. - [44] OpenAI (2025) GPT-image-1. External Links: Link Cited by: §4.1, Table 3. - [45] OpenAI (2025) Introducing 4o image generation. External Links: Link Cited by: §2.2. - [46] X. Pan, S. N. Shukla, A. Singh, Z. Zhao, S. K. Mishra, J. Wang, Z. Xu, J. Chen, K. Li, F. Juefei-Xu, J. Hou, and S. Xie (2025) Transfer between modalities with metaqueries. arXiv preprint arXiv:2504.06256. Cited by: §2.2. - [47] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195–4205. Cited by: §2.2. - [48] L. Qu, H. Zhang, Y. Liu, X. Wang, Y. Jiang, Y. Gao, H. Ye, D. K. Du, Z. Yuan, and X. Wu (2024) Tokenflow: unified image tokenizer for multimodal understanding and generation. arXiv preprint arXiv:2412.03069. Cited by: §2.2. - [49] C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al. (2022) Laion-5b: an open large-scale dataset for training next generation image-text models. NeurIPS 35, pp. 25278–25294. Cited by: §4.1. - [50] H. Shao, S. Qian, H. Xiao, G. Song, Z. Zong, L. Wang, Y. Liu, and H. Li (2024) Visual cot: advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. NeurIPS 37, pp. 8612–8642. Cited by: §2.1. - [51] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. (2024) Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §3.4. - [52] W. Shi, X. Han, C. Zhou, W. Liang, X. V. Lin, L. Zettlemoyer, and L. Yu (2024) LlamaFusion: adapting pretrained language models for multimodal generation. arXiv preprint arXiv:2412.15188. Cited by: §2.2. - [53] Q. Sun, Q. Yu, Y. Cui, F. Zhang, X. Zhang, Y. Wang, H. Gao, J. Liu, T. Huang, and X. Wang (2024) Emu: generative pretraining in multimodality. In ICLR, Cited by: §2.2. - [54] C. Team (2024) Chameleon: mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818. Cited by: §2.2. - [55] H. Theil (1970) On the estimation of relationships involving qualitative variables. American Journal of Sociology 76 (1), pp. 103–154. Cited by: §3.1. - [56] S. Tong, D. Fan, J. Zhu, Y. Xiong, X. Chen, K. Sinha, M. Rabbat, Y. LeCun, S. Xie, and Z. Liu (2024) Metamorph: multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412.14164. Cited by: §2.2. - [57] X. Wang, X. Zhang, Z. Luo, Q. Sun, Y. Cui, J. Wang, F. Zhang, Y. Wang, Z. Li, Q. Yu, et al. (2024) Emu3: next-token prediction is all you need. arXiv preprint arxiv:2409.18869. Cited by: §2.2. - [58] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou (2022) Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. Cited by: §2.1. - [59] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. NeurIPS 35, pp. 24824–24837. Cited by: §2.1. - [60] C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, et al. (2025) Qwen-image technical report. arXiv preprint arXiv:2508.02324. Cited by: §4.1, §4.1, Table 3. - [61] C. Wu, X. Chen, Z. Wu, Y. Ma, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, C. Ruan, et al. (2025) Janus: decoupling visual encoding for unified multimodal understanding and generation. In CVPR, pp. 12966–12977. Cited by: §2.2. - [62] C. Wu, P. Zheng, R. Yan, S. Xiao, X. Luo, Y. Wang, W. Li, X. Jiang, Y. Liu, J. Zhou, et al. (2025) OmniGen2: exploration to advanced multimodal generation. arXiv preprint arXiv:2506.18871. Cited by: §2.2, Table 3. - [63] P. Wu and S. Xie (2024) V?: guided visual search as a core mechanism in multimodal llms. In CVPR, pp. 13084–13094. Cited by: §2.1. - [64] S. Wu, H. Fei, L. Qu, W. Ji, and T. Chua (2024) Next-gpt: any-to-any multimodal llm. In Forty-first ICML, Cited by: §2.2. - [65] S. Xiao, Y. Wang, J. Zhou, H. Yuan, X. Xing, R. Yan, C. Li, S. Wang, T. Huang, and Z. Liu (2025) Omnigen: unified image generation. In CVPR, pp. 13294–13304. Cited by: Table 3. - [66] J. Xie, W. Mao, Z. Bai, D. J. Zhang, W. Wang, K. Q. Lin, Y. Gu, Z. Chen, Z. Yang, and M. Z. Shou (2024) Show-o: one single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528. Cited by: §2.2. - [67] J. Xie, Z. Yang, and M. Z. Shou (2025) Show-o2: improved native unified multimodal models. arXiv preprint arXiv:2506.15564. Cited by: §2.2. - [68] Y. Xu, F. Tang, J. Cao, Y. Zhang, X. Kong, J. Li, O. Deussen, and T. Lee (2024) Headrouter: a training-free image editing framework for mm-dits by adaptively routing attention heads. arXiv preprint arXiv:2411.15034. Cited by: §2.2. - [69] Y. Xu, H. Yan, J. Cao, Y. Cheng, T. Hang, R. He, Z. Yin, S. Zhang, Y. Zhang, J. Li, et al. (2026) TAG-moe: task-aware gating for unified generative mixture-of-experts. arXiv preprint arXiv:2601.08881. Cited by: §2.2. - [70] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu (2024) Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: §3.6. - [71] S. Yang, X. Chen, and J. Liao (2023) Uni-paint: a unified framework for multimodal image inpainting with pretrained diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 3190–3199. Cited by: §2.2. - [72] S. Yang, L. Hou, H. Huang, C. Ma, P. Wan, D. Zhang, X. Chen, and J. Liao (2024) Direct-a-video: customized video generation with user-directed camera movement and object motion. In ACM SIGGRAPH 2024 Conference Papers, pp. 1–12. Cited by: §2.2. - [73] S. Yang, M. Hui, B. Zhao, Y. Zhou, N. Ruiz, and C. Xie (2025) Complex-edit: cot-like instruction generation for complexity-controllable image editing benchmark. arXiv preprint arXiv:2504.13143. Cited by: §1, §4.1. - [74] M. Yasunaga, L. Zettlemoyer, and M. Ghazvininejad (2025) Multimodal rewardbench: holistic evaluation of reward models for vision language models. URL https://api. semanticscholar. org/CorpusID 276482127. Cited by: §3.4. - [75] Y. Ye, X. He, Z. Li, B. Lin, S. Yuan, Z. Yan, B. Hou, and L. Yuan (2025) Imgedit: a unified image editing dataset and benchmark. arXiv preprint arXiv:2505.20275. Cited by: §1, §4.1. - [76] Q. Yu, W. Chow, Z. Yue, K. Pan, Y. Wu, X. Wan, J. Li, S. Tang, H. Zhang, and Y. Zhuang (2025) Anyedit: mastering unified high-quality image editing for any idea. In CVPR, pp. 26125–26135. Cited by: Table 3. - [77] K. Zhang, L. Mo, W. Chen, H. Sun, and Y. Su (2023) Magicbrush: a manually annotated dataset for instruction-guided image editing. NeurIPS 36, pp. 31428–31449. Cited by: Table 3. - [78] S. Zhang, S. Bai, G. Chen, L. Chen, J. Lu, J. Wang, and Y. Tang (2024) Narrative action evaluation with prompt-guided multimodal interaction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18430–18439. Cited by: §2.2. - [79] S. Zhang, W. Dai, S. Wang, X. Shen, J. Lu, J. Zhou, and Y. Tang (2023) Logo: a long-form video dataset for group action quality assessment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2405–2414. Cited by: §2.2. - [80] S. Zhang, J. Zhuang, Z. Zhang, Y. Shan, and Y. Tang (2025) Flexiact: towards flexible action control in heterogeneous scenarios. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pp. 1–11. Cited by: §2.2. - [81] Z. Zhang, J. Xie, Y. Lu, Z. Yang, and Y. Yang (2025) In-context edit: enabling instructional image editing with in-context generation in large scale diffusion transformer. arXiv preprint arXiv:2504.20690. Cited by: Table 3. - [82] Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola (2023) Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923. Cited by: §2.1. - [83] H. Zhao, X. S. Ma, L. Chen, S. Si, R. Wu, K. An, P. Yu, M. Zhang, Q. Li, and B. Chang (2024) Ultraedit: instruction-based fine-grained image editing at scale. NeurIPS 37, pp. 3058–3093. Cited by: Table 3. - [84] X. Zhao, P. Zhang, K. Tang, X. Zhu, H. Li, W. Chai, Z. Zhang, R. Xia, G. Zhai, J. Yan, et al. (2025) Envisioning beyond the pixels: benchmarking reasoning-informed visual editing. arXiv preprint arXiv:2504.02826. Cited by: §1, §4.1. - [85] C. Zhou, L. Yu, A. Babu, K. Tirumala, M. Yasunaga, L. Shamis, J. Kahn, X. Ma, L. Zettlemoyer, and O. Levy (2024) Transfusion: predict the next token and diffuse images with one multi-modal model. arXiv preprint arxiv:2408.11039. Cited by: §2.2. - [86] T. Zhu, S. Zhang, J. Shao, and Y. Tang (2025) Kv-edit: training-free image editing for precise background preservation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16607–16617. Cited by: §2.2. - [87] J. Zhuang, X. Ju, Z. Zhang, Y. Liu, S. Zhang, C. Yuan, and Y. Shan (2024) ColorFlow: retrieval-augmented image sequence colorization. arXiv preprint arXiv:2412.11815. Cited by: §2.2.