## Diagram: Two-Stage Training Framework (CoE Tuning and R-GRPO) ### Overview The image presents a two-stage machine learning framework designed to train a Multimodal Large Language Model (MLLM) to detect synthetic images. * **Stage 1 (Left):** Focuses on "CoE Tuning" (Chain-of-Thought/CoE), where the model is trained to generate reasoning steps and a final answer. * **Stage 2 (Right):** Focuses on "R-GRPO" (Reinforcement Learning with Group Relative Policy Optimization), where the model generates multiple completions and is evaluated using a multi-faceted reward system (Answer, Think, and Multi-view alignment). --- ### Components/Axes #### Left Panel: Stage 1: CoE Tuning * **Inputs (Top):** A document icon containing the text: "Please help me determine whether this image is real or synthetic?" and an image of a bird. * **Processing (Center):** An MLLM block marked with a fire icon. * **Output (Bottom):** A text block containing reasoning and an answer: `<answer>1</answer>`. * **Loss Functions:** * $\mathcal{L}_{\text{think}}$ (indicated by a blue arrow pointing to the reasoning process). * $\mathcal{L}_{\text{answer}}$ (indicated by a blue arrow pointing to the answer). #### Right Panel: Stage 2: R-GRPO * **Inputs (Top):** A document icon containing the text: "Please help me determine whether this image is real or synthetic?" and an image of a bird. * **Processing (Center):** An MLLM block generating a group of completions ($G$). * **Reward System:** * **(1) Answer reward:** Binary evaluation of the final answer. * **(2) Think reward:** Evaluates reasoning steps: * Match: R=1 * Similar: R=0.5 * Mismatch: R=0 * **(3) Multi-view alignment reward:** Evaluates reasoning based on visual evidence. * **Match (Green check):** Reasoning -> Robot icon -> R=1. * **Mismatch (Red x):** Reasoning -> Robot icon -> R=0. --- ### Detailed Analysis #### Stage 1: CoE Tuning The model is trained using a supervised approach where the loss is split between the reasoning process ($\mathcal{L}_{\text{think}}$) and the final classification ($\mathcal{L}_{\text{answer}}$). The example shows the model identifying "uneven features" and "synthetic traces" before concluding the image is synthetic (Answer: 1). #### Stage 2: R-GRPO This stage employs reinforcement learning. The model generates a group of completions ($G$). These completions are scored based on three distinct criteria: 1. **Answer Accuracy:** Binary reward (1 or 0) for the final answer. 2. **Reasoning Quality (Think reward):** A graded reward (1, 0.5, or 0) based on how well the reasoning matches the ground truth. 3. **Multi-view Alignment:** A sophisticated reward mechanism that checks if the model's reasoning aligns with specific visual forensic cues. * The diagram highlights that "Match" scenarios involve identifying "structural irregularities" and "high-frequency artifacts." * The "Mismatch" scenario involves a model failing to identify artifacts or misinterpreting the visual evidence (e.g., claiming the eyeball "appears natural" when it actually contains artifacts). --- ### Key Observations * **Granular Supervision:** The framework does not rely solely on the final answer. It explicitly rewards the "thinking" process and the ability to correlate reasoning with specific visual evidence (Multi-view alignment). * **Forensic Focus:** The "Multi-view alignment" reward specifically targets image forensics, such as looking for "high-frequency artifacts" and "structural irregularities" in zoomed-in views. * **Feedback Loop:** The green arrow in Stage 2 indicates an iterative reinforcement learning process where the model learns from the rewards of its generated completions. --- ### Interpretation This diagram outlines a robust training pipeline for AI-based synthetic image detection. The transition from **Stage 1 (CoE Tuning)** to **Stage 2 (R-GRPO)** represents a shift from learning *what* to say (supervised reasoning) to learning *how to be accurate and consistent* (reinforcement learning). The most critical component is the **Multi-view alignment reward**. This suggests that the researchers are training the model to act like a forensic analyst. By rewarding the model when its "thought" process correctly identifies specific, subtle visual artifacts (like high-frequency noise or structural irregularities) and penalizing it when it misses them (even if the final answer might be correct by chance), the framework forces the model to develop a deeper, more reliable understanding of synthetic image characteristics rather than relying on superficial patterns.