## Diagram: Group Arena Reward and Fine-Grained Multi-Domain Rubrics
### Overview
The image presents two interconnected diagrams. On the left, a **Group Arena Reward** system is depicted as a hierarchical policy model with nodes (O₁–O₄, O_G) and directional arrows indicating priority. On the right, a **Fine-Grained Multi-Domain Rubrics** wheel outlines evaluation criteria for language models across four domains: Dialog, NLP Tasks, Creative Writing, and Reasoning. Each domain contains subcategories with specific rubrics.
---
### Components/Axes
#### Left Diagram: Group Arena Reward
- **Nodes**:
- O₁, O₂, O₃, O₄, O_G (central node)
- **Arrows**:
- O₁ → O₄ → O₃ → O_G (hierarchical flow)
- O₁ > O₄ > O₃ > ... > O_G (priority ranking)
- **Visual Elements**:
- "BETTER" badge at the bottom left.
#### Right Diagram: Fine-Grained Multi-Domain Rubrics
- **Central Node**: "Ling General RL"
- **Quadrants**:
1. **Dialog** (pink):
- Subcategories: LongQA, STEM, Emotion, Multi-turn, Role Play
- Rubrics: Specific (specific), Self Consistency, Recall, Instruction Adherence
2. **NLP Tasks** (red):
- Subcategories: Polishing, Summary, Poetry, Long-form Writing, Creative Writing, Translation
- Rubrics: Mixed Languages Penalty, Dynamic Length Penalty, Emotional Depth, Creativity
3. **Creative Writing** (red):
- Subcategories: Write a Poem About Love
- Rubrics: Mixed Languages Penalty, Dynamic Length Penalty, Emotional Depth, Creativity
4. **Reasoning** (green):
- Subcategories: Logic, Code, Math
- Rubrics: Dynamic Length Penalty, Evaluated by rule-based verifier
- **Additional Rubrics**:
- **Instruct Following** (green):
- Subcategories: Format, Length, Style
- Rubrics: Length Constraints, Detectable Format, Semantic elements (theme, style)
---
### Detailed Analysis
#### Left Diagram: Group Arena Reward
- The policy model prioritizes objectives in a strict hierarchy: O₁ (highest) > O₄ > O₃ > O_G (lowest).
- The "BETTER" badge implies that higher-ranked objectives contribute more significantly to the reward signal.
#### Right Diagram: Fine-Grained Multi-Domain Rubrics
- **Dialog**: Focuses on conversational quality, with rubrics emphasizing specificity, consistency, and adherence to instructions.
- **NLP Tasks**: Evaluates technical skills (e.g., summarization, translation) with penalties for mixed languages and dynamic length constraints.
- **Creative Writing**: Assesses emotional depth and creativity, with penalties for non-native language use and length variability.
- **Reasoning**: Prioritizes logical and mathematical accuracy, with rule-based verification for correctness.
---
### Key Observations
1. **Hierarchical Prioritization**: The Group Arena Reward system enforces a strict order of objectives, suggesting that higher-ranked objectives (e.g., O₁) are more critical to the model's performance.
2. **Domain-Specific Rubrics**: Each domain (Dialog, NLP Tasks, etc.) has tailored evaluation criteria, reflecting the complexity of language model assessment.
3. **Penalty Mechanisms**: Mixed Languages Penalty and Dynamic Length Penalty appear across multiple domains, indicating a focus on linguistic coherence and conciseness.
4. **Rule-Based Verification**: Reasoning tasks (e.g., math problems) are explicitly evaluated using rule-based systems, ensuring objective correctness.
---
### Interpretation
The diagrams illustrate a **multi-layered evaluation framework** for language models. The Group Arena Reward system prioritizes objectives to guide model behavior, while the Fine-Grained Rubrics provide granular metrics for assessing performance across diverse tasks. The overlap in penalties (e.g., Dynamic Length Penalty) suggests a shared emphasis on efficiency and clarity. The "BETTER" badge implies that aligning with higher-ranked objectives (O₁–O₄) directly correlates with improved outcomes, reinforcing the importance of hierarchical goal-setting in reinforcement learning systems.
