## Bar Chart: Rouge-L Score by Model and State Evaluator ### Overview The image is a bar chart comparing the Rouge-L scores of different models (8B, 70B, 405B) across various tasks and state evaluators (Score, Select). The x-axis represents the tasks, and the y-axis represents the Rouge-L score. The chart uses different colored bars to represent the models and patterned bars to represent the state evaluators. ### Components/Axes * **Y-axis:** "Rouge-L Score", ranging from 0 to 50, with gridlines at intervals of 10. * **X-axis:** Categorical axis representing different tasks: "Base", "Text-RAG", "Graph-RAG", "Graph-CoT Agent", "Graph-CoT Explore", "Graph-ToT Agent", "Graph-ToT Explore", "Graph-GoT Agent", "Graph-GoT Explore". * **Legend (Top-Left):** * "Models": * Blue: "8B" * Orange: "70B" * Green: "405B" * "State Evaluators" (Top-Right): * Gray: "Score" (Solid fill) * Diagonal Lines: "Select" (Hatched fill) ### Detailed Analysis Here's a breakdown of the Rouge-L scores for each task and model, considering both "Score" and "Select" state evaluators: * **Base:** * 8B (Score): ~7.5 * 70B (Score): ~10 * 405B (Score): ~9 * **Text-RAG:** * 8B (Score): ~8.5 * 70B (Score): ~10.5 * 405B (Score): ~12 * **Graph-RAG:** * 8B (Score): ~13 * 70B (Score): ~18 * 405B (Score): ~16.5 * **Graph-CoT Agent:** * 8B (Score): ~17 * 70B (Score): ~33.5 * 405B (Score): ~28.5 * **Graph-CoT Explore:** * 8B (Score): ~25.5 * 70B (Score): ~29 * 405B (Score): ~29 * **Graph-ToT Agent:** * 8B (Score): ~29 * 70B (Score): ~39 * 405B (Score): ~48 * **Graph-ToT Explore:** * 8B (Score): ~24.5 (Select) * 70B (Score): ~33.5 (Select) * 405B (Score): ~34 (Select) * **Graph-GoT Agent:** * 8B (Score): ~29 * 70B (Score): ~31 * 405B (Score): ~44 * **Graph-GoT Explore:** * 8B (Score): ~24.5 (Select) * 70B (Score): ~36 (Select) * 405B (Score): ~34 (Select) **Trends:** * Generally, the Rouge-L score increases as the model size increases (8B < 70B < 405B). * The "Graph-ToT Agent" task shows the highest Rouge-L scores for all models. * The "Base" and "Text-RAG" tasks have the lowest Rouge-L scores. * The "Select" state evaluator generally results in lower scores compared to the "Score" evaluator, especially for "Graph-ToT Explore" and "Graph-GoT Explore". ### Key Observations * The 405B model consistently outperforms the 70B and 8B models across all tasks. * The "Graph-ToT Agent" task appears to be the most effective, yielding the highest scores. * The "Select" state evaluator seems to be more conservative or selective, resulting in lower scores. ### Interpretation The data suggests that larger models (405B) generally achieve higher Rouge-L scores, indicating better performance in these tasks. The "Graph-ToT Agent" task seems to be particularly well-suited for these models, potentially due to its ability to leverage graph-based reasoning and agent-based exploration. The difference between "Score" and "Select" state evaluators highlights the impact of evaluation criteria on the reported performance. The "Select" evaluator might be prioritizing precision over recall, leading to lower overall scores but potentially higher-quality results. The performance increase from "Base" to "Text-RAG" to "Graph-RAG" suggests that incorporating graph-based information retrieval enhances the models' capabilities.

\n ## Bar Chart: Rouge-L Score vs. Models and State Evaluators ### Overview This bar chart compares the Rouge-L scores of different language models (8B, 70B, and 405B) across various state evaluators (Base, Text-RAG, Graph-RAG, Graph-CoT Agent, Graph-CoT Explore, Graph-ToT Agent, Graph-ToT Explore, Graph-GoT Agent, Graph-GoT Explore). The Rouge-L score is plotted on the y-axis, and the state evaluators are displayed on the x-axis. Each state evaluator has three bars representing the performance of the three models. ### Components/Axes * **Y-axis:** Rouge-L Score (Scale: 0 to 50, increments of 10) * **X-axis:** State Evaluators (Categories: Base, Text-RAG, Graph-RAG, Graph-CoT Agent, Graph-CoT Explore, Graph-ToT Agent, Graph-ToT Explore, Graph-GoT Agent, Graph-GoT Explore) * **Legend:** * Models: * 8B (Blue) * 70B (Orange) * 405B (Green) * State Evaluators: * Score (Solid bars) * Select (Hatched bars) ### Detailed Analysis The chart consists of nine groups of three bars, one for each state evaluator. Within each group, the bars represent the Rouge-L scores for the 8B, 70B, and 405B models, respectively. * **Base:** * 8B: Approximately 7. * 70B: Approximately 9. * 405B: Approximately 8. * **Text-RAG:** * 8B: Approximately 10. * 70B: Approximately 12. * 405B: Approximately 11. * **Graph-RAG:** * 8B: Approximately 17. * 70B: Approximately 20. * 405B: Approximately 16. * **Graph-CoT Agent:** * 8B: Approximately 26. * 70B: Approximately 34. * 405B: Approximately 30. * **Graph-CoT Explore:** * 8B: Approximately 28. * 70B: Approximately 30. * 405B: Approximately 29. * **Graph-ToT Agent:** * 8B: Approximately 32. * 70B: Approximately 35. * 405B: Approximately 33. * **Graph-ToT Explore:** * 8B: Approximately 33. * 70B: Approximately 34. * 405B: Approximately 35. * **Graph-GoT Agent:** * 8B: Approximately 28. * 70B: Approximately 32. * 405B: Approximately 44. * **Graph-GoT Explore:** * 8B: Approximately 31. * 70B: Approximately 33. * 405B: Approximately 36. Generally, the 405B model consistently outperforms the 8B and 70B models, especially in the Graph-GoT Agent state evaluator. The 70B model generally outperforms the 8B model. ### Key Observations * The largest performance difference between models is observed with the "Graph-GoT Agent" state evaluator, where the 405B model achieves a significantly higher Rouge-L score (approximately 44) compared to the 8B (approximately 28) and 70B (approximately 32) models. * The "Base" state evaluator shows the lowest Rouge-L scores across all models. * The performance gap between the 8B and 70B models is relatively consistent across most state evaluators. * The "Graph-ToT Explore" state evaluator shows the highest scores for the 405B model. ### Interpretation The data suggests that increasing model size (from 8B to 70B to 405B) generally improves performance, as measured by the Rouge-L score. The choice of state evaluator significantly impacts performance, with "Graph-GoT Agent" yielding the most substantial gains from larger models. This indicates that the Graph-GoT Agent approach is particularly effective at leveraging the capabilities of larger language models. The low scores for the "Base" evaluator suggest that more sophisticated evaluation methods (like those involving graphs and CoT reasoning) are necessary to accurately assess the performance of these models. The consistent trend of larger models performing better supports the hypothesis that model capacity is a key factor in achieving higher Rouge-L scores. The differences between the "Agent" and "Explore" variants within the Graph-based evaluators suggest that the exploration strategy also plays a role in performance.

## Grouped Bar Chart: Rouge-L Score Comparison Across Models and Methods ### Overview This image is a grouped bar chart comparing the performance of different AI models (8B, 70B, 405B parameters) across various reasoning methods, as measured by the Rouge-L Score. The chart evaluates two types of state evaluators ("Score" and "Select") for each method. ### Components/Axes * **Chart Type:** Grouped Bar Chart. * **Y-Axis:** Labeled "Rouge-L Score". Scale ranges from 0 to 50, with major gridlines at intervals of 10. * **X-Axis:** Lists 9 distinct reasoning methods. From left to right: 1. Base 2. Text-RAG 3. Graph-RAG 4. Graph-CoT Agent 5. Graph-CoT Explore 6. Graph-ToT Agent 7. Graph-ToT Explore 8. Graph-GoT Agent 9. Graph-GoT Explore * **Legend 1 (Top-Left):** Titled "Models". Defines color coding for model size: * Blue: 8B * Orange: 70B * Green: 405B * **Legend 2 (Top-Center):** Titled "State Evaluators". Defines bar fill pattern: * Solid Fill: "Score" * Hatched Fill (diagonal lines): "Select" * **Spatial Layout:** The two legends are positioned in the top-left and top-center of the chart area. The bars are grouped by method, with each group containing up to six bars (three models, each with two evaluator types). ### Detailed Analysis Each method group on the x-axis contains bars for the 8B (blue), 70B (orange), and 405B (green) models. Within each model color, the left bar is solid ("Score" evaluator) and the right bar is hatched ("Select" evaluator). **Trend Verification & Data Extraction (Approximate Values):** 1. **Base:** * 8B (Blue): ~7.5 (Score), ~8 (Select) * 70B (Orange): ~10 (Score), ~10 (Select) * 405B (Green): ~9 (Score), ~9 (Select) * *Trend:* Scores are low (<10). Larger models do not show a clear advantage here. 2. **Text-RAG:** * 8B (Blue): ~8.5 (Score), ~8.5 (Select) * 70B (Orange): ~10.5 (Score), ~10.5 (Select) * 405B (Green): ~11.5 (Score), ~11.5 (Select) * *Trend:* Slight improvement over Base. A small, consistent increase with model size. 3. **Graph-RAG:** * 8B (Blue): ~13 (Score), ~13 (Select) * 70B (Orange): ~18 (Score), ~18 (Select) * 405B (Green): ~16 (Score), ~16 (Select) * *Trend:* Notable jump from Text-RAG. The 70B model performs best here. 4. **Graph-CoT Agent:** * 8B (Blue): ~17 (Score), ~17 (Select) * 70B (Orange): ~33.5 (Score), ~33.5 (Select) * 405B (Green): ~28.5 (Score), ~28.5 (Select) * *Trend:* Significant increase, especially for 70B. 70B outperforms 405B. 5. **Graph-CoT Explore:** * 8B (Blue): ~25.5 (Score), ~25.5 (Select) * 70B (Orange): ~29.5 (Score), ~29.5 (Select) * 405B (Green): ~28.5 (Score), ~28.5 (Select) * *Trend:* Scores are high and clustered. 70B and 405B are nearly tied. 6. **Graph-ToT Agent:** * 8B (Blue): ~29 (Score), ~29 (Select) * 70B (Orange): ~38.5 (Score), ~40.5 (Select) * 405B (Green): ~47.5 (Score), ~46.5 (Select) * *Trend:* Highest scores observed so far. Clear advantage for larger models. 405B achieves the highest single score (~47.5). 7. **Graph-ToT Explore:** * 8B (Blue): ~24.5 (Score), ~25 (Select) * 70B (Orange): ~32.5 (Score), ~33.5 (Select) * 405B (Green): ~34 (Score), ~34 (Select) * *Trend:* Scores drop compared to the "Agent" variant. 405B and 70B are close. 8. **Graph-GoT Agent:** * 8B (Blue): ~29.5 (Score), ~29.5 (Select) * 70B (Orange): ~31 (Score), ~40.5 (Select) * 405B (Green): ~43.5 (Score), ~47.5 (Select) * *Trend:* Very high scores. A large discrepancy appears for the 70B model between "Score" (~31) and "Select" (~40.5) evaluators. 405B "Select" is very high (~47.5). 9. **Graph-GoT Explore:** * 8B (Blue): ~25 (Score), ~25 (Select) * 70B (Orange): ~31 (Score), ~37 (Select) * 405B (Green): ~35 (Score), ~34 (Select) * *Trend:* Similar pattern to Graph-ToT Explore, with "Agent" variants outperforming "Explore". 70B shows a notable gap between evaluators. ### Key Observations 1. **Method Progression:** There is a clear upward trend in Rouge-L scores as methods evolve from Base -> Text-RAG -> Graph-RAG -> CoT -> ToT/GoT. Graph-based Tree-of-Thought (ToT) and Graph-of-Thought (GoT) methods achieve the highest performance. 2. **Model Size Impact:** Generally, larger models (70B, 405B) outperform the 8B model. However, the advantage is not always linear; in some cases (e.g., Graph-RAG, Graph-CoT Agent), the 70B model outperforms the 405B model. 3. **Agent vs. Explore:** For ToT and GoT methods, the "Agent" variant consistently yields higher scores than the "Explore" variant for the same model size. 4. **Evaluator Discrepancy:** For most method/model combinations, the "Score" and "Select" evaluators produce nearly identical results (bars of equal height). The most significant exception is **Graph-GoT Agent with the 70B model**, where the "Select" evaluator bar is substantially taller (~40.5) than the "Score" evaluator bar (~31). 5. **Peak Performance:** The highest approximate score on the chart is ~47.5, achieved by the **405B model using the Graph-GoT Agent method with the "Select" evaluator**. ### Interpretation This chart demonstrates the effectiveness of advanced, graph-augmented reasoning frameworks (ToT, GoT) over simpler RAG or chain-of-thought approaches for the task measured by Rouge-L. The data suggests that structuring reasoning as a tree or graph ("Agent" mode) is more effective than an exploratory approach ("Explore" mode). The relationship between model size and performance is complex. While scaling from 8B to 70B provides a major boost, further scaling to 405B yields diminishing or inconsistent returns, indicating that methodological improvements (like switching from CoT to ToT) can be as impactful as raw parameter scaling. The outlier in the Graph-GoT Agent (70B) results, where evaluators disagree, may indicate instability in that specific configuration or that the "Select" evaluator is better at capturing the benefits of the GoT method for that model size. Overall, the chart makes a strong case for investing in sophisticated graph-based reasoning architectures, particularly when paired with sufficiently large language models.

## Bar Chart: Rouge-L Score Comparison Across Models and Tasks ### Overview The chart compares Rouge-L scores for three models (8B, 70B, 405B) across 10 evaluation tasks. Tasks include Base, Text-RAG, Graph-RAG, Graph-CoT Agent/Explore, Graph-ToT Agent/Explore, and Graph-GoT Agent/Explore. The y-axis ranges from 0 to 50, with higher scores indicating better performance. ### Components/Axes - **X-axis (Tasks)**: Base, Text-RAG, Graph-RAG, Graph-CoT Agent, Graph-CoT Explore, Graph-ToT Agent, Graph-ToT Explore, Graph-GoT Agent, Graph-GoT Explore. - **Y-axis (Rouge-L Score)**: 0–50 scale. - **Legend**: - Blue = 8B model - Orange = 70B model - Green = 405B model - **State Evaluators**: Score (solid bars) and Select (striped bars, not visible in data). ### Detailed Analysis | Task | 8B | 70B | 405B | |-----------------------|------|------|------| | Base | ~7 | ~10 | ~9 | | Text-RAG | ~8 | ~10 | ~12 | | Graph-RAG | ~13 | ~18 | ~16 | | Graph-CoT Agent | ~17 | ~33 | ~28 | | Graph-CoT Explore | ~25 | ~29 | ~28 | | Graph-ToT Agent | ~29 | ~39 | ~48 | | Graph-ToT Explore | ~29 | ~33 | ~34 | | Graph-GoT Agent | ~29 | ~41 | ~43 | | Graph-GoT Explore | ~25 | ~31 | ~35 | ### Key Observations - **Model Size Correlation**: Larger models (405B) generally outperform smaller ones, especially in complex tasks (e.g., Graph-ToT Agent: 405B = 48 vs. 8B = 29). - **Anomalies**: - In Graph-RAG, 70B (18) slightly outperforms 405B (16). - 405B underperforms 70B in Graph-CoT Agent (28 vs. 33). - **Task-Specific Trends**: - Graph-ToT tasks show the largest performance gaps between models. - Graph-GoT tasks maintain consistent 405B dominance. ### Interpretation The data suggests that model size strongly correlates with performance in complex reasoning tasks (e.g., Graph-ToT), where 405B achieves ~48 vs. 8B’s 29. However, exceptions like Graph-RAG (70B > 405B) imply that architectural design or training data may sometimes outweigh raw model size. The 70B model’s mid-range performance highlights its potential as a cost-effective alternative to 405B in most scenarios. The 8B model’s consistent underperformance underscores limitations in handling advanced tasks, likely due to insufficient capacity for nuanced reasoning.