## Bar Charts: Performance and Cost Comparison of Different Models ### Overview The image presents two bar charts comparing the performance and average cost of different models, including GPT-4o, GPT-4o mini, and various knowledge graph-enhanced models (KGOT and KGOT fusion) and baselines. The left chart displays the number of solved tasks (higher is better), while the right chart shows the average cost in dollars (lower is better) on a logarithmic scale. ### Components/Axes **Left Chart (Number of Solved Tasks):** * **Title:** Number of Solved Tasks (the higher the better) * **Y-axis:** Number of Solved Tasks, ranging from 0 to 70. * **X-axis:** Different models and configurations: GPT-4o, GPT-4o mini, Neo4j + Query, NetworkX + Query, Neo4j + DR, NetworkX + DR, RDF4J + Query, Neo4j + NetworkX (Query + DR), Simple RAG, GraphRAG, GPTSwarm, Magentic-One, HF GPT-4o mini, HF GPT-4o. * **Bar Colors (Legend, located at the top-center of the chart):** * Level 1: Light Blue * Level 2: Blue * Level 3: Purple * **Sections:** Zero-Shot, KGOT, KGOT (fusion), Baselines. * **Maximum Solved Tasks:** Indicated by an arrow pointing to the top of the GPT-4o bar in the Baselines section, labeled "Max: 71". **Right Chart (Average Cost):** * **Title:** Average Cost ($) (the lower the better) * **Y-axis:** Average Cost ($) on a logarithmic scale, ranging from 10^-3 to 10^0 (0.001 to 1). * **X-axis:** Same models and configurations as the left chart. * **Bar Color:** Purple * **Sections:** Zero-Shot, KGOT, KGOT (fusion), Baselines. * **Maximum Cost:** Indicated by an arrow pointing to the top of the GPT-4o bar in the Baselines section, labeled "Max: 3.403$". ### Detailed Analysis **Left Chart (Number of Solved Tasks):** * **GPT-4o:** * Zero-Shot: Level 1: 10, Level 2: 17, Level 3: 2, Total: 29 * Baselines: Level 1: 22, Level 2: 31, Total: 53 * **GPT-4o mini:** * Zero-Shot: Level 1: 4, Level 2: 13, Total: 17 * KGOT (fusion): Level 1: 20, Level 2: 18, Level 3: 1, Total: 39 * **Neo4j + Query:** * KGOT: Level 1: 21, Level 2: 18, Level 3: 1, Total: 40 * **NetworkX + Query:** * KGOT: Level 1: 21, Level 2: 16, Level 3: 3, Total: 40 * **Neo4j + DR:** * KGOT: Level 1: 21, Level 2: 21, Total: 42 * KGOT (fusion): Level 1: 34, Level 2: 33, Level 3: 4, Total: 71 * **NetworkX + DR:** * KGOT: Level 1: 20, Level 2: 18, Level 3: 2, Total: 40 * KGOT (fusion): Level 1: 29, Level 2: 24, Level 3: 4, Total: 57 * **RDF4J + Query:** * KGOT: Level 1: 20, Level 2: 15, Level 3: 1, Total: 36 * **Neo4j + NetworkX (Query + DR):** * KGOT (fusion): Level 1: 27, Level 2: 28, Level 3: 2, Total: 57 * **Simple RAG:** * Baselines: Level 1: 18, Level 2: 15, Level 3: 2, Total: 35 * **GraphRAG:** * Baselines: Level 1: 10, Level 2: 13, Total: 23 * **GPTSwarm:** * Baselines: Level 1: 13, Level 2: 13, Total: 26 * **Magentic-One:** * Baselines: Level 1: 13, Level 2: 18, Level 3: 1, Total: 32 * **HF GPT-4o mini:** * Baselines: Level 1: 14, Level 2: 20, Total: 34 **Right Chart (Average Cost):** * **GPT-4o:** * Zero-Shot: 0.017$ * Baselines: 3.403$ * **GPT-4o mini:** * Zero-Shot: 0.001$ * KGOT (fusion): 0.696$ * **Neo4j + Query:** * KGOT: 0.098$ * **NetworkX + Query:** * KGOT: 0.135$ * **Neo4j + DR:** * KGOT: 0.119$ * KGOT (fusion): 0.145$ * **NetworkX + DR:** * KGOT: 0.148$ * KGOT (fusion): 0.098$ * **RDF4J + Query:** * KGOT: 0.091$ * **Neo4j + NetworkX (Query + DR):** * KGOT (fusion): 0.146$ * **Simple RAG:** * Baselines: 0.232$ * **GraphRAG:** * Baselines: 0.006$ * **GPTSwarm:** * Baselines: 0.696$ * **Magentic-One:** * Baselines: 0.258$ * **HF GPT-4o mini:** * Baselines: 0.258$ ### Key Observations * **Performance:** The Neo4j + DR model under KGOT (fusion) achieves the highest number of solved tasks (71), matching the maximum possible. GPT-4o also performs well, especially in the Baselines section. * **Cost:** GPT-4o mini in the Zero-Shot setting has the lowest average cost (0.001$). GPT-4o in the Baselines section has the highest cost (3.403$). * **Trade-off:** There is a clear trade-off between performance and cost. Models with higher performance tend to have higher costs. * **KGOT Fusion:** KGOT fusion models generally outperform their KGOT counterparts in terms of the number of solved tasks. ### Interpretation The data suggests that KGOT fusion, particularly with Neo4j + DR, significantly enhances the performance of the models in terms of the number of solved tasks. However, this performance comes at a cost, as these models are not the cheapest. GPT-4o mini offers a low-cost solution, but its performance is lower than the KGOT fusion models. The choice of model depends on the specific requirements and constraints of the application. If performance is the primary concern, KGOT fusion with Neo4j + DR is a good option. If cost is a major factor, GPT-4o mini in the Zero-Shot setting might be more suitable. GPT-4o in the Baselines section is the most expensive and does not offer a proportionally higher number of solved tasks compared to other models, indicating a less efficient trade-off. The different levels (Level 1, Level 2, Level 3) in the left chart likely represent different difficulty levels or types of tasks. The distribution of these levels across different models provides insights into their strengths and weaknesses in handling various types of tasks.

## Bar Chart: Performance Comparison of Language Models on Solving Tasks ### Overview This image presents a comparative bar chart evaluating the performance of several language models (GPT-40, GPT-4 mini, and variations incorporating Knowledge Graphs (KGs) and Retrieval Augmented Generation (RAG)) across different task-solving scenarios: Zero-Shot, KGoT, KGoT (fusion), and Baselines. The chart displays two primary metrics: the number of solved tasks (left panel) and the average cost per task in US dollars (right panel). The models are positioned along the x-axis, and performance is represented by bar height. Three levels of performance are indicated by color: Level 1 (light blue), Level 2 (medium blue), and Level 3 (dark blue). ### Components/Axes * **X-axis (Left & Right Panels):** Models being compared: GPT-40, GPT-4 mini, Neo4j, Neo4j + Query, Network + DR, Network + Query, Network + Query + DR, RDP4j + DR, Neo4j + Network (Query + DR), Simple RAG, GraphSwarm, MagneticOne, HF GPT-40 mini. * **Y-axis (Left Panel):** "Number of Solved Tasks (the higher the better)", ranging from 0 to 70, with a linear scale. * **Y-axis (Right Panel):** "Average Cost ($) (the lower the better)", displayed on a logarithmic scale from 10^-3 to 10^-1 (0.001 to 0.1). * **Legend (Top-Left):** Level 1 (light blue), Level 2 (medium blue), Level 3 (dark blue). * **Annotations:** "Max: 71" and "Max: 3.403$" indicating the maximum values for each metric. * **Horizontal Labels:** Dividing the chart into sections: "Zero-Shot", "KGoT", "KGoT (fusion)", "Baselines". ### Detailed Analysis or Content Details **Left Panel: Number of Solved Tasks** * **GPT-40:** Shows a consistent high performance across all categories. * Zero-Shot: Approximately 17 tasks solved. * KGoT: Approximately 21 tasks solved. * KGoT (fusion): Approximately 34 tasks solved. * Baselines: Approximately 31 tasks solved. * **GPT-4 mini:** Generally lower performance than GPT-40. * Zero-Shot: Approximately 10 tasks solved. * KGoT: Approximately 21 tasks solved. * KGoT (fusion): Approximately 28 tasks solved. * Baselines: Approximately 22 tasks solved. * **Neo4j:** Performance varies. * Zero-Shot: Approximately 14 tasks solved. * KGoT: Approximately 18 tasks solved. * KGoT (fusion): Approximately 24 tasks solved. * Baselines: Approximately 13 tasks solved. * **Neo4j + Query:** Similar to Neo4j. * Zero-Shot: Approximately 21 tasks solved. * KGoT: Approximately 21 tasks solved. * KGoT (fusion): Approximately 33 tasks solved. * Baselines: Approximately 18 tasks solved. * **Network + DR:** Performance is moderate. * Zero-Shot: Approximately 16 tasks solved. * KGoT: Approximately 21 tasks solved. * KGoT (fusion): Approximately 20 tasks solved. * Baselines: Approximately 13 tasks solved. * **Network + Query:** Performance is moderate. * Zero-Shot: Approximately 16 tasks solved. * KGoT: Approximately 21 tasks solved. * KGoT (fusion): Approximately 24 tasks solved. * Baselines: Approximately 18 tasks solved. * **Network + Query + DR:** Performance is moderate. * Zero-Shot: Approximately 18 tasks solved. * KGoT: Approximately 20 tasks solved. * KGoT (fusion): Approximately 34 tasks solved. * Baselines: Approximately 20 tasks solved. * **RDP4j + DR:** Performance is moderate. * Zero-Shot: Approximately 20 tasks solved. * KGoT: Approximately 21 tasks solved. * KGoT (fusion): Approximately 27 tasks solved. * Baselines: Approximately 14 tasks solved. * **Neo4j + Network (Query + DR):** Performance is high. * Zero-Shot: Approximately 22 tasks solved. * KGoT: Approximately 2 tasks solved. * KGoT (fusion): Approximately 33 tasks solved. * Baselines: Approximately 14 tasks solved. * **Simple RAG:** Performance is moderate. * Zero-Shot: Approximately 13 tasks solved. * KGoT: Approximately 18 tasks solved. * KGoT (fusion): Approximately 28 tasks solved. * Baselines: Approximately 13 tasks solved. * **GraphSwarm:** Performance is moderate. * Zero-Shot: Approximately 13 tasks solved. * KGoT: Approximately 18 tasks solved. * KGoT (fusion): Approximately 20 tasks solved. * Baselines: Approximately 13 tasks solved. * **MagneticOne:** Performance is moderate. * Zero-Shot: Approximately 13 tasks solved. * KGoT: Approximately 18 tasks solved. * KGoT (fusion): Approximately 20 tasks solved. * Baselines: Approximately 13 tasks solved. * **HF GPT-40 mini:** Performance is moderate. * Zero-Shot: Approximately 14 tasks solved. * KGoT: Approximately 18 tasks solved. * KGoT (fusion): Approximately 20 tasks solved. * Baselines: Approximately 13 tasks solved. **Right Panel: Average Cost ($)** * **GPT-40:** Highest cost across all categories. * Zero-Shot: Approximately 0.095$ * KGoT: Approximately 0.135$ * KGoT (fusion): Approximately 0.145$ * Baselines: Approximately 0.145$ * **GPT-4 mini:** Lower cost than GPT-40. * Zero-Shot: Approximately 0.00175$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.0025$ * **Neo4j:** Moderate cost. * Zero-Shot: Approximately 0.00135$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00135$ * **Neo4j + Query:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.002$ * **Network + DR:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00135$ * **Network + Query:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.002$ * **Network + Query + DR:** Moderate cost. * Zero-Shot: Approximately 0.00165$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.002$ * **RDP4j + DR:** Moderate cost. * Zero-Shot: Approximately 0.00165$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00145$ * **Neo4j + Network (Query + DR):** Moderate cost. * Zero-Shot: Approximately 0.00175$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00145$ * **Simple RAG:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00135$ * **GraphSwarm:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00135$ * **MagneticOne:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00135$ * **HF GPT-40 mini:** Moderate cost. * Zero-Shot: Approximately 0.00145$ * KGoT: Approximately 0.00195$ * KGoT (fusion): Approximately 0.00265$ * Baselines: Approximately 0.00135$ ### Key Observations * GPT-40 consistently achieves the highest number of solved tasks, but at a significantly higher cost. * GPT-4 mini offers a substantial cost reduction but with a corresponding decrease in performance. * Integrating Knowledge Graphs (KGoT and KGoT (fusion)) generally improves performance compared to Zero-Shot for most models. * The "KGoT (fusion)" scenario consistently yields better results than "KGoT" alone. * There's a clear trade-off between performance (number of solved tasks) and cost. * The logarithmic scale on the right panel highlights the cost differences more effectively. ### Interpretation The data suggests that while GPT-40 is the most capable model, its high cost may be prohibitive for many applications. Models incorporating Knowledge Graphs demonstrate improved performance, indicating the value of external knowledge sources. The fusion of KGoT with other techniques (e.g., Network + Query + DR) appears particularly effective. The chart illustrates a typical efficiency frontier: one can achieve higher performance, but only at the expense of increased cost. The optimal model choice will depend on the specific application's requirements and budget constraints. The consistent cost profile of the models other than GPT-40 and GPT-4 mini suggests a similar underlying computational expense, while the performance differences are likely due to the effectiveness of the knowledge integration and retrieval mechanisms. The logarithmic scale on the cost axis emphasizes the substantial cost difference between GPT-40 and the other models, making it a critical factor in decision-making.

## Comparative Analysis of AI/ML Methods: Task Performance vs. Cost ### Overview The image displays two adjacent bar charts comparing various AI/ML methods (primarily large language models and graph-based retrieval systems) across two key metrics: the number of tasks solved (left chart) and the average cost in dollars (right chart). The charts share the same set of methods on the x-axis, allowing for direct comparison of performance versus cost. ### Components/Axes **Left Chart: Number of Solved Tasks** * **Title:** Not explicitly stated, but the y-axis label serves as the title. * **Y-Axis:** "Number of Solved Tasks (the higher the better)". Scale is linear from 0 to 70. * **X-Axis:** Lists 14 different methods/models. * **Legend:** Located at the top center. Defines three performance levels: * **Level 1 (Cyan):** Likely represents basic or foundational task completion. * **Level 2 (Blue):** Likely represents intermediate or more complex task completion. * **Level 3 (Purple):** Likely represents advanced or highest-difficulty task completion. * **Data Series:** Each method has up to three stacked bars corresponding to the levels. The total height represents the total solved tasks. A gray bar labeled "Baselines" is present for some methods. * **Annotations:** A vertical line separates "Zero-Shot" methods (GPT-4o, GPT-4o mini) from others. A text box notes "Max: 71" pointing to the highest total bar. **Right Chart: Average Cost (S)** * **Title:** Not explicitly stated, but the y-axis label serves as the title. * **Y-Axis:** "Average Cost (S) (the lower the better)". Scale is **logarithmic**, ranging from 10^-3 ($0.001) to 10^0 ($1.00). * **X-Axis:** Same 14 methods as the left chart. * **Legend:** Same as the left chart (Level 1, Level 2, Level 3). * **Data Series:** Each method has up to three bars (cyan, blue, purple) representing the average cost per task for each level. A gray "Baselines" bar is also present. * **Annotations:** A vertical line separates "Zero-Shot" methods. A text box notes "Max: 3,403$" pointing to the highest cost bar. ### Detailed Analysis **Left Chart - Number of Solved Tasks (Approximate Values):** * **GPT-4o (Zero-Shot):** Level 1: ~10, Level 2: ~17, Level 3: ~2. Total: ~29. * **GPT-4o mini (Zero-Shot):** Level 1: ~13, Level 2: ~4. Total: ~17. * **Neo4j + Query:** Level 1: ~21, Level 2: ~18, Level 3: ~3. Total: ~42. * **Neo4j + DR:** Level 1: ~21, Level 2: ~16, Level 3: ~3. Total: ~40. * **NetworkX + Query:** Level 1: ~21, Level 2: ~21, Level 3: ~4. Total: ~46. * **NetworkX + DR:** Level 1: ~20, Level 2: ~18, Level 3: ~2. Total: ~40. * **RDKit + Query:** Level 1: ~20, Level 2: ~15, Level 3: ~2. Total: ~37. * **Neo4j + NetworkX (Query+DR):** Level 1: ~34, Level 2: ~33, Level 3: ~4. Total: ~71 (Highest). * **Neo4j + NetworkX (Query+DR):** (Second instance, likely a different configuration) Level 1: ~29, Level 2: ~24, Level 3: ~4. Total: ~57. * **Neo4j + NetworkX (Query+DR):** (Third instance) Level 1: ~27, Level 2: ~28, Level 3: ~4. Total: ~59. * **Simple RAG:** Level 1: ~18, Level 2: ~15, Level 3: ~2. Total: ~35. * **GraphRAG:** Level 1: ~10, Level 2: ~13, Level 3: ~1. Total: ~24. * **Matgraph-One:** Level 1: ~13, Level 2: ~18, Level 3: ~1. Total: ~32. * **HF GPT-4o mini:** Level 1: ~14, Level 2: ~20, Level 3: ~1. Total: ~35. * **HF GPT-4o:** Level 1: ~22, Level 2: ~31, Level 3: ~2. Total: ~55. **Right Chart - Average Cost (S) (Approximate Log-Scale Values):** * **GPT-4o (Zero-Shot):** Level 1: ~$0.0075, Level 2: ~$0.015, Level 3: ~$0.0015. * **GPT-4o mini (Zero-Shot):** Level 1: ~$0.0015. * **Neo4j + Query:** Level 1: ~$0.0985, Level 2: ~$0.1355, Level 3: ~$0.1105. * **Neo4j + DR:** Level 1: ~$0.1105, Level 2: ~$0.1485, Level 3: ~$0.0835. * **NetworkX + Query:** Level 1: ~$0.0985, Level 2: ~$0.1355, Level 3: ~$0.1105. * **NetworkX + DR:** Level 1: ~$0.1105, Level 2: ~$0.1485, Level 3: ~$0.0835. * **RDKit + Query:** Level 1: ~$0.0985, Level 2: ~$0.1355, Level 3: ~$0.1105. * **Neo4j + NetworkX (Query+DR):** Level 1: ~$0.1145, Level 2: ~$0.2255, Level 3: ~$0.0655. * **Neo4j + NetworkX (Query+DR):** (Second instance) Level 1: ~$0.1235, Level 2: ~$0.2285, Level 3: ~$0.0655. * **Neo4j + NetworkX (Query+DR):** (Third instance) Level 1: ~$0.1235, Level 2: ~$0.2285, Level 3: ~$0.0655. * **Simple RAG:** Level 1: ~$0.1145, Level 2: ~$0.2255, Level 3: ~$0.0655. * **GraphRAG:** Level 1: ~$0.1235, Level 2: ~$0.2285, Level 3: ~$0.0655. * **Matgraph-One:** Level 1: ~$0.1235, Level 2: ~$0.2285, Level 3: ~$0.0655. * **HF GPT-4o mini:** Level 1: ~$0.1235, Level 2: ~$0.2285, Level 3: ~$0.0655. * **HF GPT-4o:** Level 1: ~$0.1235, Level 2: ~$0.2285, Level 3: ~$0.0655. (Note: The annotation "Max: 3,403$" likely refers to a cumulative or total cost not directly shown by the average bar height). ### Key Observations 1. **Performance Leader:** The method "Neo4j + NetworkX (Query+DR)" (first instance) achieves the highest total number of solved tasks (~71), with strong contributions from both Level 1 and Level 2. 2. **Cost Leader:** The "GPT-4o mini (Zero-Shot)" method has the lowest average cost per task, but also solves the fewest tasks. 3. **Performance-Cost Trade-off:** There is a clear inverse relationship. The zero-shot LLM methods (GPT-4o, GPT-4o mini) have very low costs but moderate-to-low task completion. The graph-based and hybrid methods (Neo4j, NetworkX combinations) solve significantly more tasks but at an order of magnitude higher average cost (around $0.10-$0.23 per task vs. $0.001-$0.015). 4. **Level Contribution:** For most methods, Level 1 (cyan) and Level 2 (blue) tasks constitute the vast majority of solved tasks. Level 3 (purple) tasks are solved in very small numbers across all methods. 5. **Baseline Comparison:** The gray "Baselines" bars are generally lower than the best-performing methods in the left chart, indicating the evaluated methods offer improvements. 6. **Cost Consistency:** The average cost for the graph-based and hybrid methods is relatively consistent within the $0.10-$0.25 range, regardless of their total task performance. ### Interpretation This data suggests a fundamental trade-off in the evaluated systems between **effectiveness** (solving complex tasks) and **efficiency** (cost per task). * **Zero-Shot LLMs (GPT-4o/mini)** are highly cost-efficient but limited in their ability to solve the full spectrum of tasks, especially higher-difficulty (Level 3) ones. They are suitable for simple, low-cost applications. * **Graph-Augmented Methods (Neo4j, NetworkX, RAG variants)** demonstrate superior problem-solving capability, particularly for Level 1 and 2 tasks. The hybrid "Neo4j + NetworkX" approach appears most effective. However, this comes at a significantly higher operational cost, approximately 10-100 times more per task than zero-shot LLMs. * **The "Max: 3,403$" annotation** is critical. While the *average* cost per task for a method like HF GPT-4o is shown as ~$0.23, this annotation implies that the *total* cost for running that method on a full benchmark or workload can be extremely high. This highlights the importance of considering both average cost and total cost of ownership. * **Strategic Implication:** The choice of method depends on the application's priorities. If maximizing task completion is paramount and budget is available, a graph-augmented hybrid system is preferable. If minimizing cost is the primary driver and some task failure is acceptable, a zero-shot LLM is the better choice. The data does not show a method that achieves both top-tier performance *and* top-tier cost efficiency, indicating a potential gap or a necessary compromise in the current technological landscape.

## Bar Chart: Model Performance Comparison Across Tasks and Costs ### Overview The image contains two side-by-side bar charts comparing the performance of various AI models across three categories: "Zero-Shot," "KGoT," "KGoT (fusion)," and "Baselines." The left chart measures "Number of Solved Tasks" (y-axis) across three performance levels (Level 1, 2, 3), while the right chart measures "Average Cost ($)" on a logarithmic scale. Models include GPT-4o, GPT-4o mini, Neo4j + Query + DR, NetworkX + Query + DR, RDF4J + Query + DR, Simple RAG, GraphRAG, Magnetic-One, and HF GPT-4o mini. ### Components/Axes - **Left Chart (Number of Solved Tasks)**: - **X-axis**: Categories: "Zero-Shot," "KGoT," "KGoT (fusion)," "Baselines." - **Y-axis**: "Number of Solved Tasks" (0–70). - **Legend**: Top-left, with colors: - Level 1: Light blue (#87CEEB) - Level 2: Dark blue (#0000FF) - Level 3: Purple (#8A2BE2) - **Models**: Listed below x-axis (e.g., GPT-4o, GPT-4o mini, Neo4j + Query + DR, etc.). - **Right Chart (Average Cost $)**: - **X-axis**: Same categories as left chart. - **Y-axis**: "Average Cost ($)" (log scale: 10⁻³ to 10⁰). - **Legend**: Top-right, with colors: - GPT-4o: Pink (#FFC0CB) - GPT-4o mini: Purple (#8A2BE2) - Neo4j + Query + DR: Light purple (#E6E6FA) - NetworkX + Query + DR: Medium purple (#9370DB) - RDF4J + Query + DR: Dark purple (#800080) - Simple RAG: Light blue (#87CEEB) - GraphRAG: Medium blue (#0000FF) - Magnetic-One: Dark blue (#0000FF) - HF GPT-4o mini: Pink (#FFC0CB). ### Detailed Analysis #### Left Chart (Number of Solved Tasks) - **Zero-Shot**: - GPT-4o: 10 (Level 1), 13 (Level 2), 4 (Level 3). - GPT-4o mini: 17 (Level 1), 2 (Level 2), 0 (Level 3). - **KGoT**: - GPT-4o: 33 (Level 1), 24 (Level 2), 4 (Level 3). - GPT-4o mini: 29 (Level 1), 28 (Level 2), 2 (Level 3). - **KGoT (fusion)**: - GPT-4o: 34 (Level 1), 29 (Level 2), 4 (Level 3). - GPT-4o mini: 27 (Level 1), 28 (Level 2), 2 (Level 3). - **Baselines**: - GPT-4o: 18 (Level 1), 15 (Level 2), 2 (Level 3). - GPT-4o mini: 13 (Level 1), 13 (Level 2), 1 (Level 3). - Neo4j + Query + DR: 21 (Level 1), 16 (Level 2), 3 (Level 3). - NetworkX + Query + DR: 21 (Level 1), 18 (Level 2), 2 (Level 3). - RDF4J + Query + DR: 20 (Level 1), 15 (Level 2), 1 (Level 3). - Simple RAG: 18 (Level 1), 13 (Level 2), 0 (Level 3). - GraphRAG: 13 (Level 1), 18 (Level 2), 0 (Level 3). - Magnetic-One: 13 (Level 1), 20 (Level 2), 1 (Level 3). - HF GPT-4o mini: 22 (Level 1), 31 (Level 2), 1 (Level 3). #### Right Chart (Average Cost $) - **Zero-Shot**: - GPT-4o: $0.017 (Level 1), $0.001 (Level 2), $0.001 (Level 3). - GPT-4o mini: $0.098 (Level 1), $0.135 (Level 2), $0.145 (Level 3). - **KGoT**: - GPT-4o: $0.155 (Level 1), $0.199 (Level 2), $0.148 (Level 3). - GPT-4o mini: $0.091 (Level 1), $0.145 (Level 2), $0.129 (Level 3). - **KGoT (fusion)**: - GPT-4o: $0.155 (Level 1), $0.199 (Level 2), $0.148 (Level 3). - GPT-4o mini: $0.091 (Level 1), $0.145 (Level 2), $0.129 (Level 3). - **Baselines**: - GPT-4o: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - GPT-4o mini: $0.145 (Level 1), $0.129 (Level 2), $0.006 (Level 3). - Neo4j + Query + DR: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - NetworkX + Query + DR: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - RDF4J + Query + DR: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - Simple RAG: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - GraphRAG: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - Magnetic-One: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). - HF GPT-4o mini: $0.165 (Level 1), $0.232 (Level 2), $0.006 (Level 3). ### Key Observations 1. **Performance Trends**: - **KGoT (fusion)** consistently outperforms other models in "Number of Solved Tasks," especially in Level 3 (e.g., GPT-4o: 4 tasks, GPT-4o mini: 2 tasks). - **HF GPT-4o mini** shows the highest cost in the right chart ($3.403), far exceeding other models. - **Baselines** (e.g., Simple RAG, GraphRAG) have lower performance and cost compared to KGoT variants. 2. **Cost Anomalies**: - HF GPT-4o mini has the highest cost ($3.403) despite moderate task performance (31 tasks in Level 2). - GPT-4o mini has lower costs ($0.098–$0.145) but also lower task performance (17–2 tasks). 3. **Logarithmic Scale Impact**: - The right chart’s logarithmic y-axis compresses high-cost values, making differences between $0.001 and $3.403 appear less drastic than they are. ### Interpretation The data suggests that **KGoT (fusion)** models achieve the highest task-solving efficiency, particularly in advanced levels (Level 3), indicating superior adaptability or reasoning capabilities. However, this comes at a cost: HF GPT-4o mini, while performing well in Level 2 (31 tasks), incurs the highest expense ($3.403), suggesting a trade-off between performance and cost. **Notable Outliers**: - **HF GPT-4o mini** stands out for its high cost despite moderate task performance, raising questions about its cost-effectiveness. - **GPT-4o mini** balances lower cost ($0.098–$0.145) with mid-tier performance (17–2 tasks), making it a potential candidate for budget-conscious applications. **Underlying Patterns**: - The "KGoT (fusion)" category consistently outperforms others, implying that fusion techniques (e.g., combining query and DR methods) enhance model effectiveness. - The logarithmic cost scale highlights the exponential disparity in expenses, particularly for HF GPT-4o mini, which may not justify its performance gains for all use cases. This analysis underscores the importance of balancing task efficiency with cost constraints when selecting AI models for deployment.