# A Multi-Agent System for Semantic Mapping of Relational Data to Knowledge Graphs
**Authors**: Milena Trajanoska, Riste Stojanov, Dimitar Trajanov
institutetext: University Ss. Cyril and Methodius, Skopje, N.Macedonia email: {milena.trajanoska,riste.stojanov,dimitar.trajanov}@finki.ukim.mk
Abstract
Enterprises often maintain multiple databases for storing critical business data in siloed systems, resulting in inefficiencies and challenges with data interoperability. A key to overcoming these challenges lies in integrating disparate data sources, enabling businesses to unlock the full potential of their data. Our work presents a novel approach for integrating multiple databases using knowledge graphs, focusing on the application of large language models as semantic agents for mapping and connecting structured data across systems by leveraging existing vocabularies. The proposed methodology introduces a semantic layer above tables in relational databases, utilizing a system comprising multiple LLM agents that map tables and columns to Schema.org terms. Our approach achieves a mapping accuracy of over 90% in multiple domains.
1 Introduction
In today’s data-driven enterprises, information is often distributed across multiple siloed databases, limiting the ability to fully leverage its value. Achieving seamless integration between heterogeneous data sources remains a major challenge, particularly due to differences in schema design, semantics, and data representation. Knowledge Graphs (KGs) [hogan2021knowledge] have emerged as a powerful paradigm for bridging these gaps by creating a unified, semantic view over disparate datasets.
Traditional data integration efforts, however, often require extensive manual work to align schemas and define mappings. Recent advances in Large Language Models (LLMs) [chang2024survey] provide new opportunities to automate complex reasoning tasks, including semantic mapping and knowledge graph construction. LLMs have shown impressive performance in tasks that involve understanding relationships between concepts, making them ideal candidates for assisting in structured data integration.
In this paper, we present a novel semantic multi-agent system that leverages LLM agents to automate the integration of relational databases into a unified knowledge graph. Our approach introduces a semantic layer above relational data by mapping tables and columns to concepts from the Schema.org vocabulary [schemaorg]. Each agent in the system has a dedicated role: mapping relational structures to semantic concepts, establishing relationships between entities, and validating the generated knowledge graph.
We evaluate our system using the Spider dataset [yu2018spider], a complex benchmark featuring databases from diverse domains. By mapping real-world relational schemas to Schema.org concepts, we demonstrate that our multi-agent system effectively automates the semantic integration process, achieving high mapping accuracy across several application domains. Our results highlight the potential of LLM-driven semantic agents to significantly reduce the manual burden of data integration and to build more interoperable enterprise knowledge bases.
2 Literature review
Recent research has explored various approaches to knowledge graph (KG) and ontology construction, and data integration, particularly in enterprise and industry contexts.
The Virtual Knowledge Graph (VKG) paradigm [xiao2019virtual] offers an alternative to traditional data integration approaches by providing flexible, virtual graphs that embed domain knowledge over existing data sources. The approach provides tools for Ontology-based data access, which have significant use-cases in multiple industry applications.
In the context of Industry 4.0 (I4.0) where different systems exist, the demand for the creation of an integrated view increases. One such initiative is the Bosch Industry 4.0 Knowledge Graph (BI40KG) [grangel2020knowledge] which was developed with the goal of integrating data from different sources, improving interoperability and traceability across manufacturing systems.
Many efforts have been made for developing enterprise knowledge graphs (EKGs), in line with this, an Enterprise Knowledge Graphs Framework [galkin2016integration] was created, which aims to bridge the gap between the increasing need for EKGs and the lack of formal methods for realising them.
The use of Large Language Models (LLMs) in structured information extraction has gained significant traction recently. Findings suggesting that LLMs excel at reasoning tasks, and are being applied to information extraction tasks more often [xu2024large].
Our previous work in the field compares general-purpose LLMs like ChatGPT with specialized models such as REBEL for joint entity and relation extraction [trajanoska2023enhancing]. Using sustainability-related texts, pipelines for automatic Knowledge Graph construction were built. Results show that LLMs can improve accuracy in extracting structured knowledge, and foundational models also show promise for automated ontology generation.
3 Methodology
This section describes the methodology for creating the semantic multi-agent system as well as the data source used in evaluating the approach. The section is divided into the following subsections: Data description - stating the dataset used for evaluation and the semantic vocabularies, Graph-vector store constuction - detailing the process of constructing a hybrid vector store containing subgraphs in a vector representation, and Multi-agent system architecture - detailing the entire flow of the mapping system along with the AI agents’ roles.
3.1 Data description
The Yale Spider database [yu2018spider] is a benchmark dataset used in semantic parsing and text-to-SQL tasks, featuring real-world relational data from various domains. The Spider challenge aims to create natural language interfaces for databases spanning multiple domains. It includes 10,181 questions and 5,693 distinct complex SQL queries across 200 databases with multiple tables, representing 138 domains.
In this study, the database serves as the foundation for evaluating our multi-agent approach, allowing us to demonstrate the effectiveness of mapping relational data to a knowledge graph.
For creating the mappings, we utilize existing semantic vocabularies, which are defined and maintained by established organizations over a long time period. The reason for this is to avoid hallucinating terms that do not have definitions in any well-known ontologies.
With the goal of including as many general concepts as possible, we use the Schema.org vocabulary [schemaorg], which represents a collection of schemas (types and properties) that can be used to markup content on the Web, enabling search engines to better understand the context of Web pages. It describes entities, their relationships, and actions, and can be readily expanded using a clearly defined extension framework. As of 2024, more than 45 million websites have used Schema.org to annotate their pages, resulting in over 450 billion structured data objects [schemaorg].
3.2 Graph-vectore store construction
To enable our use case for generating correct semantic mappings, we have created a custom graph-vector index for retrieval augmented generation [lewis2020retrieval] using the term definitions in Schema.org.
The index creation process is depicted in Figure 1. For each term, we extract its URI, type, human-readable comment, and label, as well as the domain and range for property terms. Based on this information, we construct a one-hop subgraph for each term. This enriched subgraph offers a more detailed semantic context. Finally, each subgraph is transformed into a vector representation using a configurable text embedding transformer model [patil2023survey].
<details>
<summary>images/schema_vector_store_horizontal.png Details</summary>
### Visual Description
## Data Flow Diagram: Schema Processing Pipeline
### Overview
The image is a data flow diagram illustrating a schema processing pipeline. It depicts the stages of loading, transforming, embedding, storing, and retrieving schema data, culminating in a machine learning application.
### Components/Axes
The diagram consists of the following components, arranged from left to right:
1. **Load Schema:** Represents the initial stage of loading schema data.
2. **Transform:** Represents the transformation of the loaded schema.
3. **Embed:** Represents the embedding of the transformed schema.
4. **Store:** Represents the storage of the embedded schema.
5. **Retrieve:** Represents the retrieval of the stored schema.
Each stage is connected by arrows indicating the flow of data.
### Detailed Analysis or ### Content Details
* **Load Schema:** A blue network graph icon represents the initial schema.
* **Transform:** Three overlapping light blue rounded rectangles contain a JSON-like structure:
```json
{
"@id": <term_id>,
"@type": <rdf_type>,
"comment": <rdfs_comment>,
"label": <rdfs_label>,
"domain": <schema_domain>,
"range": <schema_range>
}
```
* **Embed:** Three overlapping light blue rounded rectangles contain numerical values: "0.2, 0.56, -2.15, ..., 5.22".
* **Store:** A teal cylinder represents a database or storage system.
* **Retrieve:** A light blue rounded rectangle labeled "Retrieve".
* **Final Stage:** A teal brain-like icon with circuit patterns represents a machine learning application.
### Key Observations
The diagram illustrates a sequential process where schema data is loaded, transformed into a structured format (JSON-like), embedded as numerical data, stored, retrieved, and finally used in a machine learning context.
### Interpretation
The diagram outlines a typical workflow for preparing schema data for use in machine learning models. The transformation step likely involves converting the schema into a standardized format. The embedding step converts the schema into a numerical representation suitable for machine learning algorithms. The storage and retrieval steps ensure the data can be accessed and used efficiently. The final stage indicates the application of machine learning to the processed schema data.
</details>
Figure 1: Graph - Vector store construction with ontology sub-graphs
The generated vectors are stored in a vector store. The subgraphs are retrieved from the vector store during the table mapping process, depending on their semantic similarity to the columns and values in the table being mapped.
3.3 Multi-agent system architecture
The system consists of multiple GPT-4o-mini [hurst2024gpt] agents with different responsibilities:
1. Mapping agent: Maps columns to corresponding Schema.org terms.
1. Relation agent: Determines primary and foreign keys in tables, creating links between entities in the knowledge graph.
1. Validator agent: Validates and corrects the mapping and relations produced by the two previous agents.
The mapping process is fully represented in Figure 2. The process operateos in a table-wise fashion. For each table, the Mapping agent receives the table name, all its column names, summary statistics for each column’s values, a sample of the first K rows, and any available textual descriptions for the table or its columns.
<details>
<summary>images/semantic_agents_flow_small.png Details</summary>
### Visual Description
## Data Flow Diagram: Knowledge Graph Creation
### Overview
The image is a data flow diagram illustrating the process of creating a knowledge graph from various data sources. It shows how data from databases, CSV files, and unstructured documents are processed through a series of agents and algorithms to ultimately form a knowledge graph.
### Components/Axes
* **Resources (Left Side):**
* Database (represented by a cylinder icon)
* CSV files (represented by a document icon labeled "CSV")
* Unstructured documents (represented by a document icon with lines)
* **Processing Agents (Center):**
* Mapping Agent (blue square icon with a brain/gear symbol)
* Relation Agent (blue square icon with a brain/gear symbol)
* Validator Agent (blue square icon with a brain/gear symbol)
* **External Resource (Top-Center):**
* schema.org (red rectangle)
* **Transformation Algorithm (Center-Right):**
* Transformation Algorithm (two interlocking gears)
* **Output (Right Side):**
* Knowledge Graph (purple node-link diagram)
* **Flow Arrows:** Arrows indicate the direction of data flow between components.
* **Labels:** Text labels describe each component and the data being passed between them.
### Detailed Analysis or ### Content Details
1. **Resources:**
* Database: A generic database icon.
* CSV files: Represented by a document icon with the label "CSV" and the text "csv files" below.
* Unstructured documents: Represented by a document icon with lines, labeled "Unstructured documents".
2. **Mapping Agent:**
* Receives input from: CSV files and schema.org.
* Outputs: "mapping result" to the Validator Agent.
3. **Relation Agent:**
* Receives input from: Unstructured documents.
* Outputs: "primary and foreign keys" to the Validator Agent.
4. **Schema.org:**
* Provides input to the Mapping Agent and Validator Agent.
5. **Validator Agent:**
* Receives input from: Mapping Agent, Relation Agent, and schema.org.
* Outputs: Data to the Transformation Algorithm.
6. **Transformation Algorithm:**
* Receives input from: Validator Agent.
* Outputs: Data to the Knowledge Graph.
7. **Knowledge Graph:**
* Final output, represented by a purple node-link diagram.
### Key Observations
* The diagram illustrates a pipeline architecture for knowledge graph creation.
* Multiple data sources are integrated and processed.
* Schema.org plays a central role in the mapping and validation process.
* The flow is sequential, from data sources to the final knowledge graph.
### Interpretation
The diagram depicts a common approach to building knowledge graphs. It highlights the importance of data integration, schema mapping, and validation in creating a structured representation of knowledge from diverse sources. The use of schema.org suggests a focus on semantic web technologies and standardized vocabularies. The process involves transforming raw data into a structured knowledge graph, which can then be used for various applications such as data analysis, information retrieval, and reasoning. The diagram emphasizes the flow of information and the transformations applied at each stage.
</details>
Figure 2: Data flow of the semantic multi-agent system
In addition, the mapping LLM is provided with the most similar terms from Schema.org to the table and column in order to map them accordingly. The most similar terms are retrieved from the custom graph-vector store described in the previous subsection.
In the subsequent step, all mapped tables are integrated into a unified knowledge base by accurately identifying primary and foreign key relationships through the Relation agent. This structured integration enables seamless interaction with the knowledge base by both internal and external systems.
In the final step, the Validator agent inspects and refines each table mapping and the final relation mapping to ultimately produce a more accurate result. The Validator agent may remove or re-map any relations it deems necessary to improve the accuracy.
All the prompts for the different types of agents are available in our GitHub repository [milena_trajanoska_knowledge_graph_agents] under the prompts folder.
All of the agents in the system output a confidence variable which can be one of the three following cateogories: HIGH, MEDIUM or LOW, to describe their confidence in the completion of their dedicated task. The categories are averaged from all steps to produce the final confidence class.
4 Results and discussion
The mapping system was evaluated on multiple databases from different domains present in the Spider data source. The accuracy of the generated mappings was assessed through manual validation, evaluating whether the Schema.org terms chosen by the system correctly reflected the semantics of the corresponding database elements. The evaluation dataset can be found in our GitHub repository [milena_trajanoska_knowledge_graph_agents_eval] in the eval folder.
This section summarizes the findings for the entire execution of the system, including the time to generate the mapping result, as well as the accuracy of the mapping process.
Table 1 shows that the execution time generally increases with the number of tables and columns being mapped. This is likely due to the complexity of larger tables and databases in the relationships they model.
Table 1: System execution time based on the number of tables and columns being mapped.
| Number of tables | Number of columns | Execution time (sec) |
| --- | --- | --- |
| 3 | 10 | 122 |
| 6 | 25 | 179 |
| 6 | 31 | 191 |
| 11 | 42 | 342 |
| 12 | 47 | 295 |
Concerning the validity of the generated mappings, we have conducted an evaluation for five different fields, namely: retail, movies, automotive, apartments, and delivery. We gathered all the datasets from these domains present in the Spider database and manually evaluated the accuracy of the mappings in each domain.
The accuracy of the multi-agent system across the five domains is reported in Table 2. Apart from the mapping result, the multi-agent system returns the confidence of the mapping process, which can be one of the following categories: HIGH, MEDIUM, and LOW. For all confidence levels, the results are displayed in the table.
Table 2: Accuracy of the multi-agent system on the five chosen domains presented by human evaluators
| Retail | 78.72 | 95.83 | 64.29 | 55.56 |
| --- | --- | --- | --- | --- |
| Movies | 90 | 85.71 | 100.00 | / |
| Appartments | 93.54 | 100.00 | 88.89 | 50.00 |
| Automotive | 84 | 100.00 | 80.00 | 25.00 |
| Delivery | 80.95 | 96.30 | 66.67 | 0 |
The highest accuracy (93.54%) was observed in the ’Apartments’ domain, while the lowest (78.72%) occurred in the ’Retail’ domain. Even for retail, the score is still satisfactory. It is evident that the mappings with certainty equal to HIGH are, in general, the most correct ones.
5 Conclusion
This paper introduced a semantic multi-agent system for enterprise data integration using Knowledge Graphs and LLMs. By aligning relational schemas with Schema.org, our approach enhances interoperability and enables unified, semantically rich graphs across domains.
LLM agents were assigned specialized roles—mapping, extraction, and validation—resulting in a scalable and automated integration pipeline. Experiments on the Spider benchmark showed high mapping accuracy and cross-domain applicability.
The main contributions of this work are:
- The design and implementation of a semantic multi-agent system for automating relational-to-graph data integration.
- The innovative use of LLM agents to perform semantic mapping and reasoning over structured data using existing vocabularies.
- The empirical evaluation on a real-world benchmark, validating the system’s practicality and generalization capability.
Future work includes extending the system to support custom domain ontologies, enhancing the reasoning capabilities of the agents through fine-tuning, and investigating scaling strategies for integrating larger and more heterogeneous datasets.
6 Acknowledgements
This publication is based upon work from COST Action CA23147 GOBLIN - Global Network on Large-Scale, Cross-domain and Multilingual Open Knowledge Graphs, supported by COST (European Cooperation in Science and Technology, https://www.cost.eu.