How to use large language models and knowledge graphs to manage enterprise data

In recent times, data graphs have grow to be an necessary device for organizing and accessing massive volumes of enterprise knowledge in various industries — from healthcare to industrial, to banking and insurance coverage, to retail and extra.

A knowledge graph is a graph-based database that represents data in a structured and semantically wealthy format. This might be generated by extracting entities and relationships from structured or unstructured knowledge, akin to textual content from paperwork. A key requirement for sustaining knowledge high quality in a data graph is to base it on normal ontology. Having a standardized ontology typically includes the price of incorporating this ontology within the software program improvement cycle.

Organizations can take a scientific method to producing a data graph by first ingesting a regular ontology (like insurance coverage danger) and utilizing a large language model (LLM) like GPT-3 to create a script to generate and populate a graph database.

The second step is to make use of an LLM as an intermediate layer to take pure language textual content inputs and create queries on the graph to return data. The creation and search queries could be personalized to the platform during which the graph is saved — akin to Neo4j, AWS Neptune or Azure Cosmos DB.

Combining ontology and pure language methods

The method outlined right here combines ontology-driven and pure language-driven methods to construct a data graph that may be simply queried and up to date with out in depth engineering efforts to construct bespoke software program. Under we offer an instance of an insurance coverage firm, however the method is common.

The insurance coverage business is confronted with many challenges, together with the necessity to handle massive quantities of knowledge in a means that’s each environment friendly and efficient. Data graphs present a approach to set up and entry this knowledge in a structured and semantically wealthy format. This may embrace nodes, edges and properties the place nodes characterize entities, edges characterize relationships between entities and properties characterize at-tributes of entities and relationships.

There are a number of advantages to utilizing a data graph within the insurance coverage business. First, it supplies a approach to set up and entry knowledge that’s simple to question and replace. Second, it supplies a approach to characterize data in a structured and semantically wealthy format, which makes it simpler to research and interpret. Lastly, it supplies a approach to combine knowledge from completely different sources, together with structured and unstructured knowledge.

Under is a 4 step method. Let’s assessment every step intimately.

Strategy

Step 1: Learning the ontology and figuring out entities and relations

Step one in producing a data graph is to review the related ontology and establish the entities and relationships which can be related to the area. An ontology is a proper illustration of the data in a website, together with the ideas, relations and constraints that outline the area. Insurance coverage danger ontology defines the ideas and relationships which can be related to the insurance coverage area, akin to coverage, danger and premium.

The ontology could be studied utilizing varied methods together with handbook inspection and automatic strategies. Handbook inspection includes studying the ontology documentation and figuring out the related entities and relationships. Automated strategies use pure language processing (NLP) methods to extract the entities and relationships from the ontology documentation.

As soon as the related entities and relationships have been recognized, they are often organized right into a schema for the data graph. The schema defines the construction of the graph, together with the sorts of nodes and edges that shall be used to characterize the entities and relationships.

Step 2: Constructing a textual content immediate for LLM to generate schema and database for ontology

The second step in producing a data graph includes constructing a textual content immediate for LLM to generate a schema and database for the ontology. The textual content immediate is a pure language description of the ontology and the specified schema and database construction. It serves as enter to the LLM, which generates the Cypher question for creating and populating the graph database.

The textual content immediate ought to embrace an outline of the ontology, the entities and relationships that had been recognized in step 1, and the specified schema and database construction. The outline ought to be in pure language and ought to be simple for the LLM to know. The textual content immediate must also embrace any constraints or necessities for the schema and database, akin to knowledge varieties, distinctive keys and overseas keys.

For instance, a textual content immediate for the insurance coverage danger ontology would possibly appear like this:

“Create a graph database for the insurance coverage danger ontology. Every coverage ought to have a novel ID and ought to be related to a number of dangers. Every danger ought to have a novel ID and ought to be related to a number of premiums. Every premium ought to have a novel ID and ought to be related to a number of insurance policies and dangers. The database must also embrace constraints to make sure knowledge integrity, akin to distinctive keys and overseas keys.”

As soon as the textual content immediate is prepared, it may be used as enter to the LLM to generate the Cypher question for creating and populating the graph database.

Step 3: Creating the question to generate knowledge

The third step in producing a data graph includes creating the Cypher question to generate knowledge for the graph database. The question is generated utilizing the textual content immediate that was created in step 2 and is used to create and populate the graph database with related knowledge.

The Cypher question is a declarative language that’s used to create and question graph databases. It contains instructions to create nodes, edges, and relationships between them, in addition to instructions to question the info within the graph.

The textual content immediate created in step 2 serves as enter to the LLM, which generates the Cypher question primarily based on the specified schema and database construction. The LLM makes use of NLP methods to know the textual content immediate and generate the question.

The question ought to embrace instructions to create nodes for every entity within the ontology and edges to characterize the relationships between them. For instance, within the insurance coverage danger ontology, the question would possibly embrace instructions to create nodes for insurance policies, dangers and premiums, and edges to characterize the relationships between them.

The question must also embrace constraints to make sure knowledge integrity, akin to distinctive keys and overseas keys. This can assist to make sure that the info within the graph is constant and correct.

As soon as the question is generated, it may be executed to create and populate the graph database with related knowledge.

Ingesting the question and making a data graph

The ultimate step in producing a data graph includes ingesting the Cypher question and making a graph database. The question is generated utilizing the textual content immediate created in step 2 and executed to create and populate the graph database with related knowledge.

The database can then be used to question the info and extract data. The graph database is created utilizing a graph database administration system (DBMS) like Neo4j. The Cypher question generated in step 3 is ingested into the DBMS, which creates the nodes and edges within the graph database.

As soon as the database is created, it may be queried utilizing Cypher instructions to extract data. The LLM may also be used as an intermediate layer to take pure language textual content inputs and create Cypher queries on the graph to return data. For instance, a consumer would possibly enter a query like “Which insurance policies have a high-risk ranking?” and the LLM can generate a Cypher question to extract the related knowledge from the graph.

The data graph may also be up to date as new knowledge turns into obtainable. The Cypher question could be modified to incorporate new nodes and edges, and the up to date question could be ingested into the graph database so as to add the brand new knowledge.

Benefits of this method

Standardization

Ingesting a regular ontology like insurance coverage danger ontology supplies a framework for standardizing the illustration of information within the graph. This makes it simpler to combine knowledge from completely different sources and ensures that the graph is semantically constant. By utilizing a regular ontology, the group can be sure that the info within the data graph is constant and standardized. This makes it simpler to combine knowledge from a number of sources and ensures that the info is comparable and significant.

Effectivity

Utilizing GPT-3 to generate Cypher queries for creating and populating the graph database is an environment friendly approach to automate the method. This reduces the time and assets required to construct the graph and ensures that the queries are syntactically and semantically right.

Intuitive querying

Utilizing LLM as an intermediate layer to take pure language textual content inputs and create Cypher queries on the graph to return data makes querying the graph extra intuitive and user-friendly. This reduces the necessity for customers to have a deep understanding of the graph construction and question language.

Productiveness

Historically, creating a data graph concerned customized software program improvement, which could be time-consuming and costly. With this method, organizations can leverage current ontologies and NLP instruments to generate the question, lowering the necessity for customized software program improvement.

One other benefit of this method is the power to replace the data graph as new knowledge turns into obtainable. The Cypher question could be modified to incorporate new nodes and edges, and the up to date question could be ingested into the graph database so as to add the brand new knowledge. This makes it simpler to take care of the data graph and be sure that it stays up-to-date and related.

Dattaraj Rao is chief knowledge scientist at Persistent.