NODES AI 2026 - Complex Querying of Cyber Knowledge Graph for Robust Cyber Defense

This talk discusses the development of a knowledge graph aimed at improving cybersecurity operations by reducing the time between vulnerability discovery and mitigation. The speaker, Mahantesh Halappanavar, along with a team from Pacific Northwest National Labs, explains how vulnerabilities like Log4Shell can lead to rapid exploitation, highlighting the need for efficient information processing. The knowledge graph integrates data from various sources, including the Common Vulnerabilities and Exposures (CVE) and Common Weakness Enumeration (CWE), to build a comprehensive understanding of cybersecurity threats. Advanced techniques such as path queries, vector embeddings, and graph-based retrieval allow for more effective exploration of vulnerabilities and attack patterns, helping cybersecurity professionals prioritize and address threats in real-time.

Thank you very much, Hannah. Hello everybody. It is my great pleasure to present the work of a very amazing team here. So we also have several people joining us today. Joe Aguayo is my co-conspirator here and he's the deputy chief information security officer. And I'm Mahantesh Halappanavar. And I'm the chief data scientist and group leader at Pacific Northwest National Labs. So we have an amazing team here.

So Mokshudur Rahman, Adi Mukundan, Aaron Sanchez, Renuka, Luis Dela Torre and Siddharth did a lot of work on the research side. So Tina, Samantha and others have also helped quite a bit on the operation side. So Joe will cover some of this towards the end. And our goal is to minimize the time between vulnerability discovery and the mitigation it takes. So for example, the Log4Shell flaw was

widely exploited in the first two hours. We It saw probably about 40,000 attempted hacks. And within the first three days, there were close to a million attempts to compromise a system. So what it really tells us is that there is a significant amount of information that comes in that needs to be processed very quickly and mitigation mitigative actions have to be deployed very efficiently. So for some

reason the graph is not showing up. So what Tzu observed is that if you want keep on fighting for a hundred battles, then you better know yourself very deeply and you also know your enemy very well. So then you don't have to really fear the end result. And what we believe is that cybersecurity is a continuous operation. And what we want to enable is this ability to

perform self-awareness, get the real threat intelligence, and also conduct exploratory research. And we'll quickly go over the construction of the knowledge graph and what the knowledge graph has. And we will quickly go what are the different ways of building the path queries. And then we'll look at some of these newer approaches that help us still answer the same questions and answer the same queries when information really

does not exist in the knowledge graph. And we'll finally touch a little bit on the operational side. And some of this work we presented in November of last year 2025. So there is a small QR code on your top right that you're welcome to go in and get some more details on the knowledge graph itself. So here we'll just provide a very quick overview of what the

knowledge graph consists of. So we consume four main sources of information. And the first body of work is the common vulnerabilities and exposures, the CVE database that was established by MITRE in the late 1990s. as of this week, so we have close to 3 and 1/2 thousand 100,000 CVEs that are really out there. And these are all the known vulnerabilities. And there can be many more that

we do not know about, the zero-day attacks that can happen. And vulnerabilities are really the bugs that exist in the software software and the protocols or in hardware that can be exploited by a malicious actor or an adversarial actor. And the description of how to do it, how and how a vulnerability can be exploited, are very carefully described in in another database called as the common weakness

enumeration. So which can also be used very efficiently for fixing bugs for developing mitigating actions and so on. So there are close to a thousand CWEs right now in the in the database. So both of these data sets are very actively maintained in the graph that did not show up. So we basically plot how how fastly the vulnerabilities are growing. And we see roughly an exponential growth

and in 2025 there were close to a 50,000 vulnerabilities that were disclosed just as just in one year. And this is all on the side of vulnerabilities. So what happens? How do we exploit these vulnerabilities is is detailed in the common attack pattern and enumerations. And there are close to 560 CAPECs. And these describes what are the common attributes and how these adversaries exploit a known weakness

in a cyber system. And again cyber system is very broadly defined. It can consist of software, hardware, protocols and so on. And when we look at these individuals that described in CAPEC but when we take them in a connection and then establish how these advanced persistent threats happen on the network scale. For example, you could have 14 different tactics. You could do it could perform reconnaissance and

then you can perform once you get in you can do lateral movement and eventually you want to take the data out. So there are 14 different stages that are described in attack. And it provides a common framework and it provides a lot of details on which threat groups are active, what weaknesses they are exploiting and how we can mitigate some of these actions. And these are called

as TTPs. And and the presentation from last year covers much of this in quite detail. And the pipeline we have built is to design is designed to consume either on a daily basis if the information is very active like CVEs and some other attack information and so on. You can consume it on a daily basis, but if the information comes at you at slow rate like CWEs

and attacks, so then you can consume them on a weekly basis. And the data is ingested and it is processed and it is extracted and converted into rows and relationships or entities and relationships and it is fed into the Neo4j instance. And as of now the knowledge graph is quite extensive and we have close to we have over 620,000 CPEs of the platforms information and then we

have 336,000 CVEs and then we also perform a different variety of automated tools to link. For example, NVD database itself gives us certain links from CVEs to CWEs, so that way once we know the CVEs, we can traverse to CWEs and then eventually to mechanisms and so on. I'll have a few examples showing why we want to do this. But then the database itself gives us a

certain number of connections, a certain number of these relationships. But a lot of these are also discouraged or discontinued in a certain way. Like so the information can get stale. So we developed V2W BERT, which is an automated mechanism to perform these links in a very systematic manner because CWE is also a hierarchical schema. So we add over 350,000 edges ourselves and there are also a lot

of these other information that you can see and there is a lot of interconnection that is involved here as And we have another tool VWC bird that uses frontier models and then it establishes connections between CWEs and CAPECs. So that way you are have this ability to traverse through the knowledge graph. And what happens if you don't use the knowledge graph at all? Then we can simply

ask LLMs these questions. For example, you can ask it find the CAPEC that are related to CVE-2022-26069. modern frontier LLMs like Google for mini will come back with a response. So sometimes these responses can be very good, but sometimes they don't really match what we are required in in a in a response. So a rag-based response can come back with a lot more details. And sometimes for

more complicated questions, the information can be there, but it might not be really related or very helpful. So in which case a rag response can be very informative and it can give you also evidence for why it is particularly giving you a certain suggestion or So a rag-based system is a lot more powerful than simply asking an LLM that does not have the retrieval ability. So that

is our presumption. And what happens when we know the knowledge graph? When when the knowledge graph is full and we know all the information, there's not enough missing information per se, then we can perform a lot of these path-based queries. And for example, you can traverse from a given CVE to a given attack mechanism. So you know you have a particular CVE in your system and you

want to know which threat group or which or how it might be exploited or who would be interested in exploiting it. You want to ask these traversal questions. So you start from a and then you traverse to a CWE and then you traverse to a CAPEC. So, how the how a given attack mechanism is going to exploit a given weakness. And then you can go back and

see which which exact APT was using this particular CVE this particular CAPEC. Uh so, there are certain information that you can simply extract. And you can also traverse back. You can start from an attack and then you can come back and ask the questions like, "Okay, which CVE is being need to really protect?" For example, you know a certain threat group is active right now and you

want to go back and see if your system really has all these vulnerabilities that need to be protected. Uh so, the path-based queries can be very effective for these types of queries. And you can also ask a little bit more detailed questions. For example, given a particular uh TTP, which groups are actually using this mechanism? And what tools are they using? And you can also ask what

mitigating actions can be performed. Uh so, all these information can be very effectively extracted. And in some cases, you can also conduct uh very specific groups like, you know, which threat group is active, what weaknesses are being exploited. Uh so, for example, in 2023, MGM Resorts was breached uh by a financially motivated threat group. And uh we can begin from that threat group and go back and

see which uh techniques they were using, which CAPECs were being uh eventually used for exploitation, and which weaknesses were exploited. And in fact, we can traverse one more level into the actual vulnerabilities themselves, which is what would exist on your system. Uh so, in which case you can get a lot of information back. And we have several other mechanisms to uh trim down this information, focus only

on what is more important for you and so on. Uh we'll get to it in a little bit. Yeah, but eventually many of over a period of time and they're very complex. And therefore, the ability to extract information in a complex manner is extremely helpful. So that is the main idea. And in fact, uh we begin to see, okay, what happens if there was a missing link?

If there is no link between a given CVE and a CWE. Uh so that should not really uh impact too much because we can also use the structural properties that exist in the knowledge graph, as well as the semantic information. We'll come to the semantic information next. Uh but the graph structure itself can be used and you can start predicting some edges that currently do not exist,

but you are predicting them. Uh and uh the Neo4j graph data science library provides several uh algorithms for doing this. Uh so one simple approach can be that if you uh two given uh sets of entities have a lot of neighbors in common, so you can sort of do these triangular closures and you can see uh that um there's a very strong relationship, even though they don't

really have a direct relationship between them. Uh so the strength of common neighbors can be a good indication that there is probably a potential edge. Or you can also use uh other uh schemes like the Adamic-Adar metric, which tries and and uh makes gives more weightage for rare neighbors. If a neighbor is rare and if you have a connection to that rain a rare neighbor, then it

is uh it's going to be more important for you. And there are several other algorithms that you can use for prediction. And these predictions uh exploit structural properties, but not Uh so what do I mean by semantic information? Uh so for example, for a given CVE, uh so there are a lot of details and in particular there is a text description of what this vulnerability is. I

think it also has links to CPEs. It has important scores and so on but uh critically it also has a very good description. And that can be exploited and that's what we'll cover in the next step. Uh and we also want to provide a very simple intuitive interface for the knowledge graph. As you saw in the previous examples, the Cyphers can start getting very complicated and we

do not expect a lot of our users to know Cypher in that great detail that they can master these complex queries. So that's where GraphRAG is very effective. Uh and we also wanted to establish how good is our knowledge graph, how well can we extract information which sort of goes back and and depends on our own tools of how we are establishing relationships. So we created a

set for evaluation, a cyber question and answer data set. Uh and the question data set has about 600 cyber security questions and about 50 to 60 of these were developed by subject matter experts from Joe's team. And we used LLMs to generate a set of very similar questions to these human generated questions broadly covering six different topics. Some of them were very simple direct fact retrieval while

others can also engage in multi-hop reasoning. So we have 100 questions in each of these six categories. Uh and the way we work with the GraphRAG is to see we pose the question and we also have context information. We also have a human evaluated solution. What what is the expected answer for each of these questions and we also have some references that we can give. So all

of this goes in as input uh for uh for the retrieval process And then we perform the Cypher generation automatically. And the Cypher generation itself uh, consumes the graph schema. It also has a lot of other prompting and help and uh, we give it some examples and so on. Uh, so we have uh, tested it for the 600 queries. And once it comes back and then it

uses the context to generate the response. And the uh, tool sort of looks like this. Uh, so a user can enter a query. And then uh, it does the processing. It generates the Cypher. It extracts information from the knowledge graph. And then it uh, takes the context and the and the query back to the LLM and then we generate a response. And uh, we also evaluated. Yes,

it looks good. So how much can we actually evaluate uh, manually? And uh, human interpretation can also can be very subjective. Uh, so we use the Ragas framework for coming up uh, with the different metrics for evaluation. Uh, so for example, we feed a question and we know what we are expecting as a subject matter expert. And we uh, also see what the graph rag generator produced.

And we evaluate for accuracy and for relevance and for completeness. Uh, so for example, in this case the expected answer was that we wanted to avoid plain text storage of secrets. But the graph rag response did not really contain it. So it was giving a little more complicated answer. So the completeness can be marked down if that is the case if we miss something in uh, like

that. Uh, so out of the uh, 600 questions we were trying uh, we ran on a subset and we had a few errors that came uh, which were execution errors for now. We are debugging it. Uh, but overall we see uh, very good accuracy for the rag responses. And very relevant responses that we uh, are expecting as a human subject matter expert. And we also see a

good completeness for several different classes, but also for some sections like Intel intersection, we see very small completeness score. And the reason for is for the for the lack of accuracy or low accuracy and low completeness is that our knowledge graph current does not does not have the And so as a as a consequence, the graph rag is just coming up with a regular LLM based responses.

Uh and we are trying to figure out how to address these incompleteness equations. And we move on to the vector embeddings. I mentioned about using semantic information that we have rich textual information for vulnerabilities, weaknesses, and CAPEX, So we use Gemini to do embeddings for each one of them. And we wanted to see how good these embeddings are. So the next few slides sort of cover how

good the embeddings themselves are. And what we can see is that once we extract something similar, so we ask for a given CVE, give me the top 10 or top 15 very similar CVEs. So now the similarity is based on the semantic information, and we use cosine similarity. We can also use other metrics. And we can see that the retrieval is very effective, and we can come

back and cosine similarity score is very good. And we can also confirm through the mapped CWEs. So if two of these extracted CVEs, the neighbors that we extracted, also have the same primary CWE, that means that the semantic information was very well captured by the embeddings. And we see that there is about 97.5% similarity rating. And this is a compared with the random comparison. If we randomly

extracted a couple of what is the chance that with the they they will have the same CWE. So, that is about 15% and that ability to guess over a random is called the lift. So, if the lift is 2x if the lift score is two, that means that you are twice as likely to find the right information compared to just randomly finding something. And we keep seeing

that the uh the lift scores are quite high and the retrieved information is very effective. So, that now gives us a lot of confidence in going and performing vector embedding based searches as well because the links were missing themselves, so we cannot use path base. And we also performed some clustering to basically test how good the vector embeddings are and they can also be very effective if

you do not really we have the time and ability to process all the 500 vulnerabilities that came back to you. You can look at that cluster and only extract a few um sample CVEs from that cluster. And we can uh start seeing so we used two different approaches for clustering. InterSoma which is a self-organizing map based scheme and TSNE which is a multi-dimensional uh scaling approach. And

we used TSNE we uh further trimmed down the um dimensionality reduction trimmed down the scale of the vector length and we performed K-means. And we start seeing that the um the median dominant CWE share was quite high compared uh for the TSNE and K-means compared to the InterSoma approach. So, they both have certain advantages and disadvantages and we can perform many other schemes of clustering as well.

And uh we also see see a significant for the CWE clustering of the CWEs themselves. As I mentioned that there is a strict hierarchy here. Uh so, we also see significant lift over uh for the for retrieved for the um uh vector embedding based retrievals we see significant lift. As in like it can be very good compared to a random clustering of certain information. It's exploiting the

semantic information is And then we also we can also now perform these searches based on semantic information. So if we do not really have all these information, we can see what CVEs are related to CVE-2025-5688. So we can go in and say okay, use cosine similarity, use the vector embedding that we have already given it and then it comes back with really good results that we can

evaluate further down. So that's what we were able to see. And moving on to the next scheme of querying. So now vector embedding searches and all are good, but if you go beyond a certain number of hops, the information can get very distracted because the language that is used in CVEs is fairly different from the language used in CWEs. And which is quite different from how the

language is used in the CAPEX. So we wanted to explore other mechanisms to ask these same queries when we have missing information and it can also be multi-hop in a way. So there is a very good work in this area. So the first such approach was TransE, which sort of embeds each entity with its own embeds the entity using a vector representation and it also embeds relationships.

And the question you want to ask is given edge, an entity and a relationship, you want to take it closer to your target entity. Let's say you're asking a asking a question. So for a given CAPEC, which CWE might be representative? Let's say you take the CAPEC entity and then we want to ask a question. The relationship will be that this CAPEC-63 exploits CWE-79. Let's say once

I take this vector H and combine it and operate it with relationship vector R, we should get something that will be very close to CWE 79. Right, so the target vector here is CWE 79. So, we have to learn all these embeddings, so that's the learning process. And this is the TransE approach. And in Query to Box, they do something even better. So, where they learn relationship

not as a vector embedding but as an offset for the box. So, entity embeddings can be considered as a D-dimensional vector of zero-volume boxes, but the relationship vector produces a new box from a given box. And the idea is that I take an entity and I apply the relationship operator on it, and it gives me a new box. And now I can check for all the entities

that exist in this box. Right, so which happened to be in that point because entities are all zero-volume boxes, so they'll be just points. And all the entities that exist in this new box are my are my what the solution I'm looking for. Right, so my results of my query. And now because you have these boxes, you can also perform several operations like intersection. So, I can

take three different entities and ask three different relationship questions, and I can find what is in the intersection. So, the entities that exist in the intersected areas are my solution or my target entities. there is also a mechanism where we can actually combine and convert all these into disjunctive normal form, where you basically ask a question in the form of ORs of ANDs. So, this gives us

a great ability to check embed uh the embeddings for different entities and ask these questions. Like, you know, which CAPEC exploits which CWE. And uh what we saw in our uh initial preliminary exploration is that we found very good results. Um and then, again, you can ask a few more. So, we can embed uh several different types of entities. We can also embed several different types of

relationships. And we are also trying to see how we can actually scheme uh scale both of these mechanisms to the scale that we uh now have. Uh so, that's where we are headed. Uh so, with that, I will transition to uh Joe. Uh so, who is taking all of this research and putting it into operations. >> Very definitely. I think the big thing that this pushes the

ball forward with is that you have live updates, so that the graph is continually updated and stays fresh. So, I think in cybersecurity, you have this phenomenon where you could have everything perfect one day and go to sleep, nothing changes in your network, and then you have all these vulnerabilities and problems that were disclosed overnight. And so, having that timeliness and that constant refresh is a key

part of this. I think another thing that's really important is when you get sources of threat intelligence and, you know, indicators that there might be issues or things happening in the the world, they're they list out CVEs specifically in the text sometimes, but this work allows us to branch out from just the enumerated items and look for related ones, too. So, if the person is like rushing

to make a deadline to publish their article by a certain time, or they don't have time to do all the background research or stuff like that, it kind of frees you up from being dependent on just the text of the threat intelligence and looks for like the topological similarities and also the semantic similarities to other issues that might be related or might cause concern. I think probably

the third piece of that is the connection between the external data, like was mentioned here, and the internal data. So, cyber defenders have access to a lot of internal non-public data sets. They have a lot of telemetry. They have scan results from, you know, IT tools and security tools and vulnerability scanners and things of that nature. And so, being able to connect those two graphs really adds

a lot of power and control because you take the universe of cyber knowledge and you're able to drill it down to only the things that are relevant to your own environment for your corporate network. So, it allows you to prioritize the incoming threat intelligence and then kind of set up for what you want to invest resources in and what you want to take action on or then

kind of sets the stage for like, you know, agentic type remediations or agentic cyber defenses. So, it's a really powerful tool that really makes a difference for operational network defense. >> Thank you, Joe. And we have several sponsors that have funded this work throughout. So, it's been active for several years. And I will leave the screen on while we take a few questions with some of our

recent publications here and some of them go really way back into 2021 and 2022. That's when we develop the tools for linking vulnerabilities to weaknesses and weaknesses to CAPECs. And we exploit whatever information is already provided to us from these databases. So, I see one question from Varun. So, we actually create the schema by hand. For now, we have not used an LLM. And many of these

databases that exist like CVE, CWEs, they have a very good schema to begin with. And we curate all of that and it is constructed very carefully. That's why the knowledge graph is quite powerful. But there can also be a lot of irregular data sets like as Joe is mentioning. So, cyber incidents reports, they can run into millions to tens of millions in number. So, in which case

yes, an LLM can be definitely exploited to generate some of this data and it will be fed back into the system. Uh Joe, also please if you can monitor the question and answer. Okay. Uh so, how many dimensions are you reducing to 2D via PCA? I yes, so that's a very good question. Um so, I believe Maksudul would be the right person to see what was the

embedding uh size he used for the query to box. Oh, um Maksudul, I'm not able to hear you. Maybe you're in a different Oh, yeah. Yeah. >> So, the current dimension we are using one is 1,024. >> Okay. >> Yeah. >> Thank you. So, from 1,024 we have and the embedding size for the Gemini was about 3,000. >> And are there public knowledge graphs like this? >>

Uh yes, I believe there are several other tools that also provide the same feature. Uh so, one in particular is the brown BR OAN knowledge graph from MIT. And for us, the main ability comes from using some of our own work that employs uh tools like beta W BERT, VWC BERT, and we are also developing many other schemes where we can make these relationships really uh of

good quality. And like I mentioned, CVE to CWE information can exist as is, but it can also be about 80,000 of them are just uh discouraged or discontinued weaknesses. So, it is not very meaningful per se. Can you provide the links? Yes, definitely we'll be happy to give the links. Okay, one question I did not understand. um so for the question from Brian, we try to address

this questions like, yeah, how can you actually use this? So, that was the part that Joe conveyed. And because Joe is also on the cyber operation side, Joe and Samantha, so they really need uh very quick information, very diverse information. And we also believe that the cyber information has a long tail. As in there's just a lot of diversity to information. Uh so, you want to have

this ability where you can ask very different uh types of questions to the database. So, that's what we are aiming uh aiming to build through this data we have. And uh for the other one, yes, uh we are not actively engaged in doing adversarial uh analysis ourselves. We sort of consume whatever information comes to us. So, that would be a completely different body of work. Um yes,

uh how does the entity box compare to graph embeddings? Um so, they are completely different embeddings as in like you learn them separately, right? So, they through the learning process it sort of learns what would be the right. And so, for each embedding it starts from a random initialization and then it slowly moves from there and it converges based on your training examples. So, uh yeah, um

So, what this uh we get the data from National Vulnerability Database from uh data repositories that CWE maintains. So, if you just Google CV CWE, you'll find the links uh to the relevant uh link to the relevant repositories. Uh so, also our 2025 presentation had a few more details and the links from where to find the information. Yes. Uh so, it's the same repositories. So, some of

these repositories uh so, for Ritesh, uh yes, the the repositories from where we are getting the information is uh active. They're all active uh repositories where information is fed in continuously. Uh so, there are close to 500 um naming authorities that can keep feeding in Um so, we sort of consume uh all that information that uh the whole community across the world is developing. And uh we

pull in the information on a regular basis. You can change the setting and you can pull in on hourly basis as well if you need to. Uh so, that's that's what we are doing. So, thank you very much for all the So, I believe uh Hannah, we have answered all the questions. Probably it's time to stop sharing and leave the stage. to Samantha Muksadur, thank you very

much and Hannah, thank you so much. >> Mhm.