NODES AI 2026 - Agentic GraphRAG: Autonomous Knowledge Graph Construction and Adaptive Retrieval

This talk presents a project called Agentic Graph RAG, which addresses the challenges of multi-hop question answering in AI retrieval systems. The speaker explains how traditional retrieval methods struggle with complex queries that require contextual understanding across multiple documents. The proposed solution leverages knowledge graphs, utilizing entities and relationships to form a connected network that enhances reasoning capabilities. However, constructing these graphs can be labor-intensive and costly, leading to the development of an autonomous system using multiple agents for knowledge graph construction. The speaker outlines a seven-agent framework designed to extract and evaluate entities and relationships, presenting test results that show the effectiveness of a hybrid retrieval system that combines both vector-based and graph-based methods. The findings indicate that the hybrid approach significantly outperforms traditional methods and offers promising results for various fields that require accurate information retrieval.

Hello everyone. My name is Atulya. I'm a grad student here at UMass Amherst and this is my project agentic graph rag. So multi-hop question answering has been a very well-known problem in this field. Most AI retrieval systems are built for simple one question uh answering. For example, when was Apple founded? So you just retrieve the document which has this answer and you're done. But in real world,

questions are rarely that simple. Uh for example, uh at the arena where Lusedern played, how many people does it seat? So step one for this would be finding which arena Lusedern played in and then step two would be finding uh its seating capacity. So two documents, two hops, but one question. This is called a multi-hop question uh and then uh to answer this multi-hop question, the standard

retrieval fails because the standard retrieval sees one chunk at a time and doesn't have the context of the whole doc uh corpus itself. So that's where um knowledge graphs come into play uh and we'll look into how the system um solves that um question. So let's look into what a traditional rag is. A traditional rag splits documents into sentences, converts it into vectors, stores the vectors, and

then finds semantically similar um vectors to your query so it can answer to your particular question. Um so you just have to find the right document uh and then you have your answer. It's great at finding facts, it's fast, but then it's conceptually blind. So that's where graph rag comes into place. Um how graph rag works is basically it has entities and relations and the graph is

stored in like a knowledge graph DB like for example Neo4j. Um so the way it will be stored is for example let's say Apple which is a company is an entity and we have Steve Jobs who is a founder and then the relationship will be very clear and very visual in this graph. So the strength is that it treats the knowledge as a connected network and it's

great for multi-hop reasoning. The limitation is that it can easily go wrong and it's quite fragile because if any entity in this whole multi-hop is missing then the connection fails and it just goes goes cold from there. So it is context aware it can answer for the whole entire corpus but then it has its cons. but it's still a good sign right? But but then still why

isn't everyone using it? So the number one issue is that it's quite expensive to build. In a vector DB you just dump text dump the embeddings and save it and it's cheap and fast. But then in a graph you must extract the correct entities the correct relationships and so because of that the ingestion is very slow and it's very labor intensive and often it requires people who

have a lot of domain expertise to identify and make the graph and that is very time intensive as well. So our system addresses this by building a knowledge graph agentically by inferring the schema directly from the documents. And when it comes to latency vector search is very fast. It's in milliseconds but then graph it take time because it has to again traverse across multiple paths. So, a

graph rag is really it can come in handy especially in fields where you need the exact information. You need very accurate information. For example, health care, finance, legal. And it's good with global summarization and it can answer multi-hop questions. But at the same time, it needs to be very properly constructed because if you miss any one entity, it the retrieval fails. since just building the knowledge graph

itself is very labor intensive, I was wondering if we can automate it somehow. So, that's when I came up with the system, the agentic graph rag. And with this with the help of multiple agents, I am able to build a knowledge graph which is of a decent quality. You can't still compare it to the ideal graph which is hand built by people, but it's still decent and

it provides good results. And once you've constructed an via agents, then later maybe a specialized person can come in and fix the things. But for this experiment purpose, I haven't done that. I built it only using agents and to experiment and see the results. So, in this particular pipeline, I have seven different agents. First agent is a schema inference agent which infers the schema from the documents

and identifies the entities and relationship types. Then comes the entity extraction agent and the relation extraction agent and the schema alignment agent. Initially, I had a setup where I didn't have so many different agents. I just had two or three. Just the entity extraction and schema agent. But the quality of the knowledge graph that was built was not very great and the performance wasn't that good. That's

when I introduced a seven-step system and the performance has significantly improved. We also have a conflict resolution agent. For example, let's say there's a fact that a particular company was founded in a year and there's another fact saying contradicting that the year in which it was built. So, the conflict resolution agent debates between these two facts and comes to a conclusion on which one's the right thing

and then saves that information. We also have a confidence evaluation So, only uh things with a higher confidence uh gets stored. So, this way our knowledge graph was constructed autonomously using agents. Parallelly, I've also used sentence transformer and uh created a vector DB and uh stored all the document text as embeddings in it to compare the results. So, this is what the knowledge graph uh looks like.

And we can query this by using a simple Cypher query. Cypher query is very similar to SQL and it's easy to learn uh it's very like you can interpret it uh just by looking at it. So, if you look closely, uh Doctor Strange is one entity and it is connected different things like the year, Walt Disney, Marvel Studios, Marvel Comics, etc. So, uh So, now we have

our system. We have our vector DB uh things stored in our vector DB and our graph which is autonomously built. So, I wanted to test different retrieval uh methods to find out which is the best retrieval. So, we have a vector retrieval, a graph retrieval, which is graph traversal, and the vector retrieval uses a dense retriever, and a hybrid, which is a fusion of both of these.

And then I've also have an orchestrator. Right now, the orchestrator just routes queries based on very specific simple rules, but in the future, we can, after we have a lot of data, we can train it to route particular queries to those retrieval, which will give us the best results after learning from the results. I also have a few agents to track the performance. And this was the

initial test setup. Took a small corpus of 25 documents, 25 queries, and its related documents. And the these are the basic stats for nodes and relationships of the graph, and then the vector embedding size. And the results are in. Hybrid has outperformed both the vector and graph. So, these results are interesting I expected graph to do better, but graph actually fails more often than we think because

the entities are not perfect. Like I mentioned earlier, if the very first entity is missing, like it can't really hop on anything, so it just goes cold. Or sometimes, when we are hopping multiple times, if there's one entity in the middle that's missing, it goes cold again. vector is just a straight dense semantic similarity based um, retrieval, and the hybrid is actually the best of both worlds.

So, how hybrid works is that vector runs first, uh, it searches for semantically similar sentences, and gets the correct entity. And once the entity, uh, is at hand, the graph expands upon it. And once we have that, uh, we combine it, deduplicate it. Um, so, basically, graph and vector both has its uh, strengths. And hybrid is a good combination, and it it covers both. It misses It

catches things with, uh, which hybrid, um, sorry, vector misses, and it catches things which graph also misses. So, um, that is a conclusion. The hybrid, uh, is actually doing pretty well, uh, with an exact match score of 68%. Uh, outperforming both the vector and the graph-only retrieval. And the other key finding is that the agentic graph, uh, rag which was built, the results are very similar to,

um, ideal knowledge graph which was built. Um, so, I think the results are like pretty comparable. And the ceiling for this another thing which I've noted was that the ceiling is, um, we have an LLM answering layer. That particularly sometimes, even if you have you give it the right answer, um, right document, if you fetch the right document with the retrieval, and give it to it, it

still sometimes gives us the wrong answer. It's because of various reasons, like LLMs hallucinating and things like that. So, that's the ceiling, but, um, but overall, this we found that hybrid retrieval is a very good method to go ahead with and it outperforms both. And that concludes my presentation.