About this talk
In this talk, Niya Macklin discusses the importance of context engineering in artificial intelligence, particularly how it can improve the reliability of AI agents. She introduces the concept of structural similarity, explaining how traditional retrieval methods focus on text similarity but often overlook important contextual relationships between data points. By incorporating knowledge graphs into AI systems, engineers can provide agents with relational knowledge that enhances decision-making capabilities. Macklin emphasizes that better models alone do not solve issues of context, and she presents techniques for integrating structured knowledge into AI applications. The session includes a live demo showcasing how context graphs can inform decision-making within financial services, emphasizing the need for transparency and accountability in AI systems.
Full transcript
Hello everyone. Oh, look at that. Good morning. Good morning. Um all right, well, it's afternoon now, but welcome everyone. So, uh thank you so much for coming to my talk. We have a little bit of time together, so I want to really get into it. Um but, I have a first question for you all. So, my first question before we even dive in is how many of
you have ship already shipped an agent to production? What? Just a few? Okay, a lot less hands than I thought, but that's probably fine. Don't worry, I'm going to teach you how to do that work um a little bit later. Um how many of you have had that agent do something in production that has made you question your career choices? Okay, you poor soul. Me too. Me
too. It's rough out here, right? Um okay, so let's dive in. So, first of all, my name is Niya Macklin. Um I am a researcher of graph and graph algorithms, um as well as artificial intelligence and context engineering. So, I'm a a software engineer who leads technical communities, um and I like to think critically along with my fellow engineers. I like to think critically about how to
center ethics, how to center governance, safety, and responsibility in the work that we develop every day. So, um I've been in this game doing computer vision work and facial recognition work long before this big uh mainstream bust of artificial intelligence around 2022. Um but, I also know that my teaching style is not everyone's cup of tea, right? So, I like to make sure that when people are
learning with myself up on the stage that they're having memory mechanisms that help you remember this talk, right? And it's very important that you're not just sitting here getting lectured at, but instead engaging with the um with the material so that you're better remembering and better learning along the way. So, this is going to be a collaborative effort. Um that's the way I like to do my
uh my talks, and I hope that you gain some real uh insight from this conversation. So, um let's get started. Okay. Well, the slides didn't really work, but that's okay. All right, so we have three different table stakes here. Jessica works at Apex Global. This is the the different different chunks that your agent is actually going to be taking in. So, a compliance team is going to
run an agent in a pipeline, right? The agent is then going to retrieve three separate chunks. So, Jessica works at Apex Global is one chunk. The second one being that Apex Global itself is on the sanctions watch list as well. And then the third chunk that Jessica requested a $25,000 credit line increase. So, all three of these chunks have been received and all three are sitting right
there in our context window. So, the agent then approved this uh this credit line increase and we can say, "Oh, right, you know, it doesn't seem too bad of a decision that that agent is going to make." However, right? That's this decision is not bad because either the model itself is bad or it's not bad um uh simply be- because of that reason. However, the connections between
these facts was actually not represented anywhere. So, we had these three separate chunks. The model was not able to put them all together uniformly to understand that this is actually the big picture for this particular case. And so, it just approved it, right? It just approved it all willy-nilly. And so, the problem wasn't the data itself. The problem wasn't the retrieval process itself, but the model had
to guess and the model then guessed wrong, right? And so, nobody caught this until, as you can see, the lawsuit happened, okay? So, now we put ourselves in the engineer's standpoint, the engineer who built out this agent, your senior leadership is now running down the halls and searching for you, okay? And you're like, "I I'm so sorry. I did not understand how this agent worked." The your
senior leadership is asking you, "Why did your agent approve this increase?" You have no idea. You have no idea this is a problem, right? And these are the true table stakes. We're seeing this type of situation happen time and time again, right? And and most of us were on Hacker News, we're on X, we're seeing these cases happen, right? Rogue agents are making decisions for us and
then like dropping production tables, right? Like all of these different things that have just happened within the last week. This is massive and the scenario actually keeps landing on my desk. And so, what I'm going to talk to you today is to show how that happens, the techniques to fix that process, and then prove it all in a live demo, as well. So, you as the engineer
say that you don't know what happened and now you have been fired. We're trying to avoid this situation. So, a little bit of background to get us all all um all primed for this. So, text similarity specifically finds documents that have a similar meaning, right? And then structural similarity, on the other hand, finds entities that have similar connections, okay? So, almost nobody is building that second half.
So, I want to give you this thesis up front so that you know exactly what I'm trying to argue. So, every retrieval pipeline that people are using, either rag or a hybrid search or re-ranking, right? All of those operate on one specific dimension, text similarity. And how close is this one chunk to the meaning of this query? And that does work, it gets you about 70 or
80% of the way there, but there's the second dimension that nobody's really studying and nobody's really building, right? The structural similarity piece. So, not do these documents mean the same thing, but actually are these entities connected in a similar way. So, if we think of this concept that I was talking about before, the credit line increase, right? The credit line increase for a corporate banking client, and
then this employer is listed on the federal sanctions watch list, those two do not mean the same thing. Hence, a regular vector search is not going to bubble those up to be in the same area, right? Your vector search is never going to surface that when you're looking for one or the other. The relationship is between Jessica in general. Jessica, her employer, and then those sanctions list.
And that's not a meaning problem, that's a connection problem. These things are not connected together. And so, right now, like I had mentioned, almost nobody's agentic pipeline can see those specific connections. So, I want to get into this dimension of context engineering that most people have not built. And before we even dive into that, the research that backs this up at scale, right? MIT's Nanda Initiative and
their 2025 GenAI Divide Report found that 95% of um of enterprise AI pilots are delivering no measurable P&L impact, right? 95%. That's a ridiculous amount, right? We're investing months of engineering time, and then the project is getting killed because there's no clear bar for quality, there's no clear bar for privacy, and there's no clear bar for transparency as well. So, this central gap that MIT had identified,
systems are not retaining feedback, they're not adapting to context, they're not improving over time, as well. And Gartner, specifically as well, is predicting that 40% of agentic AI projects will be canceled by 2027. So, the models aren't the problem, the context around those models is a central issue. Yes, and I do advise you to read the the white paper yourself. That's one thing that I'm going to
do throughout this whole thing, provide you the research so that you could read it. Never just blindly trust anyone who's up here talking to you. Always read the papers yourself. So, this that's one of the white papers there. Okay, so I want everyone to understand that better models do not fix fractured context. This is what all of this has in common, right? Oftentimes your model's going to
retrieve by text similarity, and text similarity has a blind spot the size of a building when it comes to relationships, like I had mentioned before. Better models do not fix So, they just reason over the broken pieces, which is such a problem. So, this is the key. This is the things that I want you all to focus on. Step one, build a system that cuts hallucinations. Everyone's
trying to do that. Okay, there's there's no doubt that we're all trying to do that, but number one, step two is to force your agents such then show their work. You're wondering how? How do I go about that? >> [snorts] >> Let me show you what I mean, right? So, we have here I have a representation of an apple, and there's three different ways to represent it.
>> So, on the r- Oops, sorry. On the right, you're going to see this um On the right, there we go. Okay. Yep, my right, your right. >> [laughter] >> Um on one side you're going to see a vector representation of an apple, right? So, the vector representation of an apple is going to be uh pretty opaque, right? It's going to be a wall of floating point
numbers, and this is what your embedding model is picturing, right? Um this is what vector search specifically uh operates on. But then on the other side you're going to see the knowledge graph version of an apple. And so, the knowledge graph version of an apple, you see that the apple connects to a tree, which connects to a fruit, stem, uh every property specifically is what we call
a node within a knowledge graph. And every relationship is then explicit. You can see the actual relationships. A human can read this and understand this. And then that's exactly what the model knows about this apple. And then in the middle we have a human view of an apple, right? You just see that an apple in the brain instantly processes those relationships implicitly, right? You don't have to
think about it. You just know that an apple's round. You know that this apple's red, right? It's a fruit. It comes from a tree. We're not talking about the company. But here's the thing. The vector is going to capture this meaning. It knows that this is close to a fruit and then it's red and then it's food, but it's all still a black box. You can't inspect
that vector representation of an apple. You can't ask, "Does this model know that the apple is round?" You can't audit it, right? But a knowledge graph then captures that structure and the meaning itself. It's transparent. It's queryable, right? And to overcome AI's black box problem then we need that knowledge to be transparent. >> So going into the past. A lot of you came to this talk to
learn about context engineering. So can anyone give me a definition of context engineering? Not one hand. >> That's okay. That's totally fine. Well, >> [cough] >> Pardon me. sorry. Sorry. I'm just going to take a drink. Context engineering is a term that was uh coined by Dex Horothy. Anyone Dex Horothy or heard of Dex Horothy before? Okay. [snorts] No hands. That's fine. That's fine. Well, um Dex
Horothy, he's an incredible engineer, but he defined this discipline of systematically providing models with relevant information, relevant tools, and relevant instructions that it needs at the correct time, right? At the correct time, in the correct format, to accomplish a specific task. >> So, unlike prompt engineering, >> where we're trying to say the right thing to the right LLM, or you're trying to get your agent to do
what you want, right? Context engineering on the other hand, centers building dynamic systems that then assemble a concrete and structured uh context for each LLM invocation. So, this shift in focus, right? This shift in focus is why context engineering is now considered a critical skill for AI engineers. And let's see what that actually looks like. So, most engineers are using some combination of these methods within context
engineering. So, there are several different techniques that practitioners can use to improve the contextual relevance of your agentic development. One of them including a rag plus hybrid search, or naive rag. And that's a classic approach. It uses vector databases of embedded documents, um and then performs a semantic similarity search specifically, um to find the text that are related to that query, right? And so, however, pure vector
search on its own can retrieve off-target information. So, you have your target, it's going to It can retrieve off-target information, which results in what we call context poisoning, okay? So, context poisoning is exactly what leads to those wrong answers, or those hallucinations that a lot of us are seeing when we're building out agentic systems. So, we then try to combat that with using rag plus hybrid search.
So, using rag plus hybrid search is an excellent way to try to combat those hallucinations. However, there's still a trade-off. So, the trade-off there is that you have to decide what those splits and what those labels are within context engineering. So then we can also use memory management for example, right? So memory management I have represented by Dory, which is this really adorable forgetful fish. So memory
management is where the agent is going to interact with long agent um sorry, with a long session and across sessions. And it accumulates a lot of the information that we need to remember. So we use techniques such as sliding context window or recency-based memory for example, where you drop the older history, all right? You're dropping the older history to prefer the most recent or the most um
the most uh yeah, the most recent contextual history. But then again, there's a trade-off specifically. The trade-off is that your agent is going to lose those important details that it just deems older. So then again, we can try another method, structuring and ordering of context. So you can use context structuring and ordering and Anthropic themselves have uh provided a guideline for how to deal with long prompts.
So you can place the most relevant or critical information at the top of your uh prompt, and since models pay more attention to the beginning of the context window, you we try to remember that the most important you always put at the top, right? And therefore, we need order in the court. We need that order. But then additionally, we can use tools and function calling. And I'm
just getting into these things quickly before I actually dive into the one that I really want to talk about today. Um so function calling, which I have represented by MacGyver for those of us who are a little bit older. But instead of stuffing your context window with raw data or instructions to do complex calculations, we can offload those tasks to tools. And so for example, instead of
giving um instead of your giving your LLM a large table, right? And then asking a question about that table, which is going to consume all of your tokens, um it might be better to give that that LLM a database query tool, which would then um allow your agent to execute that query and get a summarized result of the uh summarized result to then feed into the context.
And by using those specific tools, we can then have that agent focus on what it's actually good at, which is reasoning, right? So, and while we're we're then offloading a lot of that heavy lifting. And so, for tools and function calling, again, remember MacGyver. Um and the many tools that he would use in order to um solve any problems. But, the one that I wanted to get
into today is this top one with Sherlock Holmes. Anyone seen Sherlock Holmes before? Such a good show. Yeah! See a fan in the back. So, Sherlock Holmes is represented by knowledge graph augmented context. Knowledge graphs, specifically, are an extremely extremely powerful tool, right? Um to introduce structured and relational knowledge into an agent's context. So, and that's why I like to always use Sherlock because he's able to
synthesize information and track down clues, like actually track down clues, uh to solve a problem. And this is exactly how knowledge graphs improve context engineering, right? When you integrate the knowledge graphs into your AI stack. But, how do they specifically work? And before [snorts] I go deeper into that that um that technique, I just wanted to show you a couple of the numbers that convinced myself, right?
That this is actually uh not just a theoretical concept. So, this is a white paper as well done by Jiang et al., um which is pub- published in the IEE Communications Magazine just in March, right? March of 2026. And they and the entire team built Com GPT. So, which was a domain uh specific foundation model for telecom. So, they ran an ablation study. And for those who
are not aware of ablation studies, ablation studies are used often in machine learning, um and are used in artificial intelligence to then determine the contribution of an individual component to a system's overall performance. So, they had used a ablation study on 3GPP question answering process. And based on model accuracy, 37% right? With the just the base model accuracy had a accuracy point of 37%. And then they
then fine-tuned that model on domain data and got an accuracy up to 54%. We would say that you know, that's an improvement. However, for those of us who are building out agents of production, that's just not enough, right? That's just not enough. And so when they then tested using both a knowledge graph and retrieval augmented generation together, they saw a massive increase of an accuracy point of
91%. So this is not just using a knowledge graph alone. This is not just using naive vector based rag alone, right? But it's both, right? Adding both together. So that's the critical point. Rag alone did not solve that gap, didn't close that gap, right? It was the knowledge graph plus rag together. So it went from 37 to 54 to 91% accuracy with a graph. And there are
hundreds of different white papers and research papers that are detailing the impacts that adding a knowledge layer has on common architectures, on common agentic architectures. I gave this talk down in San Francisco cuz I'm from San Francisco. And actually the author of the LinkedIn paper was in my audience and they said, "Thank you so much for showcasing our work." So that was really incredible. This is real,
right? This is This is not just fake, right? So really, And from an architecture standpoint, I'm just going to go through this quickly, everyone. From an architectural standpoint, having a knowledge layer in your AI stack will then allow you to number one, hold structure and meaning. Number two, create that uniform memory and the retrieval layer for your agents. Number three, link and unify those data sources and
objects. And number four, all while bridging the human and machine understanding together. >> And at the bottom, right? We have our existing data platforms, right? Whether that's Snowflake, Databricks, or your relational systems. And on top of that, we then have that knowledge layer piece, which connects all of our structured and unstructured data in a consistent and semantically rich way. And then from there, you can have your
GenAI apps and your orchestration layers like LangChain or Semantic Kernel, for or agent different agent frameworks, which can plug in and access that data contextually. And then that's where the knowledge layer fits, right? It's how you retrieve that grounded contextual information from the graph. And the benefit, specifically for agents, is that by adding that knowledge graph into the pipeline, when paired with the MCP architecture and the
schema of the knowledge graph, that now provides so much context, so much context, that we then break and when we specifically break down a user question or a prompt, that we start to see very clear and very direct logical flows of information that the that then go through your agent's response right back to you. >> So, the present and the future. Has anyone seen this You probably
haven't, but has anyone seen this paper up at the top? The AI's trillion-dollar opportunity for context graphs. No one? Okay, well, let me enlighten All these different papers, please please read them, right? Please read them. But at the present, the conversations are shifting from just using AI agents, specifically using naive rag to then using graph rag to then using knowledge graph augmented context, like I mentioned with
Sherlock Holmes, to now what we like to call this concept of context graphs. So, context graphs, you know, it's not just a simple change in terminology, but it's actually a a big shift, right? So, context graphs, it made waves on Twitter and Medium X, and when it was just released in December 2025. So, it just came out, but context itself, the why behind the decisions that agents
are making every day. This is becoming the pivotal piece, right? Because we're moving away from agents just making simple decisions about our lives, you know, just answering simple questions for us. And now we're moving now to agents actually both theoretical and hefty decisions about our everyday lives, right? Agents are now being used in medical terms, right? Making medical decisions and medical recommendations for people. Agents are then
being used to make financial recommendations, and we're going to get into that in the demo. Agents are being used medical, financial, lots of different impactful ways that are impacting our daily lives, and this is what's so important. And this is why we need our agents to be trustworthy. We need our agents to actually be able to describe the decisions that they've made throughout this process. Because if
you don't, you're going to end up just like the engineer who got fired a a bit just a little bit before. So, this is a bit of the the urgency I feel when it comes to building out agents. And so, we [snorts] can also talk about this missing why, right? Why do we need to understand the decisions that your agents are making? Because usually agents have this
this concept of no memory, right? You're asking an agent about last week's conversation, it has no idea what you're talking about. You also have this no audit trail that's happening normally, right? When you something goes wrong, no one knows why that agent had made that decision. Exactly the same that we saw before. And then, on the other hand, you have no shared learning. We're deploying multiple agents,
and they can't share what they've learned together. This is a vital problem. So, a context graph like I had mentioned before, a context graph is fundamentally a knowledge graph. Remember that apple that I showed? You have the vector representation and then the knowledge graph representation. A context graph is just that same as a knowledge graph that contains all the information we need to make decisions about our
organization. And so this differs from simply opening your agent to an audit log. You might say that audit log will suffice this, um, which would be a line-by-line transaction history, but your agent now, with a context graph, has all the knowledge from Slack channels, from messages, right? From your email threads, from Zoom meetings, all connected in a human-readable and machine-readable graph to show that full context and
to show why a single decision actually was made. And this is now going to include the causal chains and the decision traces that can actually be queried and traversed to ensure that your agents are more reliable and more trustworthy and more auditable. So, I have a financial services use case here, right? Anyone work in financial services? Oh, a couple. Lovely. Love it. Love it. Usually that's more
of a New York City type of type of question. [laughter] But, okay. So, in the state of model, we have what the graph world would call entities, and then, which is really just people, places, and things. And then we have these events, the things that happened, the what specifically. And then lastly, we have the context, the why, the decisions, the policies, etc. And that's the model. That's
the model. So, let's get into what this looks like in practice. So, here we have this context graph demo that Versed up app. I want you all to scan this QR code so that you can pull this down. It's a Git- GitHub repository, completely live, completely open source. At Neo4j, we firmly believe in open source. But, let's see this live. All right. So, depending on the Wi-Fi,
because Wi-Fi's absolutely trash often times. Okay. So, and this is super hard to see. But, the question that I want to ask, um, our context graph here is should we approve a credit line increase for Jessica Norris? She's requesting a $25,000 increase. So, we'll see what the Wi-Fi does. If not, I have it on a already populated. But, I want you to see how this context graph
is not only built, but also that we see the decision traces that are going to be populated on the right as well. So, that we can then dissect the decisions that this agent is actually So, this might take a little bit of time and if so, that's okay. I'm just going to describe what's happening here. >> So, all in all, how this is working is that we
are we have we're using a natural language search to search for customers, reviewing decisions, and analyzing the different patterns that are happening. And so, the agent is going to access this full context graph that's going to grow if the internet works for me. And then, your agent is going to traverse the specific nodes and edges to understand that context. And then, we can explore that context graph
when it generates and then visualize the entities and the relationships. This is going to take some time. So, just because it is, I want to make sure that you actually see it. But, the visualization is also very very cool. all right. I already did this ahead of cuz I knew the internet would be trash. as we can see here, this is a representation of that knowledge graph.
It's all live, all hosted here. And we can see that there's a number of different tool calls that happened in order to get a recommendation from that agent. The agent said that we should reject this credit line increase on a number of different red flags. Being this this this is actually the third request that Jessica has had. There's an unsolved fraud detection history that has happened where
she had 14 transactions happen within a within I think it was 2 minutes. And then, IP anomaly as well, high risk margin account, lots of different reasons. But all of this goes directly into the decision traces that we see on the right as well. So, we see that there was a a rejection a compliance rejection. We have about 83% confidence of that rejection. We have risk factors,
uh credit rejection as well, and then a fraud rejection um a rejection based on fraud as well. So, this is the decision trace that's happening on the right-hand side. But you can see why your agent has decided to make this this uh overall decision. And that's vital information. So, we are short on time, so let me make sure to get back. So, like I had mentioned, I'll
just go through this quickly. What's happening initially is that we're clearing the context graph. We're then tracing that decision history that we saw on the right. We're running a hybrid search, which is semantic similarity text embeddings plus the structural similarity. And for those of you who are graph enthusiasts, the fast RP uh graph embeddings. And then lastly, we're making a represent uh recommendation based on that information.
>> And I I don't have time to get into the the uh background behind the vector search the the hybrid search, but I wanted you to have the slide as well. >> Okay, and I was going to get into memory too, but we're already over. So, lastly, and for those who are new um or who want these slides, I'm happy to provide them. But where do you
learn more? An excellent place to learn more, everyone, is called the graph Academy. And so, graph Academy is completely free hands-on courses, hands-on keyboard. You're not just listening to a lecturer, but you're building as you go. You're building as you learn. So, I wanted to provide this. And actually, one of the founders of graph Academy is actually in the audience today, Adam Cowley. So, shout out to
Adam for his incredible work building out this learning platform for you all. YES, ADAM! >> ALL RIGHT. AND LASTLY, you can run this yourself. Create context graph. It's a terminal command you can run within your terminal UVX create context graph, and there is a QR code with the documentation to provide that. And there's lots of research, but I want to end with a thank you all for
your time. It's been a pleasure. Thank you for learning about context graphs with me. Take care.
