NODES AI 2026 - Build Intelligent AI Agents With Context Engineering

This talk focuses on context engineering and its significance in the use of large language models (LLMs) and agents. The speaker, Michael Hunger, discusses the transition from simple prompting to more complex context management, emphasizing the importance of providing relevant information for effective reasoning in AI systems. The talk highlights how context can be sourced from various elements, such as conversation history, memory, and structured data formats, including JSON and YAML. The speaker also outlines the benefits of using graphs for context structuring, which allows for better navigation of information spaces and improves task completion efficiency. Additionally, challenges related to context management, including memory handling and tool output relevance, are addressed, along with potential solutions for effective context creation.

[music] >> Thank you, Ryan. Uh so, welcome everyone. So, this is my seventh notes and my first notes AI. Uh so, really excited to be here and as as Ryan said, I'm stepping in for someone who couldn't make it. Uh so, this is a very short notice presentation. So, please apologize if anything is not perfect. Um this is a presentation that I actually uh give a few

times uh last year before contest graphs came along and and and so on. So, I did a tiny bit of updates, but uh mostly it's about context engineering in in in general. Uh I'm Michael Hunger, I'm VP of uh product innovation and AI at Neo4j. So, working on the product side. Uh have been here for a long time as such and I do everything around graphs, LLMs,

MCP, and agents, and so on. I want to talk about a little bit why do we care about context engineering, uh what are sources of context, how does it relate to agents, and how can we use graphs for creating better context as well. Why is context engineering such an important thing right now, right? So, we started out with prompting, but now lots of people are talking about

context engineering, context engineering, agentic engineering, and so on. But why why is this important, right? So, basically uh prompt engineering was something like you did a one-shot request uh to an LLM, and then basically you got some answer back, right? But now nowadays actually it's much more about basically preparing the stage for complex tasks, right? So, when we are there, uh we want to take uh all

the relevant information that we need uh to provide to to the LLM or the agent to do the right reasoning, to take the right steps, uh to actually have a very narrow view of the world as such. Like I enter capacities that's here is kind of an art and science uh and that kind of put exactly the right things into the context, which is a repeating uh

pattern that you'll see today um uh for the next step as such. Uh so, why does it matter now? So, of course, as always in in engineering and data engineering, if you put garbage into a system, you get all the garbage out, right? So, that's also means kind of the quality of information that we put in and give to an LLM, which which you already have experienced

yourself, is a determining also what you get out. Uh there's more interesting work happening on dynamic prompts. How can we uh create prompts that are adapting based on on what is currently available, uh the user information, uh tools available, and and and so on. >> [snorts] >> if you look at something like chain of thought, that's almost like the LLM doing a little bit context engineering to

itself, because each LLM is basically what it does, it completes text. And it completes text based on what is already there. And of course, in an LLM at the beginning, there's a system prompt and the user question. But then, as you build up a conversation, all this conversation also becomes part of this context that the LLM tries to complete. So, you can almost use the context as

a as a steering mechanism. Where does the model collapses its answers, and where does it focus its attention, is basically that you something that you steer through the context. So, if you can look at this from this perspective, then something like chain of thought is almost like the LLM steering itself into the right direction of its whole uh kind of training weights as such, right? And these

days, especially since December, of course, uh we work a lot uh more with um coding agents, and we also know that, for instance, things like spectral development, agent engineering, or harness engineering are becoming more important, where we basically focus a lot on basically um working with the agent to get the right pieces of information for a task, uh all the right tools at the point, and then

execute the task in coding as such, right? And uh the nice thing about context engineering is also that it happens at at at inference time, at test time, not at training time. So, it's much faster, much cheaper. You can flip around, change things at a moment's notice, basically. So, you don't have to retrain a model or do things like that. It's something that you're super flexible in

terms of what do I want to put there. From that perspective, it comes basically from the few-shot examples and prompts as such, right? Something that's really interesting is also even if you have a million token context windows, better context doesn't mean better context, right? We have the needle-in-the-haystack and I'll talk a little bit about some of the issues that are arising in such context. And I think

this information architecture for humans, the better you structure the information that you put into the context, the better the results are as well that you get. So, as I as you mentioned, in prompt engineering, it was kind of you sent one question, one thing to the LLM. Sometimes you added a few examples, other things. Tried clever phrasing, you know, remember the my grandma's dying things and and

so on to get the LLM to answer you. Now, with reasoning models, this has mostly gone away because they've been trained on reinforcement learning, so they are actually much more task-oriented as such and completing tasks as such, so you don't have to bribe them anymore. But now it's basically we have more context available, so we can put more information there. It's it's more dynamic. We want to

focus on goals. What what's kind of the task or the step at hand that you want to do. We want to curate what we put in there. Is it relevant? Is it not relevant? Can we actually modify context so that irrelevant things are kept out of them or are compressed or condensed as well, right? So, can we put in structured output inputs? Things like, you know, can

we do put in um structured formats like JSON or YAML or other other other ways where we can basically give the information not just a wall of text, but can we have headers, subsections, annotations, things like that and focus on really having more signal and noise in context. So, that's why we for me it's kind of almost like thinking like information architecture for humans. And this kind

of thinking like for humans is also an recurring theme that I will have in this Um Lance Martin from uh from LangChain had a interesting point. He said basically context engineering is building dynamic systems that give the right information and tools in the right format so that the LLM can accomplish the next task at hand, right? So, that's basically how can we slice and dice the information

so that it's exactly has all the information. It's like like for the human, right? If you give someone a generic task and miss out some of the information that they need, then they can't fulfill the task. If you overwhelm them with too much information, they also can't fulfill the task. If someone like if you have an assistant or an a colleague that you want to achieve a

certain task, you give them a try to give them exactly the right amount of information to do this. And the same is what we do with LLMs as well. So, what are sources of context, right? So, what where does context come from? So, of course, traditionally context came from prompts. So, humans basically created templates from that perspective that achieved certain tasks. Um other information came from retrieval

augmented generation. Basically, you retrieve information from data systems where you of course also adhere to things like permissions, access control, other things. That's also where graph right comes from. There you want to go beyond something that vector search just returns text documents. You want to return and traverse structured information in in your database and make this available in a structured way to the LLM. There's the conversational

state and the history. You can look at things like you know, what has been discussed before. You don't want to repeat yourself every time that you talk to the LLM because it's already in the conversation, right? And then what happens then with the conversation state as it grows, if it reaches context boundaries and and so on. So, then this is where memory comes into play. Can we

basically store pieces of the information of the conversation as memory that is then later available to the LLM either actively so that the model can actually retrieve this with tools as memory as needed or something where in some systems you also inject memory at at the right points into the system. So you basically look at the task at hand, look at the memory system and see what

is relevant from the memory system and what's the right information to pull from the memory system to give to the agent. And then basically structured output from tools as you know agents have the ability to do tool calls. So whatever APIs, databases, other systems can provide structured outputs go also into the context. So there's a lot of sources of context. I've just broken here also a little

bit in what's coming from the human kind of the question task the prompt and then the dynamic prompts. But then there's also cognitive sources of context like for instance if there's meta information, there's guardrails from the system, right? Things that the model should not do productions from protections from prompt injection, things like that. Policies, chain of thought is also kind of the self-reflective cognitive thing where the

model brings in its own context basically. Judging of results so for instance if there were two calls, the model judges the results. Are they useful? Are they not useful? Do I have to call other tools? Uh Can I use this? Do I need to drill down further? These are kind of more cognitive sources. And then as I mentioned in the memory short-term memory which is kind of

just what is currently at at hand. But then also long-term memory where we can look at either textual like more episodic, what kind of conversations did you have in the past, what did you talk about? But also structured ones where we actually extract entities from from the memory and then use this kind of world model that's built up building up which was also discussed in the in

the opening panel as well. Can we do this? And then procedural memory which is more taking the agent interactions and storing them as part of the reasoning memory, but then also can I derive new procedures from that, right? So, like we do with, you know, with recipes, with patterns and other things, with guidelines and so on. Can we actually use memory to extract new learnings that then

can, for instance, be applied to new agents, right? So, imagine you bring a new agent online and can already access, you know, playbooks or other things that were derived from information that other agents collected And then of course the all the tools data MCP, so all the APIs and where you can get information from other systems, of course, while adhering to security and policy concerns. Which includes

rag and and graph rag as Um some of the challenges I already mentioned, large context is not kind of the solution, right? So, many people said, "Oh, rag is dead because we have large large context. We can just put a million tokens into the context." First of all, most data sources in the enterprise have more than a million tokens of content. And then you also don't want

to drown the model in into crap, basically, right? So, you have basically these long contexts lead to context words. So, if you accumulate more and more, you see the encoding agents with with rising, for instance, where you basically try to reset the context after each task so it has a clean uh set of tasks and information, but then all the other craft that has been accumulated go

goes away and you don't diffuse the model, right? So, there's a lot of confusion comes um uh in uh which also happens if you overload it with tools. So, which is really interesting, right? So, while you a few tools are useful, the more tools you provide, the model doesn't know which tool to use. It tries to use the wrong tools. So, things like tool filtering and and

and and push down or tool grouping or tool dependencies is something that's really interesting Clashing contexts, um especially if you have um contradicting information. Uh also in multi-agent system, what happens if multiple sub-agents return with data that's actually contradicting to each other? How does the supervisor model resolve these kind of things as well? And then um large uh context also uh lead to distractions. So, there's a

lot of uh things in this in this area as well. So, one idea I had was kind of something like a minimal viable context. So, what's the minimum amount of information that we can put into context to have efficient usage, correct task completion, and um basically efficient uh uh processing in terms of latency and so on. So, what do we have? We have basically system prompt, instructions,

we have the role of the model, uh constraints and information, examples from an example store for instance. So, we don't want to blow up the context with putting all the examples all the time in in in into the context as well. Then, what is the minimal set of tools that I need for this task? So, can I basically pre-filter the amount of tools that I have available?

I've seen agent system with, you know, hundreds or thousands of tools. if you can filter it down to what is actually relevant right now, uh that's uh important. And then, of course, the user request. So, and then we exclude all the the filler stuff that's not uh relevant, right? So, we want to have an high signal-to-noise noise ratio in in in the context as So, if you

go back to information theory, cognitive psychology, and and so on, you have you have basically what you can be holding in our short-term memory, right? What is the kind of size of our working memory? And if you can uh as humans, you can do the 7 plus minus 2, right? But, what's interesting is if you create structure, then suddenly we can get take can also keep seven

of the higher-level structure and seven of the higher-level structure as well. So, basically, by adding structure to it, we get much more total uh story or took total accessibility because you can suddenly navigate information. It's not just a wall of text, basically, but it's something that you can navigate and and and and go deeper into as well. So, again similar things as applied for humans in terms

of information theory and information architecture apply here as I already mentioned memory as a search. So, we have short-term memory, the the current context, the long-term memory, both structural when we extract entities and relationships, procedural and episodic. And we have the recent memory and the traces from the ancient executions. And then the question is kind of what is actually relevant of this memory, how much is needed

at at a given time, and how can we fill the gaps in memory? Right? So, because not everything is perfect and then the agent needs to decide, "Oh, actually my memory is incomplete here. I need to actually go back and ask the user, call some tools, and and and and so on." And what's interesting as well is how do we basically compress, distill memory as well. So,

there's some things that are talk about later. Connect structure, the information hierarchy, information structures. If headers, tags, kind of animal adjacent structures which give us substructures that the LLM can navigate. That's something that's really useful. If you have a local LLM that has file access on remote LLM with database access, they actually can proactively use these kind of tools to fetch exactly the right information that they

need so that you don't have to put it up front in and then so similar to to agent skills where you have in the front matter just an overview and then the LLM can basically use >> the information from the front matter to dive deeper and and get more information out of it. So, you kind of build up an information information hierarchy as Try to remove the

duplication. You should better to refer back to something that was is available somewhere else and then the LLM can or the model can go there and fetch this information from the other places such. And this also applies to tool output. So, if you can, try to deduplicate tool outputs. Right? So, don't try to try to return for instance from a database just a million rows if it's

actually just five rows of data and this was just an expansion or join and because of the join you have basically lots of duplicates, then try to aggregate this into sub-collections and have only like three lists with 10 elements each or something like that, right? And And instead of 1,000 rows basically. So, if you can aggregate and and and group things, then this is definitely preferred as

such, right? Um But sometimes it can be helpful to at the end have a summarization of the information so that the LLM is reminded of oh, what's actually there and can refer back to that as such. So, what are different approaches for offloading and and dealing with large context? The one is kind of something that coding agents do a lot or local agents do a lot. They

use file system, Obsidian, wikis, and other things to offload information. They can edit files, they can add things to files. You can do have plan You can use vector database for episodic memory. You can use graph databases or other structured databases like relation databases for more structured memories. But if you have highly connected information, you would of course use a graph there. So, you can offload it.

What's interesting is the agent can read and write. So, if it's more active in its context management, then you can also automatically consolidate things. So, you can compress memory. You can use a small LLM to summarize information from the context. You can also say, I actually just the summary, but still it stored the full information still in a file and then only provide a small summary to

the LLM. And if it wants to dive deeper, the original information is still there. So, you don't compress and throw away, but you compress, store [snorts] in on disk, but have then only the small summary or the key information available so it can use this almost like a database as such, right? Um At boundaries, especially if you have multi-agent system, you need to prune as well because

you can easily imagine if you have like an research system of tens of agents, each of them produces a lot of context. You immediately load off your context overload your context. So, each sub-agent also needs to basically already condense and summarize before it hands it back to the um to the main agent and and kind of determine what are the main findings from for instance from my

from from my research. something that's also interesting if you have existing memories and you know that these these are not complete, is there are there existing mechanisms that can enrich existing memories? So, basically they can say, "Hey, I identify gaps." and in the background something goes off, fetches this additional information and then enriches the memory as as you come in um And then for retrieval of memory,

you can of course retrieve information from third-party systems. You can go to Wikipedia. You can go to uh other data systems, Discord, and other other where you basically have all this the information available. You don't have to store everything that's available there. So, for instance, in Cloud Code they also said, "We will not build up a local vector search. We have all the code available. We can

always use grep and find to find the information." And the code is always the source of truth as well, right? So, because as soon as you build up caches and and and downstream stores, you always need to keep them in sync. So, basically having the source of truth with good tools for retrieval there can be much better from from that perspective, right? And um when you have

information that comes from different places, you want to consolidate it in in memory so that then you build up a busy on the presentation that puts things into place, right? And I've seen yesterday or day before there was something talking about even a memory palace for agents, right? So, you remember the spatial representation of information as And something you could also imagine this as like almost like

context primer that has like an foundation piece which is like system prompts and core facts and then dynamic elements and the ephemeral outputs are stacked on top of it. So, that's also a mental model how you can imagine this to be structured Okay, um what's also really interesting that information is not static, right? So, often times we see a book and the information in that book is

static, but in in the information age actually none of the information is static. Everything changes all the time, gets adopted, gets changed. We need to of course track and trace how information changes because if something changed under our feet, we need to notice it and also look at things like provenance and uh and and lineage of information. Um but what's actually interesting here is uh kind of

self-modifying systems, right? So, because if the information itself, the context is kind of the programming language of an agent, then by changing the context the agent can also change itself, right? So, and you've probably seen that in a in a few places where you have an agent out of open open claw or or others where you actually have uh the context uh that the agent um uses

is something that the agent can modify itself, then you can get to a self-modifying uh systems, which is a yet another level of interesting behaviors like you get to emergent behaviors and other things like that. So, that's something I think that we are still at the beginning of understanding what's happening there, but that's a very powerful way as needs to have some guardrails, I guess, because otherwise

it can go off also quickly. For agents, uh so originally we started with chatbots that had like one-shot retrieval basically where uh rag was more in a push principle and now with um uh context engineering it's more pull principle. Um so one interesting other point from LangChain was uh often times when the agent is not doing the right thing, it's actually not a model problem because modern

foundation models are actually pretty good. But the context is not right, right? So it's basically like you give someone a task to do XYC and you give them information that's either outdated or wrong and and then you complain about that it didn't do that task correctly, right? So and basically it's similar to that, right? The capability is there, but if the information is not there, then you

can't execute on that. If you look at the agent loop, there's actually multiple places where context plays a role. So in the reasoning phase where you basically get access to memory, to guidance, to metadata, to plan and act and break down information. In the action phase where you use tools to either take action and retrieve and get um insights. And then in the observation phase where you

basically look at the these results from the planning and from the tool retrieval and then you judge, okay, do I have everything that I can that I need to complete the task or generate the answer or do I need to do another loop in the agent loop? And if not, can I also basically use in the in the observation phase the learnings and compress this into memory,

compress this into context that is kind of persistent for the next loop as well. So this is actually really interesting to see that in all the three phases of the agent loop context and kind of external memory and storage is really really important as well. Kind of breaks this down again into the multiple phases here. So there are things like, you know, the LLM can extract parameters

from the conversation to do tool calls. There are things like, you know, accessing memory to during the reasoning or planning phase. So you can have I done this task before? If I have done this task before, what procedural information is available? How can I use make use of that? And and so on. And then in the observation phase is kind of how do I compress and update

and distill So there's more more to that. This is also something that I really like from Jason Lu and structure. And that I really nice context engineering CS. And he said basically context engineering is a tool that helps the agent to navigate a complex information space, which I think is really nice because it has this kind of concept of the context is basically your map or your

travel guide basically that helps you to navigate the real world. And what you put in a travel guide is basically guiding the agent to the next step. And then in the next city perhaps you have a different travel guide and and and and so on. So from that perspective this navigating a complex information space resonates at least with me really well. Um if you look at tools,

tools have a really important aspect in context both in terms of like how do I make tools uh to the agent. Uh they basically provide agency. So because in in rag it was we pushed information into the context. In tool calling we have the LLM and the agent be able to pull information as needed as such. But then of course how do I make tools available? How

do I do it tool design? Like I think many of the good API design principles apply for tools as well like naming descriptions. There's dependencies between tools. Are they explicit? Are they implicit? Uh I already mentioned tool outputs, structured tool outputs. How much do you output from the tools? How relevant is the tool output? Is there a lot of noise in there? And then do they also

come with um explainability? So the whys and the hows. Is there something that you can actually provide from your tools? So that's some things to uh think about uh here as well. And for multi-agent systems it gets one level more complex. Just looking at the clock I'll uh continue to my graph section. So graphs are actually really interesting, right? So I've been doing graphs for 16 years

now, actually much longer, 18 or so. And graphs are really good for highly connected information. And if I look at context especially under the view of information theory and structuring information, graphs are actually able for me to connect the dots, right? So both on the memory side, on the building a world model catching reasoning traces. I did a lot of graph um analytics of software and many

of the things that we've done uh back then already are now coming back as well. And as I said, I I uh put this uh presentation together last year before context graphs became a thing, but many of the aspects are also relevant here. So, from an graph perspective, LLMs bring all the language skills and the reasoning and the creativity skills and the the graph pro- provides the

knowledge graph provides the the context and the enrichment and the knowledge, so the facts basically as such. And together they can uh do really great things as Um I won't go too much into graph rag. Uh just really high level is graph rag is a way of retrieval augmented generation that uses graph data structures as part of the retrieval process, so you don't just use a vector

uh store to retrieve the information, but you use structured information to retrieve uh the information uh as such. In a graph structure, which gives you more explainability, so you know where this information comes from. You don't have a black box. You can navigate complex patterns and and go through the graph until you have all the information collected that you need to answer. And you basically also have

uh con- have more control about relevancy because you have things like confidence score on relationships. You have weights. You can basically determine is an entity relevant for me? If yes, then I continue in this neighborhood of this entity. If the entity is not relevant, I can stop there or even skip it as well. And so I can basically almost navigate my way across this uh map of

information in in a graph as such. But also graph capture context, right? So, you've heard probably the the context graph discussion at the beginning. And uh this is actually clearly relevant because uh graphs don't just hold data, they can also easily hold metadata, right? So, it's basically graphs all the way down. And so from that perspective, they can both on top of the data they can capture

metadata and beneath the data they can also capture uh execution uh and execution traces as well. So, you can also capture basically how did an agent operate, what tools were called, things like reasoning memory. And so, basically you can bring all these three worlds together. My colleague Will had a few really good blog posts like one that came out yesterday on this, how does this come together

basically the the metadata, the data, and then the reasoning information about agent execution. So, that's really interesting because you want to basically take the reasoning again condense and distill it and you can use things like graph algorithms for instance on top of these and traces to distill for instance what were successful patterns, what are there like frequent patterns, what were things that were had shorter paths to

success versus longer paths to success or low cost versus high cost. So, if you think about it, it's almost like an traffic routing problem in a in a way, right? So, how do I get to my goal on the shortest or most efficient way? And then can I memorize this for the next time I get a similar task because then I can save a lot of tokens

by not going through loops in the agent loop, but only one search. And so, if you heard about context graphs at the beginning, so that's what Agile is basically is a it's a knowledge graph that's designed to capture decision traces. That's what I just talked about. And what's interesting is there's decision traces from humans which are often times not available because they're like tacit implicit knowledge that

we don't materialize somewhere. But interestingly from agents, we can actually look at what decisions did they make, what did they do in reasoning, how did they make choices about tool calls and and and so on. And we can use this information to augment the human decisions as well. If you want to try out context graphs, there's this really cool create context graph uh application and and and

system that my colleague Will wrote. I really encourage you to try it out. It has lots of different domains where you can try this. And from us from a Neo4j product we are building on top of the graph database foundation using our tools, and then we basically put an agentic layer on top of that that has like agents, agent memory, um and uh agent integrations in the

ecosystem as well. So, if you want to build agents anywhere, we try to integrate graphs everywhere both on the memory side, but also on the retrieval, tool call, and um uh data representation side. So, I think like semantic layers and so on. So, if you want to learn more, uh feel free to hit us up on on on these kind of things. And to close, basically, communication

with an agent is not an is is a conversation. It's not a rigid "Do this." You need to express your intent, and you don't want to micromanage, but you want to give it as much control as as you can to make its own choices, but you also want to give it a framing uh to uh to work inside. So, these are kind of the things that we've

learned. Uh basically, we need context engineering for the specific task at hand. You can look at a minimal viable context context pyramids as some mental models for things. Graphs are really great for context because they have connected information and rich semantic information on the nodes and relationships, and they can do with the all the three layers, metadata, data, and reasoning traces, and there's lots of best practices.

Speaking of best practices, um in Graph Academy, we have a lot of courses on uh context graphs and agent memory, on uh GraphRAG, and then other things like that. And I already mentioned the quick context graph uh dev as well. And uh I also added all the resources. I'll share the slides later uh after after conference. And with that, uh thanks a lot, and have a great

rest of the conference. Thank