NODES AI 2026 - Agent Reasoning with Graph World Models

This talk addresses agent reasoning with graph world models. The speaker, Lucas from the Mind Synth agent lab, discusses the foundational concepts of world models within the context of agent applications, emphasizing how graph representations can effectively leverage these models. He illustrates how human agents navigate their environments, the importance of representing causal relationships, and the potential of graph schemas to provide a deeper understanding of domain structures. Lucas presents simple demos that showcase the transition from raw agent interactions to reasoning based on causal models, highlighting practical applications such as clinical trial simulations. He encourages attendees to rethink their data models and consider the implications of integrating world models into their existing architectures to facilitate agent-driven tasks.

Brilliant. Thank you so much. So, yeah. Hi everyone. Hi Andreas. Welcome to agent reasoning with graph world models. My name's Lucas. I run the Mind Synth agent lab and before that was a PhD researcher working on generative models interactive maps. So, in terms of what I'll cover today, um, I'll give some background just set the scene so we've got like a baseline understanding of world models in

the context of agents and then we can focus on how the power of graph representations can be used to leverage like world model concepts and techniques in your applications, hopefully. Um, so just kind of a few things on scope. Obviously, as I said, do a bit of an intro, explain the links between world models and graphs. I prepared some like toy demos just with like a simple

learning algorithm to convey some of the concepts. So, I'll go through those as much as I can in the time. Um, I'm assuming that this is really about focused on people who work at kind of the application layer. So, this is not to do with like a fine-tuning the underlying LLM. I'm assuming you're working on the harness, shall we say? Um, but I will look at kind

of focusing on how this topic generalizes to non-spatial domains and and give you a foundation to learn more. So, the key takeaways, um, if you're building graph-based applications around agents, you're probably probably quite far along the road to already being able to implement a world model, actually. It's quite simple to take your existing graph and turn into a world model. Um but the the big takeaway I

want to land today is that even if you're not going to implement RL, you're not going to go down the whole world model rabbit hole, thinking about your your graph schema as if you're going to build a world model is just a really powerful way of of thinking about your schema and thinking about your domain, and something I just really really recommend. So that's kind of the

big takeaway that I wanted to get from this. So just to give you a quick intro, uh human agents reason about their environments, um about how their actions in their environments can lead to progression towards goals, uh the ability to perceive and interpret the content and structure of the environment is like a foundational aspect of this, but kind of beyond sort of forming this underlying representation, the

ability to reason based on knowledge of the causal effects of actions is really a a key skill we have to achieve like multi-step goals in complex environments. So a key thing I want to land today is that world models are not just about spatial context, but obviously you kind of need to start in a spatial context just because it's so inextricably linked. So here if you imagine

like a simple uh 2D uh grid world, yeah, if we then have a kind of root graph on top of this, although that's kind of derived from the spatial environment, here we can see this kind of compressed uh topological could describe some arbitrary environment. In terms of moving that into something that we could use in a world model, then as you start having agents uh executing actions

and then explicitly tracking observations of the effects of those actions, you can see that this kind of can be tied in the graph directly to uh in this case phenomena like the sun room the balcony, but in some arbitrary case it could be some other estates of your application. Just kind of give a quick um a quick intro. We like this quote, LLMs learn the structure of

language, world models learn the structure of causality. I think that's a nice summary. if we start to think a bit more about about this issue kind of data modeling and and how world models give you a slightly different perspective, consider scenario. There's a person, a conversation agent, and a memory sub-agent. In this example, how many agents are there? From a world model perspective, there is three agents

here because obviously the person, even though they're not in your code base, they play a causal role in the success of your application. Um so, I think that's kind of like a a nice example of of how this is slightly different perspective. Another one here, if you imagine this agent in the middle is a conversation agent, what is the agent communicating with? In our schemas or in

the way that we talk about things, we might say well it's communicating with a person, but often that is not true. It's communicating with the server, which is communicating with the client, which is communicating with the person. So, here you might say that the client could be a mobile environment or a desktop environment, and that might have some kind of causal effect in the performance of the

agent in its world. So, it's kind of just some simple examples, but just to land this idea of a different perspective. So, moving on to some of the demos, uh again starting starting with the spatial environment, this kind of simple 2D environment where the goal is just kind of getting to garden. So, here on the left you see this kind of uh the actual simulated representation of

the environment itself, and then on the right this kind of state action space that that is kind of grows how how to like running the agents over this environment, um you can see this nice example because in the spatial context, even though you have like a topological structure, the spatial structure sort of naturally emerges um out of these kind of agent actions over time. That wouldn't necessarily

be the case um for your applications, but it's a kind of a good starting point. give a sense of what we're actually going to be showing um it's just like a kind of toy uh toy learning algorithm um where we're evaluating actions using the expected discounted total reward of future trajectories from the current state. So in the demo I'm about to show, we'll have these like rollouts

and all that means is you're just looking at potential futures from the current state and then evaluating what the best course of action uh would be. So if I show that in a if I show that in a demo um So in this interface, there's obviously quite difficult to explain this quickly in the time. Um so if I just show a a sped up version, you see

the model is turned off and the agent is just exploring this environment and you can see how um this kind of state action space is just like growing through this like simulation over time. And on the left, you see the rollouts of um of the predictions. Um we don't really need to go through this, but you can obviously you can just see how that grows over in

terms of like then turning the model so if we imagine we then turned our world model on and now the agent is able to act based on this kind of causal knowledge about the effects of the actions over time. So, this is like the simplest example that you you could do essentially, but it's just kind of gives us a baseline of of of of how this environment

works. moving on to something that's closer to what you kind of would be used to in um you know, in kind of like things that world labs do or like the Genie Google platform, things like that. That's more of a a scene-based scenario where you're not just traveling around the roots, but you have to pick up objects, you have to complete tasks within uh the scene. So,

to give an example here, again, if the model was turned off the it's very difficult for the agent to actually complete this. I think here it unlocks the door, but never um actually gets through to the garden. Um but again, if you turn on the world model, then it has this it has this ability to think about the uh causal effects of its actions over time, and

it actually completes the task. So, if you've got this far in the presentation, you might be thinking this seems like a bit complicated, and what does this have to do with what I work on? but if we think about how you can actually get your graph to do this, it's actually it's extremely simple. Um there's not a lot of stuff in this talk, but I've provided this

as a as like a simple example. So, here at all of that those examples I've just shown you, um it's literally just having basically two statistics on a transition edge. Um and it it's quite kind of straightforward to see how you could um build out this kind of model on top of your um existing schema. See if there's any questions or anything. Okay. So, moving on to

what I call a generalized environment, so a state action space that can be expressed topologically even if it doesn't derive from a spatial context. So, here the key example is LLM agents, which I assume is what most people working on if they're watching this talk. Um Yeah, we're doing okay for time, so I'll explain this. So, here you see this kind of state action space represented and

you see how the kind of like white arrows are disjoint. The path is kind of broken up. And this is just to kind of communicate how because we're not in a spatial environment now, this kind of transition of states doesn't necessarily correlate with some like initial topological structure. So, here the states are still spilling out over time even though um it might be say one agent is

calling a tool over and over again rather than moving like across the initial structure. So, I just thought it was an interesting thing that shows up in the in the graphic. Um but if I go to the if I go to the example generalized stuff. So, here um in this example uh clinical trials. So, the human is a researcher, the human agent is a researcher. He's getting

this network of agents to help them uh create a clinical trial. Um So, the goal here is probably closer to what you guys might be interested in where it's not a spatial goal, but it's like an informational artifact. And then in this case, the researcher accepting that artifact acts as a reward signal essentially. Um so, if I show you that with the model turned on. So, you

don't get rollouts for this initial thing just because there's not actually a prediction for what's happening in these nodes. But then once you get through to the uh agent network, obviously then you start seeing the the roll out guiding basically the tool selection. yeah, and on the left here, see this uh state action space just kind of growing over time. Just fast forward. You get the idea.

And here, yeah, as I said, the the human has accepted this uh this result, uh if I go back to the station. So, in this environment, um maybe easy to see in this view, uh we've got current state, got this view of like roll outs, some representation of what this environment looks like, um a cipher console cuz obviously if you've got all this stored in your graph,

you can run all sorts of queries against it. I haven't shown that um now, but you can imagine the power of that. This graph view obviously becomes quite complicated very quickly, so but it's just to give you a sense of of what's um in the data. in terms of creating world models for your own domain then, the the key thing that I would recommend is to uh

to revisit your data model and and and and think through what plays a your domain. Uh and and and and just kind of take take a step back from it, even if you're not necessarily going to be implementing the learning algorithm or anything like that. Um I think that's just a really great exercise. You can create a simulation environment similar to what I've shown you to test

causal hypotheses that relate to things that aren't in your code base. Often, you know, we might instrument code base to test things which are not explicitly testable. If you think about the world that your agent inhabits, lots of things that impact the agent's performance are not in your code base. And that doesn't necessarily mean it's not testable at all. You could still create some kind of simulation

environment to look at the causal effects of your agent's actions in your use case. Another thing I would say here is that you may have noticed that some of the things I've shown you if you have an observability platform hooked into your application, then you might already have a system that's kind of generating graphs over like your architecture or over like your agent runs. So, it's possible

that you might be able to um like use the spans or essentially kind of hack the data that you're getting from your observability platform as a basis for building your world model. Yeah, there's there's a number of tools which you could see how you could sort of hack them for that. Use your you know, professional graph DB understanding to manage the data volume from simulations and learning

runs. You like if I go back to this example, so here I mean the it's kind of a an odd example, but you can see how easily you start generating like massive amounts of of nodes and this is just from like test runs where you've not really achieved much. So, um yeah, it's just thinking through like how to manage that data using the kind of usual techniques

you would do in your your work as a graph database engineer. It's just something uh to to be mindful of. Um but the kind of the main thing I would say here is be mindful that you're being introducing a kind of a second reasoning engine into your architecture. So usually if you're using you're working with LLM agents or building agents, the LLM acts as the reasoning engine.

But then if you have this kind of world model mechanism happening in your harness, then there's this other reasoning engine kind of coexisting with the calls to the LLM. And this is just like a bit of a a nuance with um with having kind of a a world model um mechanism working alongside your existing system. So we probably do have time for questions, but so maybe if

I go through some conclusions and some references and things and then we could look if there's any um questions from the room. so yeah. Concluding if you're working with agents, world models are for you. This is the main idea I want to get across. I'd say get familiar with the key concepts and take another look at your data model even if you're not going to take the

step into RL. I think it still think it's massively valuable to have this perspective of agents as things that act, we can observe the cause and effects of those actions, and think more broadly and more deeply about kind of the world that they inhabit. Uh if you're interested in RL and you already work with uh a graph TV, which I'm sure you do if you're watching this,

um it's kind of quite straightforward to dip your toe in with like a simple experiment. and also I think just the fact that has a kind of practical basis for world models with kind of bigger and bigger topic. I think I think it was even mentioned in the opening keynote how context graphs could be a basis for a world model and it is kind of a natural

fit like so many things are with with graph representations. So I think it will be a bigger and bigger topic and if you're watching this I think you're you're early. So Is there another a quote? The rabbit hole goes as deep as you want it to but it's world models all the way down. I partly picked this quote because I really like it but also this is

actually very recent article this one by McCormick and uh De Wit De Wit not sure how you pronounce but um it's in the references and it's got quite a deep dive um on on world models. It's got a very very informed well thought out kind of perspective and I'd recommend that if you're looking for some more reading. I won't go through all the references now but uh

yeah that's the one I was talking about computing the uncomputable. So that's the URL. And yeah so that was the main content that I had to go through. Happy to take uh take any questions or if there's something that I you want me to go into more detail in. I appreciate I skipped over a lot of the the detail of the demos just because there's a lot

there um but yeah should we look at the messages? >> Got the questions in the Q&A tab if you can pop over to it. >> Cool. Brilliant. Thank you. So first question is can you explain slide 15 once more? Sure thing. So, here I've modeled in in the all the examples I've shown, there is an edge called the transition. So, and this transition edge goes from um

all the phenomena that are represented in each of these worlds. On that transition edge, it's two values, our mean visit count. Exactly what these are, I wouldn't worry too much cuz there's different ways of implementing this. My my basic point of this slide is showing that by having simple statistics stored on the edge, and then being able to do queries of the form that I show uh

here, you can This is basically the entire basis of being able to do the roll outs and the like um like causal predictions Um it's literally just storing those stats on the edges. Um and and using that as the as the basis for for the roll outs. So, I I hope that um answers your question, but feel free to feel free to drop a follow-up and I'll

see it. The second question is, in your slide on creating a world of Uh ah, sorry. Is it Is there a requirement to precompute a correlation matrix on the types of phenomena you code from so that highly correlated phenomena are not duplicated in the code? So, I don't think it's a requirement. Let me just go to the slide this person is talking about. Here's the world model.

Um see I think the I think the way to think about this is that you you just need to think very carefully about the data model and like structure you're running the statistics over. Um if you just have some large graph structure and you kind of embellish your edges with stats and then just like let agents take actions and make observations, then you kind of end up

with what you saw in some of the earlier simulations where it's like a random walk and you're just generating loads of data that's not really that useful. So, I I'm not sure you need a correlation matrix in the context of what I'm talking about, but you need to just think carefully about the the schema, really. Ah, thank you. So, Desirée asks, "Is this application on GitHub?" So,

it's not on GitHub, but I'm happy to share it with um with anyone who's um who's come to the talk. Um Yeah, I thought it'd be a nice little gift for people who actually came to the talk rather than it just being on GitHub, but I'm happy to uh happy to share it. So, anyone who wants it, um you can get in touch with me and I'll

um I'll invite you to the repo. It's you can use it in any way you want. Just uh example for you guys, really. So. Uh the last question in the chat is, "Is this reinforcement learning?" The short answer is it is reinforcement learning, but it's like a very simple uh yeah, very kind of a simple example of it. Really what I want to demonstrate with what I've

shown here is is it's pretty straightforward to dip your toe into world models if you already work with graphs and world models are very useful thing even if you're not working with spatial data and are useful even if you're not working with RL. If you want to really go down the world model route, then fundamentally is a rein- a reinforcement learning practice. if you're interested in some

of the things I've shown here like this can form a basis of then doing like RL on fine-tuning models and things like that. So, this this can be like a stepping stone to that. Um but the short answer is yes, you you you do need reinforcement learning to make a world model kind of useful in practice. these are all the questions. I don't know if there's anything

else anyone wanted me to go over again or cover again. Uh recommendations for reading references. so, if I put the references up again, I assume you'll be able to get access to this presentation again, but um here is some references. I'm not sure necessarily these are the best references for world models per se. They were things that I think were relevant to this talk. Um but I

it gives you a starting point, I guess. Um but yeah, I definitely recommend this one. This uh world models computing the uncomputable. It's uh yeah, it's pretty detailed, quite fun, and I think it's just a a really good perspective on on what's becoming a bigger and bigger Lots of people asking how they get in contact. I think you can contact me through the uh platform, but I'll

just put my I'll just uh email address here. It's just easier for me. I'll just put my email address in the chat, so. All right. Sorry, just do it again. Ah, yeah. Yeah. Thanks. Thank you, Neo. So, a few people are asking about um a few people are asking about the um getting your hands on the code. So, maybe I'll just talk about what this actually how

to use the interface if some of you are actually going to like uh get the code off me. here in this top left you see that there uh three These are like three demos. So, the spatial environment is kind of scene-based one and a generalized version. So, that's just a way of toggling between them. You can control the speeds of the runs here. You can toggle the

model on and off here. you can't quite see here, but you can run arbitrary Cypher queries and there's some prepackaged queries just for the examples. In terms of Neo, the way it works is it's just assuming Neo is running in a Docker container. But, if anyone has any problem running the code and like getting Neo spinning up in a Docker container or something, then you can uh

reach out to me. yeah, probably the key thing to know about this is whether this model thing is turned on or off because that's going to massively impact whether whether the agents are converging to the goal or not. Um but yeah, reach out to me and I'm happy to share share the the code. Um any final questions? Don't suppose the slides talk will be shared. I'm pretty

sure all the slides are going to be on the platform, but um Okay, so another question. Did you check this approach on how it performs in open-source benchmarks to the So the idea here is not necessary to show this outperforms um any benchmark. This is just giving you a sense of how you could um start experimenting with world models and using your your past knowledge um within

this like emerging topic. So uh I'm not making any claim here about how it might quantitatively improve your systems. It's something for you to experiment with really. The learning algorithm that I've used in these examples is like extremely simple. So it's just a kind of a toy example. Um I hope that answers your question. Someone's talking about the Nvidia stack. Yeah, absolutely. So this this person says

when it comes to world models, I think about Nvidia AI stack and their movement towards world graphs and models. I think you're saying Like I said, it was mentioned in the opening keynote of today's conference and I think it will just become a yeah, a a bigger and bigger topic. Both you know, in terms of the kind of more let's say traditional 3D environments and in terms

of context graphs and organizational data and and all the other places that we have agents running. Amir asks, are you modeling how the agent answers user queries by tracking the sequence of tool calls it's doing? Yes, exactly. So in the case of if you're looking for basically user to accept a response that your agents have provided through reasoning with the model of the LLM. Um. Yes, exactly.

Tracking the sequence of tool calls is like the key thing to understand the causal effect. Obviously, you also need to um also need to look at the actual like content that's come back. And you might use LLM as a judge or something. Um but yeah, feel free to reach out to me if you've got more questions. I think wrapping up now. Um thanks so much for your

time, everyone. Really appreciate you coming.