NODES AI 2026 - MemMachine: Agents That Learn, Memory That Lasts

In this talk, Christian, a principal architect at Memgraph, discusses the concept of AI memory and its applications within the Memgraph ecosystem, including tools like Memlight and Membox. He explains the architecture of AI memory, which consists of three layers: model weights, key-value caches (KV caches), and contextual information that assists in generating accurate responses. The speaker delves into how these components function together to streamline computations and improve user interactions with large language models (LLMs) by caching intermediate results. Christian also elaborates on different types of memory, such as episodic, factual, and semantic memory, and how they can be integrated into user profiles for personalized experiences. Additionally, he presents the advantages of Memgraph's memory solutions, emphasizing privacy and transparency over traditional model provider lock-ins.

[music] >> Um, yeah, as you said, like Memgraph agents that learn memories that last. I'm Christian, principal architect with Memgraph. And I'm going to talk about AI memory for Memgraph and then the Memlight and Membox. And just a little shoutout. I I used Neo4j for a long time, like for over 10 years now. That's from my my blog like over 10 years ago I created an HPC

inventory system for Slurm. Doesn't matter really, but just to say like I'm happy that I'm on this uh in this conference. Very informative. Learned a lot already and hopefully I can contribute to this with more basic beginner's talk. I think I want to give an overview of the AI memory and how this connects together and how we help we help to uh yeah, make AI memory work

for you. Okay. Little uh overview about memory or data in LLM and how this plugs into the ecosystem. So, there are three layers of AI memory, if you will. Like if you want to call it that, right? So, there are the model weights, which are some an idea of like try to create the brain or the the connections that the neurons in the brain have and and

represent information with a lot of parameters within a network of uh of this LLM, right? So, it's trained on public data. Tries to mimic how the brain works and it uh you put something in and it's like you want to generate text for instance, you put a short sentence in and you want to predict the next word and this is what goes through this model weights and

then at the end it turns out a word or token, a fragment of a word. And this is at training time when it's finished, you cannot really change it anymore. You can of course change or create a new model out of it, fine-tune it, but uh it's basically once it's trained, it's done. And when you use this model, what you actually do is you want to, as

I said before, create like what if you want to create tokens, you want to create text, then you have a fragment of a sentence, in this case like machine learning is, you want to predict the next word, next token. Uh it's going through this all this layers and then it predicts the next word and then you go back to the front again. Put in machine learning is

fun and then it predicts another word and all these iterations take a lot of compute. If you don't have a KB cache, it's going to predict every single every part of the sentence every time you go through the loop again. So, that's a lot of waste of compute, right? So, smart people came up with KB cache, so key value caches, which is basically the working memory of

a running model. So, all the intermediate results during the inference is going to be cached and when you put in machine learning is for instance and it's already computed before, then they will take this and they don't need to go through all the layers again. And this, of course, speeds up computation a lot. And if you have like another user of the model at the same time

who wants to print something else, he wants to use the same prefix like machine learning is, then maybe he's predicting another word or reasons, but you still can use the same um yeah, the same previous work that is done. So, this is very important and it scales up and that's actually use large language models with longer text in front of it, So, that's the second layer. The

third layer is and this is kind of as as I said, the working memory of this model weights that were um that were put down when or or manifested when the training was done. And the relationship between the two is model weights and KV caches that you cannot port the KV cache to some other weights, right? So, they are tied to the model weights. If you change

the model, KV cache is gone. So, the third layer, and that's like the the the talk that we are going to talk about is like the context. So, if you want to interact with the model, you need to put information in front of the model so that it knows what you you're talking about. So, it of course can just generate text, but you want to generate text

with a purpose. You want to put in background information to help uh provide an appropriate answer, right? Maybe that's private or timely data. Well, if the model training was cut off half a year ago, you are not going to get information that is newer than that. And it's also maybe generic public data and not uh like your CV, for instance. So, if you want to include this

data, you need to make sure that the model has has access to this information. And how this is we we have a context window where we can plug information in, and then we put the the question of the user in front of it, and then it can start generate questions. And my connect uh my example that I will use throughout the talk is I will put in

my name is Christian, and I live in Berlin. Um super super long context, but that's stuff that is put in front of the of the user query. So, if I put this in front and ask, "What do you know about me?" the model will respond, "You're Christian. You live in Berlin." And let's dive a little bit deeper into context, and I would like to frame it in

the in the human sense, basically, for for a starter. So, there are three types of memory. There are sensory memory, which is a very short episode of what you what your um what your senses take in, and they are hammered, of course, with a lot of information. And it's kind of the first filter stage that tries to maybe highlight something uh in your in your environment to

to make sure that you realize it, right? So, maybe a baby is crying and the echo of the cry is sticking around for a little bit so to make sure that you know about it, right? Or the tail light of a car that unconsciously your brain wants you to realize or recognize, right? So, that's kind of the the first stage of After that, we have a short

period of uh of temporal storage for processing which is called short-term memory. It's couple of items for a couple of seconds and it's just to to create a buffer for longer processing or processing for long-term memory to occur. And one way I think everyone at least me, but I guess everyone is doing the same. If you want to memorize something, you're repeating a phrase over and over

again. Maybe you want to memorize your PIN or your password or whatever, you you repeat the phrase couple of times and the reason behind this is that you want to to escape the short-term memory basically and commit it to long-term And long-term memory as the name suggests is the active like knowledge base, right? The and I'm getting older now, but it used to be like indefinite of

course. Uh so, you never watches movie two times because you know already the answer, right? But uh of course it's it's it's decaying anyhow. Um that's beside the point. Uh long-term memory there's two different varieties. It's explicit and implicit long-term memory and explicit means that you are yeah, you you consciously recollecting or trying to to get a memory back, right? So, maybe you see a face and

you want to get the name or you want to remember a password or something like this, right? So, you're explicitly recollecting. And the implicit one is when you maybe walk into a room full of friends, your brain will put you in a good mood and will maybe lower your barriers to like your your your defenses because you're with with friends and good company. If your arch enemy

is there, then you will be in a different mood or you move into a room where a song is playing that you are emotionally attached to, then you maybe you get emotional, right? So, it's kind of the framing that your your brain does to make sure that you maybe are ready for more input to be taken in. And at Memgraph, we took this long-term concept and put

it into four buckets. Like episodic memory is past events in conversations. It's just the history of what what you talked to with the with the with the model for instance. Then we extract factual information about this from this. And so maybe like my example, as I said, my name is Christian, I live The this are like the episodes and factual memory is the name equals Christian and

location is Berlin, right? drawing relationships between those facts, that's what we call semantic memory. So, kind of like an understanding of the relationships between those facts. And to build up like a dedicated store or a dedicated profile for a user or an agent, we we use profile memory, which is detached from the other memories, but of course like a similar similar inputs, but like the name and

the location will also be in my profile memory. And what we what we built was like last year we started with this in spring or summer. We created a open source project called Memgraph. It's Python code that has couple of APIs, Python SDK, restful API, and an MCP server. It uses, as I said, episodic memory and personalized memory and it uses graph databases, SQL, so that you

can interact with agents. And I have a little demo to just motivate what do I what I just talked about. So, I have an empty graph database, so nothing in here. And within N8N, you can of course create like a little chatbot, right? So, let's say my name is Christian. Boom. I was listening and I can change that. And first, I would like to do like a

little anti-pattern thing, which I think is a little anti-pattern. Uh so, I would like to use the AI agent. And for this, we use Open Chat GPT and we use uh 4.1 mini. Here we go. And we use the memory node. So, we have a memory node, which is uh can be plugged into the AI agent. And what's going on is that when I send in message

here, let me reset this, it's going to go through the AI agent. The AI agent asks the memory for like memories. The memory node here for memories. If there are none, then it will return like an empty um response, of course. And then the model will reply and afterwards, both of the messages are going to be sent to Mem machine. And eventually, they will end up in

the graph. So, we can see we have hello, that's the answer. And then we have my name is Christian. And that's already what I dislike about this, that I like pollute the memory with chatty answers, right? As we all know, large language models, they want to chat, right? They want to stay connected and interact with you, so they they ask stupid questions when you just state a

fact. That's what I dislike about this. But nevertheless, um I can do like, what do you know? And oh, let me first do like, I live in Berlin. And one two phrases. And then I about me? And what's going to happen is as I said, they will retrieve the memory and they know um that I live in Berlin and I'm Christian. And we can see this now

is even more polluted because I have like six or three interactions already and we can see it the traces here. So, we can see that there's a storing and then it retrieves and it stores again. So, a bunch of stuff going on. That's why I call it like antipattern. Of course, it's also a little bit due to our memory node. We could do maybe more here, but

actually that's that's how most of the models work. Okay, let me get rid of this and let me remove the graph. Let's do this in a more straightforward fashion. So, first I'm going to use not the big node but the manager node to store memories. And I am tracking just for the sake of it. Okay, so now I can do the same. My name is Christian how

old are you old? We have only one name here one node here. And let's do the other one as well. Oh, no. Let's first do the the branch. And I'm going to do a stupid or like simple let's say simple uh if and else [snorts] here. So, if it does not end with a question mark, I think it's a fact. So, I will root to this. If

it does end question. So, I want to root it here. And what I want to do is I want to enrich it. Here is a context template. So, this will be filled by the uh N N M mem- mem- machine node. So, it will just take the stuff from memory and then fill it out. And that's about it. And this I will just give to the same

AI agent. Use the same model. And I I won't change it yet because I I just want to make sure that we are all on the same page. So, if I do like ask "What do you know about me?" Then it Oh, and maybe I didn't have a question mark at the end, right? Ah, stupid me. What do you know about me? Question mark. It's going to

be go and through going through the enriched node then the AI agent and it answers, "I don't know much about you." But it it's very detailed, "I don't know much about you." And that's because we didn't add the system prompt. So, you can see here the user message is, "What do you know about me?" But there's information beyond that. So, what I'm going to do is I'm

taking this filled out template, putting it in here, and now let's do the same with the question mark at the end. It's going to go through and it will respond, "Okay, I know you're Christian and then you are living in Berlin." And then play to the demo gods. Here we go. "Your name is Christian and you live in Berlin." So, that's kind of like a pattern that

we can use here, right? So, this of course is a very very simple, straightforward way of like having just a question mark or not, but we could think of this maybe as a little model that extracts information, puts something to the store node, and then once it's stored it routes it to the enrich node to also capture what's in the what's in store basically. So, that's kind

of the idea behind this. All right. >> [clears throat] >> That's the demo of the first demo. I have a second one. Okay, and uh of course there's a couple of things we we we need to discuss here. Dumping the user like what do we want to do put in the system prompt that again the context of the question that we that the ask, right? So we

can of course just dump the user history. That would be the stupid or like the simple version of it. Like beginner's version. Can try to distill a summary of the current session and that's what we do with my version and lights. We once you have a sufficient amount of of episodes, we will try to extract more information from it as that extract facts that we can put

it on top of it of the of the context. Um we also summarize the the short-term like the the actual current session for instance. we could also try to derive the goal of the user prompt which that would be awesome, right? So if I know we are going towards a certain topic, then we can try to only get information that is relevant for this. And but this

is of course kind of also a no-no because we don't want to delay the response too much, right? So like maybe a couple of hundred milliseconds, a second or so and then users got impatient. So we don't have much time to spare, so we need to keep it short and concise. as I said, here we go. So like we have the system prompt which is kind of

also giving more context like the filling the context window of the LLM request that we are sending. And this is going to also be put in the KB cache, right? So if I ask this question twice with the same system prompt, I'm getting a faster Um but I need to make sure that I don't mess up the system prompt too much, right? Because if I do this

like split the order, then the cache is invalidated and yeah, computation needs to be done to keep up with this. So that's also another complication of that. I need [clears throat] to speed up a little. Um and maybe I'm over 40, so I use computers back in the days was the normal way like having an application typing on the keyboard and then do stuff with it. But

of course we are moving towards a more yeah, LLM accelerated world where you maybe not interacting as much anymore with actual tools with IDEs, but you are more chatting with AI agents or assistants or frameworks that in the background are using foundation models or other models, within this world, what we need is or what we are building and what others build as well, but we try to

make it private so that's our our idea here. The memory is important in so many ways, right? So it's a data platform across different sessions, across models, and it's also delivers the context to the models, right? Because it knows over time, it builds up over time and knows stuff about you. So and of course we are not alone in this. All the big companies they want to

create their own memory of course and lock you into their ecosystem, right? Of course Amazon, Google, you name it, everyone has a memory bank or a memory in general and it's a big market because it's of course very important information. It's not only important to you, it's also important to the companies, right? Because the more they know how people use their tools, they can build better tools,

right? So if they know, okay, people go trend in a different in a certain direction, then by knowing this, that's very valuable information. And by having big service providers provide their own AI memory, of course we have couple of problems. I mean, I'm in Germany and Europe, so we are very sensitive about our data, so the lack of privacy is a concern. At the model service providers

can use the data however they choose, right? They can maybe train future models with this even though maybe the print, you know, doesn't say or say it otherwise. Um there's a lack of transparency. We don't really know what's in there. Um in ChatGPT, you can export everything, but it's also like just a blob of JSON that it's pretty uh hard to distinguish what's important, what's not important.

And you cannot really prune it, remove information that you don't want into in in your memory. That's kind of like transparency is not And um you have a model or a provider lock-in, right? So, if you use ChatGPT or Claude, and you they build up a memory about you, it's highly coupled to the model provider or service provider. So, you cannot break out. And that's why we

built last year we started with Mem Machine, and now we have Mem Light and Mem Box, and I going little [clears throat] demo I have time after and after this couple of slides. So, we build memory that is owned by you, right? So, we the Mem Machine, as I said, is like a Python uh based microservice approach with um Neo4j as a graph database and SQLite. And

um Mem Light is a Rust um backend written in Rust, which uses file systems as a backend using MD files and SQLite data so that you can put on on everything, but it's private. That's the basic part, right? All the data is stored on your device or on your backend. It's transparent. You can inspect what's stored. You can prune it. You can remove stuff that you don't

want to be um stored. And it's also multi-model and um yeah, multi-service uh able. So, you you can switch models, model providers. I use Open Code a lot, so I can use Open Code. I could use Open Claw. I didn't try, but I will. Uh and like persists and and moves with you to other approaches, right? And as we know, in AI world, there's always like another

model, another tool uh coming up every day. So, and I I think I need to skip this a little faster, but I had like last time I gave a talk here in the global AI community, I had this problem statement of the Bob problem, right? So, a couple of statements about developer project. And what I wanted to know is who should I talk to about a certain

aspect of the project, like the dependencies for the project Atlas. And as I said in the beginning, if we can derive the what the what the actual query is about, we can put better memory in there, right? We don't want to clutter too much information there, so I just pick the top five. But, if the top five suck, then that doesn't work well, um I looked into

it. I said, "Okay, graph database is not really used much, so let's get to it. It's open source anyways. Let's improve the graph." I had a lot of fun with spectrum development using like open spec, open code, and entropic models, and went away And at the end of the day, I made it so like the the model or the the memory that was received and put into

the context of the user query was much better. Um I had a much more like who approach basically, like showed me who I should talk to. And that's what the graph looked like at the end. Of course, it's maybe not very generic what I created because it was maybe more specific to the use case. But, and we saw this throughout the conference building a knowledge graph is

kind of hard, right? If you want to create a generic one, that's even harder. But, that's that was uh quite a nice experience. Yeah, but memories still have hard, like effective and efficient memory, of course. It takes longer to create the right context. And one size fits all is always like with a specialized memories, of course, also hard. But, we create we created a couple of APIs,

like restful SDKs and MCPs, so that you can plug it into your uh easily and and use tools that maybe your your use case best. What we could also do, which I didn't do, but we could use higher level backends, right? So, like >> Neo4j knowledge graph anyone? So, maybe we could use this to build up the graph and then plug into this, right? But there's like

so many Lego parts and it's hard to choose, right? But that's certainly also one option to to deal with this. Okay, last couple of slides, last two slides actually. Um What we also did with the experience we have with with gained with Mem Machine with this Python microservice, we created Mem Light, which is a Rust-based AI memory tool. Uh uses um or learns a lot of lessons

from Mem Machine and it puts the the the memory, like the episodes and the raw data uh into files, markdown and SQLite. And the cool thing about this is that, of course, you can put it on a shared file system, like iCloud, Google Drive, what have you, and you can sync this across devices. So, if uh Mem Light uh runs on a on a TV, for instance,

or on a phone, and on your laptop, and you you talk about like maybe an actor, then the TV can pick this up and promote certain actors, like movies where the actor that you talked about or you you researched about is is going to last or is going to to occur. [clears throat] And it's very lightweight. Uh it's in Rust, so that's pretty cool. And we have

also um um like a GUI for this, and that's my last demo just to show you. That's my memory box data folder. It's currently empty. If I do use Memory Box, it'll come up like this. Okay, let's continue as a guest. We'll see that it will already create a couple of files And I can do [clears throat] like My name is Christian. and we'll see that it

shows up. Mem light, so we can see that Mem light is here, right? And then we have the episodic memory and we can see that the information is stored and we have also tags and so on. So, and yeah, I guess you believe me when I write to what do you know about me and what about me, it will return the two facts. Okay, and while we

wait, that's about it. So, I will I have like 5 minutes, 4 minutes left. Let's wait for the answer and then I'll get back to the tool. Here we go. Yep. Your name is Christian. >> it was too fast for like I don't know. Anyway, cool. Let's get out of this and stop sharing and yeah, we have 3 minutes left. So, any any questions? Are you also

storing the agent's chat history as part of the long-term memory? Um you mean for like Mem light and Mem box? Yeah, we have like episodic memory in both actually. So, the the episodes are not pruned. I mean, of course, if you want to, you can prune it after a while if you have too many episodes there, but they will be stored uh for long. Yeah. There are

questions in the Q&A tab. How do you deal with files that are shared in the shared context that may be relevant later? Um this can also be put down. How does Mem machine scale? Yeah, of course. I mean, Mem machine is not using files, right? So, Mem machine scales with with Neo4j and SQL or Postgres. Pretty nice. Um and for Mem light and Mem box memory box

of course I mean does go so certain lengths I from the top of my head I I don't know like how episodes or how many raw messages we we store but since we have like and I I didn't show this but we we have explore like an explorer of the moment of the sessions and the episode that you have so you can prune them you can also

export the chat GPT where you get a directory or zip folder and you can import this in Mem box memory box and then you have like your session history chat GPT it's also pretty nice. What else like to see memory agent that there's also connection to improve the referral memory presentation interface yeah I mean I think for this just play around with the SDK or the or

the MCP even so there I think and you can do and for Mem light and yeah you can ping me or connect on LinkedIn or ping me here. Yeah there like we can of course talk about how to to to get access to this and because Mem light also has an API endpoint right and with this very thin rust binary that can run anywhere it's pretty nice.

What's the edge oh yeah yeah yeah I do this personally like I use Obsidian as my LLM wiki Yeah no we we use markdown files in Mem light memory box but also SQLite vector databases so it's not only um the the markdown files markdown files are more to exchange information and then derive from this raw sources derive the memory. What do you have any ideas for how

with memory context blow without losing yeah Yeah, that's as I said throughout my presentation that's kind of the the golden goose when the problem is filling up which is long term memory. Yeah, long term memory like we have Yeah, one minute left. So, we have um Yeah, episodes for the the global episodes and the the session episodes, right? And I guess the session yeah, the session episodes

will be more prominent if you want to chat within a session because that's actually what you're you're you're dealing with, right? So, you interact with an with a chat with an LLM in sessions and throughout sessions or cross sessions you might want to extract information, but um that's the important part and I think that's more the and you know, we are wrapped up in seconds. Cool. Um

I think that's it. Thank Andreas. Are we are we offline I guess? So, any stage? >> [music]