NODES AI 2026 - Smarter MCP Servers: Using a Graph to Solve the Context Window Problem

This talk discusses strategies for optimizing Model Control Panel (MCP) servers, particularly focusing on managing contact window problems associated with AI agents. The speaker explains the concept of context rot, where exceeding a language model's token limit can lead to inefficiencies and increased costs. The session emphasizes the importance of careful tool definition and selection to prevent unnecessary token consumption when a client requests available tools from an MCP server. The speaker presents three approaches for tool exposure, including providing a full list, lazy loading, and a balanced method that utilizes usage data to prioritize commonly used tools. The discussion highlights the trade-offs involved in each approach and the need for tailored solutions based on specific implementation scenarios.

[music] >> Uh thank you for that. So, for the next 30 minutes, I'm going to talk to you about considerations for making smarter MCP servers. And in particular, we're looking at contact window problems. you might ask yourself, well, what is that? So, this is where an AI agent's token limit is exceeded. And that can cause a number of symptoms. It's often referred to as context rot. And

a [clears throat] contributor to this, and I'll come on to a bit more detail about context windows, is how we build MCP servers, and in particularly around tool definitions. this isn't something which is merely theoretical. If we look at a real-world example here, here's something which somebody posted on Reddit. And they posed this question about they'd looked at Claude code, used the doctor utility which can show

you tokens consumption, and discovered they'd consumed 27,000 before they'd even done anything. And that all contributes towards the context window filling up. And that context window is essentially a term for describing all the things which your large language model has to deal with. And it also, if we look beyond the context window, all those tokens are burning money. And that's before you've actually done anything with your

MCP server and [clears throat] your large language model. And if we dig a bit more into what's going on, you can look at this particular picture. And if we look at the top, we can see what's going on here when we look at the interaction between the model, the MCP client and server. And one of the first thing that happens is your MCP client will ask for

the list of tools which the MCP server has available. That tool list, >> [snorts] >> so the name of the tool, its definition, description, is processed by the model. And that consumes total tokens and it takes up space in your context window. And like I've just mentioned, you haven't actually done anything yet. when we think about context and the context window and how tokens are being used

here, we need to take this into account when we think about how we're designing our And how we structure the tools an MCP server has and how that server delivers them, we now need to take that into consideration along with everything else you do when you're building an MCP server like what communication protocol you're going to use. Are you going to use you're going to use standard

IO? You're going to use HTTPT? Are you going to use auth? You're going to What are you going to do about logging? Now I have to think about how can I optimize the presentation of tools to so I can optimize the token burn and I can optimize the context itself. And when you're thinking about this, you can roughly say for every single tool definition you have, it

consumes 200 tokens. And if you were considering building an MCP server, which was going to surface a bunch of traditional REST APIs, that could be multiple endpoints you were presenting as tools. So, maybe it could be like 60 plus tools. So, if you do the math, that gives you around 12,000 tokens right at the start of the conversation between your MCP server and your model. And that

And that tool count matters, right? Because we've seen it burns tokens. impacts our context window. And when MCP servers first appeared, we kind of ran to get on board that bandwagon. And we just jumped straight in, right? So, we simply said, "Let's give out all of our tools." Because then, the model's aware of them, right? by doing that in our haste, we kind of looked before we

Well, we did we forgot to look before we leapt, right? So, if we take a moment and pause, we take a deep breath, and take a step back, we need to consider what tools the model needs to get a thing done. And we need to try to avoid the model drinking from a fire hose. So, we need to think about how can we give the model what

it needs whilst avoiding cluttering up with stuff it doesn't need. here's where I'm going to talk about three potential patterns you can consider when you're looking at MCP And they have trade-offs, right? Nothing's for free. So, if we talk about the full list of approach, which is what I just where we give the model everything on startup. And that's what the MCP specification says happens. So, the

client says, "Give me Give me the tools." The server responds with the tool list. So, we know it will always happen. It's a reliable way of doing this. But, it burns tokens. If we look to the far right, then we can see what you could describe as lazy loading. So, here we're giving the model the ability to discover tools. So, the idea being that it finds the

tools that it needs, can ask more questions about the tool, and then runs it. And then, in the middle, we're kind of trying to strike a middle ground where we want to give the model the tools that it's most likely to need first. And then, we'll also give it the ability to find out more. And one of the things we have to bear in mind across all

of these is a model doesn't always do what you expect. So, it turns out it may not always ask. So, that's one things which need to bear in mind. So, let's jump into looking at each of these approaches in turn. Let's start with the full list. So, this is guaranteed almost by the MCP specification itself. According to specification, when an MCP client starts up, it always will

ask an MCP server for the tool list. So, you know that the model will be aware of all of the tools. token usage, right? And the context If we scale this up to incorporate lots and lots of tools, then we start to run into some of those challenges. But, there are some things we can do in terms of mitigation. And one of those is to implement some

kind of capability capability filtering. So, we try and change the way we look at the design of MCP server from how is the model discovering to how can we give the tools which are relevant. one of the ways we can look at this is to kind of look at filtering tool sets as [clears throat] a product decision, not just an engineering one. So, we can look at

this in terms of >> at what point should we give the tools out? do we go about doing that? So, we can look at it in terms of if I'm in a development type environment, then my MCP server could expose tools for debugging and to help me with development. But, those tools would not appear in a production environment because they're not relevant. We can also look at

the role. the end user, if you like, who's connecting by our agent to this MCP server to get a thing done. We can look at who they are and see what they need and then give them a subset of the overall tool list. Um for example, you could look at scopes in a JWT and based on that, you could assign a bunch of tools. We could also

look at the context. So, for example, we could decide that some tools you can always use but other tools are locked away behind some form of authentication. And there's a great example of this when you go look at how GitHub has done it. And they allow for selecting of groups of tools or individual one. So, in that particular example there, you can see that I'm asking for

a subset of the overall list of tools which GitHub can give me. So, I'm calling out specific ones. But it also allows you to to get hold of tools based on a category as well. So, that's an example of using the full list but then filtering what tools you get. So, the approach on the right-hand side from that diagram I showed you just a few moments ago

is to do what I would describe as lazy So, this is where you literally have three tools. And the first tool allows the model to discover what capabilities exist. So, it's literally asking the question what can the MCP server do? And it will get a list of [clears throat] the tools by name and you might give each tool a brief description. The model can then go for

a particular tool, ask the question, well, how does this tool work? And then based on that, it can then go ahead and execute that tool. And this, as you can see, there's only three tools. So, it dramatically reduces and dramatically reduces the number of and there's always a button, right? With lazy loading, you are heavily relying on to follow what you to follow this pattern of listing

capabilities, choosing the right tool, and then using it. You're really at the whim of that model. And I say in those terms because models can't be compelled. So, um if you haven't come across that yet, >> [clears throat] >> you will. And you need to coach and encourage. And if you've ever dealt with teenagers, it's a very similar kind of thing. Now, there are some things you

can do around that in terms of you can look into uh skills, which can help describe how a tool can be used and what it can be used for. But, you're still relying on that model to take that advice. And we can see that where occasions where we've actually seen that with our MCP server, which we have for Neo4j. And that has the capability for a model

what graph data science tools algorithms are available for it to use. And what we've noticed is if you phrase the question very carefully, it will go ahead and use that particular route where it discovers and then goes ahead and calls the correct GDS algorithm. But, we've also frequently seen if you don't phrase your question in that way, then the model will take the shortest path available to

it. And I apologize for that. There is no pun intended there. And it will take that path and would write cipher. And that gives you a result, but it's not necessarily the best result. It's not necessarily the best path for the model to have taken, but it will do it. >> And so you can see there so the model gets a clear specific task. You can see

there where it will just try and do it straight away. Or it will take the easy path, or it will infer the wrong parameters to use. And one thing just to be be aware of is if you're going to go down this lazy loading approach, is it really depends how you're structured. So if you're writing an agent working with the model, you have more control over what's

going And that allows you to take advantage of this type of approach. But if you've got a public MCP server, this doesn't work spectacularly well. So the middle approach, the balanced approach if you like, when we looked at that initial slide, this is where we've got a graph sitting behind our MCP server. And here what we're doing is we're using our knowledge we've encoded in our graph

to help with our registry of tools. And one of the things we're we're doing taking advantage of the fact that graphs are really good at answering questions, which a flat list cannot. So, here we can model the tools. We can model them into categories. We can record how often they're used. We can look at how well they're used, etc. So, kind of like usage data as well.

And the reason this is important is it allows us to on that initial tool call, that tool list command, which the MCP client gives at startup, is we can give the model the common tools first. So, for example, we could give it eight commonly used tools, and we know they're commonly used because we've described that in the graph, which is supplying that list of tools. And then

we give the discoverable tooling as well. So, we allow the model to find out more tools, and we allow it to do it by categories. So, if you've got tools which allow you to interact with a database, you've probably grouped them into things which may query your database, like read-only tooling, tooling that allows you to do imports, tooling that allows you to do mutations, change of data.

So, you can use that information, those categories, to group that tooling together. we're ensuring that the model can't miss what tools to start with, but also cuz we're baking that discovery into that initial tool list, the model is aware of it. And the other thing we want to do as part of our MCP server functionality is every time a model uses a tool, we're going to record

that usage. And so, that allows us to when we front up that initial common list, that list is built off real-world usage. So we can ensure those commonly used tools appear first. And then all the other stuff, we make it findable. So if you look into me a bit more detail about how this could look at the back end, a graph there, and we've got our gold

nodes, they're categories. So if I look at that database example I that could be queries, it could be mutations, it could be importing. That's a category. And then we've associated in lavender, those are all the individual tools associated with those categories. A tool may belong to more than one category, And this allows us using Cypher to find the common tools based on usage. And that will be

the initial list we give out to the model, and we could put a limit on it. So we can say, "Give me the top 10. Give me the top five." Whatever it may be. And then every time a tool has been we're updating the graph. So that common tool list stays relevant. It's based on what our model is being So here's here's an example. Um so if

I was to create an MCP server for our Aura API, and I could roughly model in an MCP the Aura API with 26 endpoints. Well, 26 tools, if you like. And if I gave that set of tools to the model every time on startup, that would consume approximately just over 5,000 tokens. if I took the approach I've just been talking about, I can look at my usage

data for the Aura and then I can use that data to say what tools should I give initially, those commonly used tools. And I can see here that top four, based on actual usage of the Aura API, getting information about an Aura instance, creating a snapshot, listing all my Aura instances, and then listing information about the the tenants, the projects which I have. So, that top four,

that's my initial And then the ones underneath it, they're the ones which I'm going to start giving out when the model asks for more. And if if I take that approach, for example, I list out eight commonly used tools, so I go for those top eight, for example, and then I give two more, one's discovery, one is execution. Then I'm now down to approximately 2,000 tokens at

startup. And that's roughly about 62% saved. Right. So, that's one one approach that I could take here. And the thing to be aware of is what I'm the MCP server is storing its tools in in my graph. So, it doesn't have to necessarily be a graph which I'm using with my MCP server, it could be an entirely separate And the tool list is living in the graph

itself, and the tools are the graph. And so, you could imagine for things like restful environments where you want to front those with an this is a great way of doing it because you've got or likely to have users information about your rest endpoints. So that can help you with that initial set of common tools, and then based on usage, you can adjust as needed. And it's

entirely possible that you bootstrap this if you've got your endpoints in an open API spec, you could bootstrap from there. there's three possible approaches here. And it really depends on your situation. if we consider what scenario you're in. So, we know from the NCP specification that it will always ask for the list of tools. So, I know that the full tool list approach will always work. I've

got fairly good confidence because I'm giving out that common batch of With the graph backed approach, that's good as well. >> Um lazy loading, like I've mentioned before, you're relying on the model. If I also consider questions like reducing my initial context, is it deterministic? Will it work with any type of client? So, by that I'm talking about am I using something like Claude Desktop or open

API ChatGPT, or am I writing my own agent? That's one client need to consider. And then I also want to think about can this adapt over time and adjust to what's actually going on with the way my NCP service been used? And then finally, will it scale? So, those are some of the questions you want to go ask yourself. And then you can see how each of

those maps to those questions. And in in many ways the question you're kind of not really asking is which pattern, it's understanding the trade-offs. That's the question you really need to look at. And I've put in that text there that black box with that white text. >> This is literally when I asked Claude why it had taken a particular approach. And I wanted to understand why it

hadn't some of the tools which it had available to it in the MCP server I had. And Claude told me that you essentially you cannot tell me what to do. it cooperates and it does so voluntarily. Not because in my tool description I'd used the word mandatory or I described it as you must do this. And so that's one of things you you need to be aware

of. So, [snorts] if you're writing the agent, so I'm writing some kind of chatbot for then I've got much more control over what's going on. And again, if you ask Claude about what's the hierarchy it uses to determine what it will do, the ultimate determiner if you like right at the top is Anthropic itself. Then the agent, then the user. And so that's how it decides it's

going to do a thing. Ultimate control is with Anthropic. But if you're writing the agent, you have a bit more control over And so in that environment, you can use lazy loading if you wish. Right, cuz you have more control. If you don't own the agent and you've got no real idea who the clients could be, so I'm hosting a publicly available um then I'm looking at

listing all the tools, like doing that I could also the graph-backed approach in all of these cases as well, if I want to have some kind of responsive MCP server that almost gives the impression it's learning over time. The other thing you you need to bear in mind is um go read what the MCP spec says. And lean into that. There's no point in trying to fight

that. Leverage the fact that information is always sent on startup. Like the tools list call always happens. And then, when you consider your MCP think about what the large language model will see and at what time it sees it, and can you do some controls around that? So, those are some things, some patterns, some thoughts, things to consider when you're designing your MCP server. Um so, I'd

like to thank you for your your time today. I've >> I'm not quite sure how to help you with >> Uh that's uh Alexa dropping in and prompting me. Probably classic example of of models. Um thanks for your time. And I shall uh let me see if I can cover off some of these questions in the couple of minutes we've got left. Um the graph of tools

manually. with the example I gave, I actually had usage data coming from an API, which was in use today. So, I could use that to determine what my common tools would be. Um if you haven't got that kind of information, then you can take an educated guess what those common tools should be. Start with those and then use the data you're getting back from your NCP which

is updating that graph, which is recording that usage to start to um trim and adjust that list. the order of cooling tools, um yep, that could vary and that's another thing you could start to bake into that graph. And you could start recording in the graph that models were using tool A and then they're going to use tool B. And that could help you categorize tools, but

then also in your tool descriptions, you can tell the model it's often the case that tool A is is then called and then tool tool B follows it. So, you can bake that into the graph, you can also bake that into the descriptions which you give out with your tools um to help educate your uh model on what to do. Um so, again, thanks for your time

today and um enjoy the rest of the event. >> [music]