Beyond Local Tools: Deep Dive into the Model Context Protocol (MCP) - James Ward

This talk provides an in-depth exploration of the Model Context Protocol (MCP) as it pertains to AI agents, focusing on its architecture and functionality for tool calling with large language models (LLMs). The speaker introduces agent systems, explaining how they interface with LLMs to perform tasks, using protocols for tool calling and data retrieval. Key topics include the JSON-RPC format, the development and role of MCP in providing standard communication for AI systems, and practical usage demonstrated through code examples using the Spring AI Java SDK. Further, the session discusses advanced features such as elicitations, sampling, and the handling of authentication for MCP servers. Overall, the talk emphasizes the importance of standards and protocols in building efficient AI agent architectures.

Thanks for joining me today. We're going to dive into MCP and learn all about that. Um in addition to the the nice intro there. I also now work on the Agentyk AI Foundation, which is under the Linux Foundation, where we're pulling in standards for building Agentyk applications. MCP, Goose, and Agents.MC Agents.MD are the current founding projects of the new foundation. So, this is the the home where

MCP lives and we're we're looking at bringing in additional projects into that yeah, so let's let's dive in. We got a lot of code to to go through. But, I want to just set a little bit of context for how MCP kind of fits into the broader ecosystem around AI and agents. I'm sure you've seen many AI talks here this week and just to to refresh you

on what is an agent. Essentially, it's a loop around an LLM and primarily tools to be able to loop with external data, external systems until the agent has decided that it has completed what it it has been asked to do or decides it it wants to give up. So, that's kind of our general architecture for AI agents. And a quick quick little recap for all of you

on how what's called tool calling works with an LLM. So, this is an important protocol that'll help you see where MCP fits into the mix. So, we have our app. If we want to ask an LLM like what is the weather in Berlin, then what actually happens underneath the covers with our AI model is we send a message to the LLM and we include in that message

here's some metadata about tools that you have available to you. And so in this case a tool that we may provide is a get weather tool and the LLM then looks at the thing that the user has requested, which is get the weather, and it looks at the tools that it has available, and then the LLM may respond and say, "Okay, I need to get the weather."

So, it actually responds back to the AI agent and says, "All right, I need you to actually call this tool for me." So, the LLM is just a model. It can't call tools. It doesn't have access to the internet. It needs the agent side, the client side of the LLM to actually perform these tool calls. And so then the agent side is going to make that tool

call. So, this is within your application on your side of things and it's going to then get the weather for for Berlin. Then it's going to make another call back to the LLM and say, "Hey LLM, you asked me to get the weather in Berlin. Here's the weather in Berlin." And then the LLM says, "Great, I've accomplished what the user has asked me to do. I used

a tool." And then it's going to respond usually with like a summary or something like that or potentially more tool calls that need to happen. and then it says, "I'm done." So, that's our kind of basic inference architecture with an LLM and that is has been around a couple of years now and has become the primary way that we provide integrations into AI systems. It's really the

the foundation for for how we build everything that I'm going to show you with MCP. So, if you were before MCP, which was just a year and a half ago or 37 years in the AI time scale cuz things are moving very fast, you may have have before MCP said, "Okay, great. I've got to provide all these integrations to my AI." And so I'm going to build

all these custom implementations, custom tools for connecting to all the different services that my AI system may need to be connected to, and you would build all these different tools, and likely if you built enough of these, you would probably at some point say, "We should probably have a standard protocol for how we actually do this communication between the tools and our AI." And so, sure enough,

Anthropic and others thought about that and said, "Great, let's create a standard called MCP, which is the model context protocol, which gives us a standard way to define tool calling for the LLM." And so, uh it's now a standard, as I said, under the Agentic AI Foundation, and this now allows an ecosystem around how we create and publish tools that can be consumed by our AI agents.

Uh so, that means that you can create your own MCP servers, that means that vendors can can and have created their own MCP servers. Uh Salesforce uh just announced their new Salesforce headless with their MCP server. And so, that means that now you can take standard protocols and plug them into your AI agents. You can also build your own MCP servers, share those, you know, internally within

your firewalls to your customer or to your internal employees, for example. And now we have one standard protocol for doing essentially tool calling as the fundamentals, but we'll see some other pieces in a little bit. Okay. So, great. So, we've got this standard protocol. What does it actually look like? So, the underneath the covers of MCP, we use JSON-RPC as the packaging format to actually be able

to to make these standardized calls to our our MCP servers. So, JSON-RPC has information about uh the tool call the tools that are available, uh the calls that actually invoking a tool call. So, this is a RPC format. This allows us the foundation to then build these integrations on top of. Most of the time, we're going to be working with higher-level libraries that have taken this protocol

and wrapped it up into, you know, nice abstractions for us. And so, the MCP SDKs that are out there now are exist for pretty much every language. Uh the Spring AI team created the Java SDK for MCP, and that's what I'll be showing today. But, no matter what language you're in, I'm sure that there is great MCP support already So, here's kind of a a quick overview

of MCP and and some of the capabilities, and we're going to look through these in code. But, here's the essence of what you need to remember. MCP is a client-server architecture. We've got our client, which is our agent side mostly, and then we've got our MCP server. And of course, we can have many different servers that get used in a single uh agent. So, client-server architecture. And

so, what we do is our client is going to through interactions with the LLM generally decide when to make a call to the MCP server to perform a tool call, which we'll get into in a minute. But, it can also do other things beyond just tool calls. So, the MCP server can expose resources, uh it can expose something called prompts, and a few other things that we'll

walk So, let's actually dive into some code so we can see what this actually looks like in reality. So, uh again, I'm using Spring AI and the Java MCP SDK. And if I want to start with just a basic tool call, which is kind of our hello world of MCP, I can define an MCP tool. It has a desc- ription here, add two numbers. That description is

important because we need to be able to provide some information to the LLM around when it should call this particular tool. And then, we also uh may need parameters to be able to invoke this tool call. Right now, my tool parameters are don't have any annotations on them for this one, but we'll see how we can also add descriptions later. So, it's the LLM that is actually

deciding when to call this tool, and what parameters to pass. And so, in the get weather in Berlin example, the LLM is deciding, "Great, I have a tool called get weather, but it takes a parameter, which is the city, and it knows how to then take that one piece out and fill it into the parameter slot that's needed." But, that description, very important, because that's what's going

to tell the LLM when it should actually use this tool. Okay, so I've started up my MCP server, and really with with Spring AI, this is this is pretty much all the code that you need to define an MCP server in a in a tool. So, I've I've already started that up. Let's go see what this actually looks like in a few different interfaces. So, if you

have an AI coding assistant, of course, you can plug MCP servers into those, and I've set up this particular MCP server. We can see it's the Spring AI MCP demo. I've given it the URL to my MCP server, so that's localhost:8080/mcp in this case. And so, great, we're connected. Now, let's actually give this thing a try. Let's do add six and seven. So, natural language, and when

I send that message to the LLM, or in this case, when Kuro sends that message to the LLM, the LLM gets the list of tools that are available. One of those tools is add, and so you'll see that, sure enough, it has figured out that it wants to call this tool. It's figured out what the parameters should be, and then it asks me for permission. I'm going

to trust this particular MCP server. I built it. I trust it. And so, now it sends a message to my MCP server. That is going to invoke that add tool, and then send the response back to the LLM, and then you see the LLM has has summarized the result and said, "The result is 13." So, that's our very basic tool call with So, great, so our that

works there, but of course, we can also use other things like if you're using an IDE like Kuro or IntelliJ, you can also plug in MCP servers to those. And so, I can see that I'm connected to my Spring AI MCP demo server. We can see the configuration down here for that, and then I can see my list of tools that are available, and then I should

be able to do the same thing right here. Add six and seven. And when I do that, same thing should happen as before where it's going to figure out, "Great, I need to do a tool call. I have MCP server that can do that." And if my network stays alive, network's been a little flaky, but but you I think you get the idea. You know, it's it's

able to do those those tool calls. Okay, great. So, that's our basic basic usage. I'm showing these in the context of code assistance, but if you're building your own agents, you know, for your customers, your employees, whatever it may be, then those agents can, of course, also use MCP servers. And this is where I think that MCP really is most useful is in the case of like

enterprise integrations. So, we've got our enterprise agents that we're building, and we can connect those to our internal data systems through MCP, connect those to the other service providers that we may be using, third-party MCP servers, and so then we can build our agents with our own MCP servers as well. So, you don't always have to build MCP servers for code assistance. That's just one of the

agents that you can potentially use MCP in. Okay, so let's go on to the next interface here, which is there's a package that the MCP folks have created called the MCP Inspector, and I started it up a little while ago. Let's see if we can see that. So, I started up the local MCP Inspector, and this is what I'm going to be using to explain a lot

of the protocol parts of MCP, but this is just a local debugging tool that helps me to visualize what my MCP server is doing and be able to test it locally. So, I've started that up. Let's go over to the browser and to to the MCP Inspector. So, there's the MCP inspector. I can put in a URL for my MCP server and hit connect. Then I can

do all the MCP things. And we're going to walk through all the different parts here. But let's go to tools first and let's just go to list tools and then we can see there's add. Um, before I do that, let me show you there's the message history down here. So there's a life cycle to MCP. First what happens is on connection, the client sends an initialize request.

And this is where uh there's a a negotiation between the client and the server around protocol version to use, capabilities that are available on both client and server. And so we can see that information there. So then what I did was I listed the tools. And so we'll see that was another call to the MCP server. And this is showing us the actual bodies of those JSON-RPC

messages. But we can see here's my list tools. We can see the tools that are available. We can see the descriptions. We can see the input parameter schemas. We can see potentially the output parameter schemas, which we'll look at in a little bit. Um, but that's that's the kind of just basics of the protocol. So usually when your agent starts up, if it has MCP servers configured

for it, it's going to do the cycle of initialize, list tools, and then it's going to hold around in memory that list of tools, and then be able to send that to with our LLM request. So um, so now let's go actually invoke that add two numbers tools. So I can put in my parameters. And in this case there is no LLM involved. Let's see if I

can click in the right place. Uh, there we go. There is no LLM involved. This is really just like raw MCP protocol stuff here. I don't have an agent in this case. Um, I'm just debugging my MCP server, which has no LLM. The LLM exists uh is connected to the the agent side, not the MCP side. Side note, you can have an MCP server that is also

an agent. We uh I this paradigm uh agents as tools. and so this is a way to do agent-to-agent communication with MCP, where we have not just an MCP server that has access to data, but also has access to an LLM, and therefore it's an agent itself. And so that is possible, but this particular no agent involved on the MCP server here. Okay. So that's our our

MCP inspector. We're going to be using the MCP inspector to to walk through a number of other parts of the MCP protocol. Tool calls are kind of the foundation of many of our agentic architectures now, but there are other capabilities that we can use when we're building our agents that if you're building your own custom agent, then great. You can use, you know, all the capabilities that

you want of MCP. If you're just distributing an MCP server for others to use, then it depends on what that client actually has capabilities for. So maybe it supports resources, but not elicitations. We'll get into those in a little bit. And so it's up to to essentially the client side, the agent side, what it wants to do with certain parts of the MCP protocol. But if you're

building your own agent, great. You can, you know, use all the parts of MCP. Okay, so that's our basic tool call example with MCP. Let's go back to the code, and now let's look at more kind of filled out using more of the parts of tool calling. So first thing is I now have a data object, a multiply result, which is kind of trivial, but gives you

the idea that we can not just pass kind of the primitive types, we can also pass larger data structures over MCP as well. We can take those as input parameters, we can return them as output parameters. We always include in MCP the input schemas, so what's required to call it, but it is optional if you want to include the output schema in the actual metadata that gets

sent to the LLM. So you'll see that on this one, generate output schema is set to true. And then I've also added in some additional annotations here. So, this is like additional metadata that you can specify on your MCP tool. So, I've told it that it's read only, it's not destructive, and it's item potent in this case. And so now I've got my multiply, it returns a

multiply result, and now I've also added some additional metadata to the actual input parameter. So, there's an MCP tool param annotation in Spring AI, and so in in this case we can specify some information descriptions about the input parameters as well. So again, natural language to give the LLM some information about what it needs to actually fill into a particular input slot. And then we can say

if it's required or not, and there's probably some additional parameters there. Okay, so let's go try out our multiply one here in the MCP inspector. So, we'll go to multiply, and 5 * 2. But one of the things you're seeing that's a little bit different here is now the MCP inspector is showing us the output schema for this particular thing. And so, if you as if you're

building your own agent, then you could even use these output schemas when you get a response back from the LLM. So, up to you how you want to use that output schema, but you can optionally turn those on. But let's run that thing and just make sure that we get the result 10. Great. So, you'll see that now that came back as a structured JSON object that

should conform to the schema that was also specified there. Okay, so that's that's kind of our basic We're going to use We're going to do a few other tool calling things that add in a few additional things that are available in MCP. So, the next one is there is the ability in MCP to do to send messages from your MCP server back to the client, and you

can there's a number of different ways that we would want to use these. One is just like login. So, context.info here is saying, "Hey, send send the client a login message at the info level to tell it something." So, let's go see what that looks like here in an MCP So, that one was called my UI got all There we go. Let's scroll back up. Okay, subtract.

So, now if I come in and put in some numbers and hit subtract, we'll see that down in the bottom right I know it's now showing me an MCP Inspector that sure enough we got a message sent from the server down to the client. And so, you may be wondering how does a server send a message down to a client and we're going to get into that

part in a little bit. But, there is the capability for MCP to essentially push a message from the server to the client. Okay, so great. So, we can send messages, log messages, whatever. But, a interesting use case for this is progress notifications. So, let's say that you have a particular MCP tool call that takes takes a while to run and you want to be able to provide

progress on how you're doing with that process. Well, there's a context.progress which is going to send a specific like subset of a notification that really just indicates to the client how far it is along in a given process. And so, you can imagine if you're building your own agent, maybe you actually want to show the user visually like how a long how long task is progressing. So,

up to you how you would implement this or up to you know the code assistance or whatever how they would implement these messages, but they can receive them and do whatever they want with them. Okay, moving on to the next piece here is in the case of Spring AI we've done all these synchronous calls for our tool calls, but of course you can also do asynchronous calls

as well which in the case of Java and Spring I would return a mono which is our async construct for for a deferred value. So, that's another thing we can do is is run these things in async mode. Okay, so now let's look at a little bit more of an advanced feature of MCP. There's this concept of elicitations in MCP, which is pretty new. And elicitations allow

an MCP server to trigger human in the loop flows. So, if something is needed, always needed to be able to call a tool, then you're going to put that into parameters. But sometimes the logic inside of your tool call may realize that it needs more information from the user. So, I need to to elicit something else from the user in order to continue. And so, we can

trigger an elicitation and you really only want to use this if it's sometimes. If it's all the time, put it into a parameter. So, a good use case for this is let's say that I've got like a flight search tool and I've got a user profile. And I in my tool call, I go in look up the user in my my system of users and I realize,

oh okay, this particular user, they don't have a preferred airline set in their profile. Others might. And but I don't want to continue my search of flights until I get what their preferred airline is. And so, in that case, you are sometimes triggering this elicitation and asking the user, what is your preferred airline? And then once that's entered, then the tool call continues. So, let's look through

the code for how this actually works. And this is my my silly contrived example of sometimes. Sometimes it's just a random Boolean. So, randomly we will trigger an elicitation to the user. And I'm going to also check to make sure that elicitations are enabled on in the context. And then I'm going to send an elicitation request for a typed object, which is this user random number. And

then, once the user has accepted the elicitation, meaning they've provided the input, then I'm going to assign what they provided to that maybe number value. So, that's uh the sometimes we're going to ask the user for a random number instead of um instead of generating a random number every time. So, if we didn't need to do an elicitation because that Boolean was false, then we're going to

just generate an actual random number. So, let's go give this one a try over here. I'm going to hit go to my random tool and hit run, and you'll see that triggered the elicitation. It sent me over to this other UI where now I'm being asked for my random number and put that random number in. And so, that was that triggered the elicitation with the human in

the loop. So, MCP Inspector is again just giving you this like debug view of elicitations, but if you're building your own agent, then you'll need to come up with, you know, how do you want to handle these elicitation requests? How do you want to uh you know, push that out to the chat client or whatever you're building to be able to do that human in the loop

cycle. Let's run it again see if we can get There we go. So, that time uh it it uh Boolean was was true and so um or false and so it then generated an actual random number. So, we didn't do the elicitation that time. Okay, so that's a bit more of an advanced feature of MCP, but enables some pretty cool stuff. Okay. The next uh feature of

MCP is also a bit more of an advanced feature. Uh this one is called sampling. And sampling it's it's kind of this weird concept because we've got our MCP server over here. We've got our agent over here. Our agent has access to an LLM. And as I said before, the MCP server may or may not have an access to the LLM. But what if the MCP server

doesn't have access to the LLM, but wants to be able to use the LLM that's over on the agent side? For that, we can use what's called sampling. Sampling allows the MCP server to ask the agent to take something, run it through its LLM, and then give the response back to the MCP server. So, there's a bit of It's a bit backwards than the usual like client

calling a server. It's in this case, I've called the server, but then the server is going to say, "Hey, I need you to sample this." And then the client's then going to send its result back to the MCP server and then be able to continue the So, the way that this works in uh Spring AI is we are going to make sure that our sample uh sampling

is enabled, and then we're going to say context.sample. This is where we're sending this string over to the agent side and getting it to run this through the LLM, and then we get the result back. So, this essentially blocks here until we have the result back from the agent side from And then in this case, just to show that something is happening, I uppercase that text and

then return it. And if sampling's not enabled, just Yeah, whatever you want to do in that case. Um but okay. That's our simple sampling example. Let's go run that one so we can see what it looks like here. Loud joke, there it Okay. So, you'll see that now I get this UI, which is enabling me to simulate sampling. Because again, I'm not running in an agent. I

don't have access to an LLM in this case. And so, I'm simulating uh what what would happen essentially on the agent side to simulate this. And I'm just going to put in some That's not a very funny joke, but we'll put it in anyways. And Oh, I took too long. There's a timeout on that. Let's try it again. Let's see. Run. Approve. Great. So, now we see

the uppercase text that I entered. So, um so, I was simulating me being a uh LLM. Um but in the case of a real agent, that sampling would happen on the agent side. So, that's kind of our our basics on uh tool calling with MCP. And um that's that's great. This is provides the foundation for a lot of different types of integrations that we can do with

our agents. Uh but there's a number of other pieces of MCP that I want to dive into. So, the next one is one where we um we may need to to we may want to provide resources to our agent. And resources are typically like like uh documents or database records, uh things that we want to the user to be able to go go uh probably go through

some UI to then indicate that they want this resource to be pulled into the context of their conversation. Uh so, to set up what's called MCP resources in Spring AI, we first need to give this thing a URI. So, every resource has a URI, just whatever you want it to be it. Uh needs to be in URI form. And then we can provide some metadata around the

name, the description, that sort of thing. And then there's a few different ways to actually provide the contents of that resource. In this case, I'm just returning a string as a kind of static resource in this case. So, let's go see what that one looks like over on the up here there's the resources menu. I can say list resources, and that was the server info resource. And

when I click on that, it loads that actual content over here. So, uh so this again is just MCP Inspector, just the debugging tool. But if you're building your own AI agent, you may at times want to either give the user or based on your agent actual kind of architecture and workflow, go read resources and pull them into the context. So, this can be used for a

lot of different things. I'll show you one interesting use case for this in a little bit. But that's our kind of basic resources. We can also do templatized resources. So, with templatized resources, what we get is the ability to put a a URI template into the actual URI, and you can do this in a variety of different ways. You know, you can have uh subsegments, whatever. And

again, we have the name and description, but then in this case, we are able to get that parameter and then, you know, go read a particular file, like let's say we've got our files on S3, whatever it may be. Uh we want to go based on that key, go actually read that file. So, that would allow us to do that kind of more dynamic templatized version. Let

me show you what that looks like. So, we can say list templates. Go to my configuration. Now, it's asking me in MCP Inspector to provide the key to that template. And I can just a free form a key in there, but there's another piece of MCP, which once I start typing, you'll see that it is it is now come up with a drop-down list of options that

match what I've typed, and then I can come in and select app.name, for example, hit read resource, and now I've read the config colon slash slash app.name, and it's returned that resource. Uh so, that piece where it did the the um the drop-down list based on what I typed, that's another part of MCP, which is called MCP completions. So, let's go look at the code for those

So, for completions, what we do in Spring AI is we have an annotation at MCP complete. We give it the URI that it's going to trigger on. So, this is the URI is creating the connection between the resource and this completion. And now, we can provide the list of strings. There's a number of different ways to provide these. You can do it dynamically, whatever. But now, we're

able to to uh filter and return that list of possible completions to the user. So, again, in the context of your own agent uh or in a context of a code assistant, it's going to be able to do like tab completion through resource names, that sort of thing, depending on whatever if you're building the agent, however you want to encode that interaction. So, that's MCP completions and

Let's go into the next feature here, which is called prompts. And prompts are a part of MCP that allows us to take a short name and then expand that into a longer string. And so usually where this is useful is if you want like a shortcut in your in your agent, so the user is using their agent, they want to be able to type like {slash} greeting

for example, and then you'll that will expand into a larger prompt. So this is really just a way to create a tab completable, knowable way to to have a much larger prompt behind that thing. So I've got the greeting prompt. Let's go run that one. I put in my name. Again, we'll see that there's completions enabled on this particular one. So I can complete my name and

then we'll see that it responds back and says, "Here's the actual prompt that would be used on the agent side based on the user entering that prompt." So this one is a little hard to to kind of see how this all comes together. So let me show you this one over in my code assistant. So now if I say {slash} greeting, you'll see that it knows about

{slash} greeting, right? And so now I can tab complete and then I can give it my name. And when I give it that {slash} greeting in in this particular agent, it's going to take the prompt which was "Hello James" and send that to the LLM and then the LLM responds with, "How can I help you?" Great. "Thanks for being so helpful." Okay. So that's prompts. And again,

up to you how if you're building your own agent, how you would implement those And then if you're if you're using some other agent or code assistant, it's going to decide how it wants to deal with that particular feature in MCP. Let's go look at the code for that one. It looks very similar to what we've seen before. We've got an MCP prompt. We give it the

the name and the description. And then these prompts can optionally, as you saw, take parameters. So we can give those parameters names and say if they're required or not, that sort of thing. And then like before, we can do completions, which you saw. Okay, so that's our our prompts. Another feature of MCP that is great and pretty pretty well supported across different Okay, let's see. So we've

got that. Let's go on to something that kind of rolls together a couple of these features that we've seen. So this in the world of MCP in the specification, there's a side part of the specification called MCP extensions. MCP extensions are the place where kind of newer more experimental work is being done around MCP. And so the I think it was the first MCP extension that just

got added and this was just like a month ago is called MCP apps. And MCP apps are pretty cool because they enable us to not just return text. So if you're in a text-based agent, great. That's that's, you know, a good experience. But what if you're in a browser-based agent and you have the ability to do something beyond You know, if you're in a web browser, you

can do full-on HTML. So why not allow MCP servers to render not just text, but also render rich interactive content. And so this is the concept of MCP apps. And so we actually built this on top of the existing MCP thanks and just created some conventions. So I have an MCP app running in my server that let's go down to the next one. So this is a

shopping cart example where I I have my MCP resource. You already saw resources, so that part is familiar. The one thing that's a little different here is I'm setting a special mime type on this MCP resource and I'm putting on this that it's text HTML and it's going to use this profile equals MCP-app. And that's going to indicate to our agent that when it gets this resource

it's a special kind of resource. So, that in that that case I'm just returning a static template. I can go show you that one. I'm using the Java templating engine, but this is all just HTML for my shopping cart app. And you can load in external resources, JavaScript, images, whatever. This is just standard HTML. And so great. So, I've got my HTML. That's what I'm actually returning

in that resource. But then I likely need a way to allow the agent to trigger this MCP app. So, there needs to be some way to like to know when to load this thing onto the screen. And so the way to do that is generally with a tool call. So, when the the tool call happens, uh in this case this shopping uh list tool call happens, then

what we're going to do is we're going to uh just say, "Hey, I need to associate this tool call with the resource that is the MCP app." The way that we create that association is through this meta provider. This meta provider down here is just setting up the structure of uh of metadata for that tool call. So, the metadata in this case is that we have a

UI key that has a sub key of resource URI. And then it has the URI to my resource. So, that's the correlation how we correlate the tool call to that resource. Okay. So, that's all there is to to be able to build uh an MCP app. I'm going to show you what it looks like in Inspector, which is not the the most exciting thing, but we will

see all right, the tool call and then the shopping list. In this case in MCP Inspector, when I run this tool, it's not going to actually trigger the app. It just returns the the response. But you can go explore the apps that are available, the MCP apps that are available. So, there's the shopping list. Oh, it did actually trigger the shopping list. It just didn't switch over

the tabs. Probably just a bug in But, there we go. We see our MCP app for shopping list. And now I get this interactive component. When I use this interactive component, it can actually call back to my MCP server. Uh it can load context into the agent. So, it's it's able to interact with the agent as well. It's not just a standalone siloed thing. It does get

sandboxed in the browser so that it can't do, you know, bad things. But, um but yeah. So, that's that's the shopping cart uh MCP app example. And I'll show you in a little bit something a little more exciting around MCP apps. Okay. So, that's um MCP apps. And that's kind of our our quick walk-through of a bunch of the features of the MCP protocol. And again, everything

I just showed you is is supported uh in pretty much all the languages and SDKs that exist for MCP. Just slightly different programming models for how you do that. Okay. There's a few other things that are in the specification uh that are new and exciting in MCP that I wanted to to give you a little bit of information on. And these are coming into the SDKs now.

So, usually what happened or what happens in the world of MCP is we innovate in the specification. And then the SDK implementations of MCP then add those features after the specification is finalized. And so, two new ones. One is called tasks. And the other is called URL-based elicitations. So, with tasks, what I can do is basically do a tool call that is going to do a bunch

of work in the background and then allow the agent to check in on the progress of those tasks. So, asynchronous um progress uh polling, those sorts of things. And then the other new one that's exciting is called URL-based elicitations. So, you saw the elicitations and how they worked um kind of by default in the in the I don't know. It was the November version of the MCP

specification. But, sometimes you want the human-in-the-loop piece to actually like open a new browser tab to allow the user to like go put in their credit card number number on a on a webpage or something like that. And so, you are all based elicitations is also um in the spec and now coming to the SDKs. Okay, so that's our quick run through on the kind of features

of MCP. I want to go a little bit more into the actual transports. So, in MCP, there are two primary transports that are defined in the specification. There's one um that is called standard IO, and we'll talk about that. And then the other one is over HTTP. Uh but, the key part is that the transports are decoupled from the actual message payloads. So, the message payloads are

the same depending no matter what your actual transport is. And interesting in the MCP specification, it doesn't restrict transports to be only those two, uh standard IO and HTTP. You can you could run MCP technically over any transport. I've seen people build MCP over uh SQS and SNS on on AWS. I've seen them build it over web socket. So, really it's open to whatever. The challenge is

is that if you don't control both the client and the server side, then uh if you publish like a web socket-based MCP transport server, it may be only you that could actually use that MCP server. Whereas the the standard transports are, you know, standardized uh and and implemented pretty much everywhere. So, it's nice that you can potentially build other transports, but um but often, unless you control

both sides, the client and the server, uh you may not want to use something abnormal. Okay, so with local transport, we actually use standard IO uh as the the process. And so, in this case, you're going to start up a fork a local process on your machine. I'm sure many of you that are using AI code assistants have already done this. You've started an MCP server on

your machine and or configured one with your code assistant. it is then started that process and that can be anything. You know, it could be Java minus jar, it could be TSX to run a TypeScript program, it could be UVX. You know, those are some of the common ones, but really could be anything. The key part is that it's forking a process and then using standard in

and standard out to communicate with that process. So great. So we get the same everything you just saw happening in this case locally. And this is great for developers who want to spin up these processes on their local machines, but if you have an agent in the cloud, like this doesn't necessarily make sense for how you would spin up your MCP servers. And so while standard IO

works well, uses the same protocol and you can spin up multiple of course. That for a lot of scenarios is not the right way to do things. So if we want our MCP servers to be networked instead of running forked processes, we can then use HTTP, which is what I used for the examples earlier. So in this case, we you know, have a internet endpoint or potentially

in our VPC or whatever, we have our MCP server and then our agents can connect over HTTP to that server. So that's what we've seen already. Of course, this works with the code assistant, works with your own custom agents, whatever it may be. And same same actual payloads in those messages So in this world of HTTP, I mentioned a few times where there was places where the

server was sending messages to the client. And this is a part of MCP that works great in standard IO mode, but how do you actually have on the internet a client server architecture that allows the server to send messages to the client? That becomes a challenge. And so the way that this is done in MCP is that the client, when it makes a connection to the server,

can potentially ask to open a server sent events channel where it keeps a connection open to the server, which then allows the server to send messages down to the client whenever it wants. So, this is an optional part of MCP, where you can build the this capability with with server-sent events in uh but you don't have to. You can go into a mode of MCP called stateless,

where you actually turn off the server-sent events part, and this uh tells the client when it tries to connect, "Hey, I can't do server-sent events." But in that case, you're restricting the protocol to a subset that is the things that don't require the push. So, like elicitations are something that requires a push, and so in that case, if you want to do elicitations, then your server needs

to support server-sent events. so yeah, so we've got some different options there in the world of HTTP. So, remote MCP servers, there's great they're great there's, you know, now thousands of those out there, many vendors providing their own MCP servers um that allows you to, you know, manage and and host those MCP servers, you don't have to worry about how to update those MCP servers. Um but

for local development use cases, you know, there's some some challenges like like maybe uh maybe the MCP server uh needs to talk to a local resource or something like that. And so, um so in that case, you know, it'd be hard to do with a remote server. So, there's some trade-offs to to where Okay, so one of the things that's built into MCP, which is is a

unique part of MCP, is that there's a way to do authentication of our MCP servers. And if my network's working, I'll show you a quick demo of this piece. Um so, let's go over here. And let's see if this works. Reload. Yay, okay. Internet's working. Okay. So, I have a MCP server uh the I have a few MCP servers um but one of them is for a

book that Josh Long and I are writing about uh Spring AI, and we're going to have an MCP server for our book. But, we you know, we're going to sell the book and we don't want people to get access to the book through the MCP server unless they have actually purchased the book. And so, we need authentication for our MCP server. So, in order to do this,

I I have my beautiful Spring AI MCP server. And when I hit connect on this the MCP server says, "Hey, in order to connect to me, I need you to authenticate." This redirects me to our login page and I'm going to put in my email address and hit email login. I'm using Spring Security in this case, but you can use any uh thing. And then I have

a PIN that I need to go look up on my email. There it is. It's a good thing that it's live demos. You never know, but so far so good. Okay, don't don't take It's a one-time PIN, so don't take it before I get to hit that button. Okay. So, now I have authenticated uh in clouda.ai to my MCP server. So, now I'm authorized to make requests

to this beautiful Spring AI MCP server. So, let's go give this a test and see what this looks like. I've only got the book's not done yet. So, we're still working on it, but I do have one little thing in this MCP server, which is an about page. So, I can say, "Tell me about the beautiful Spring AI book." And if I hit this little plus button,

you'll see it that sure enough I've configured it so that the beautiful Spring AI MCP server is going to get used. And when I hit run there, if everything works, then it's going to go make a tool call to my MCP server. That tool call is going to Oh, it's listing the chapters. Oh, cool. I guess I have a list chapters tool, but Okay. So, it did

now call my MCP server, made the tool call uh to the about page. And then this is an MCP app that my MCP server is returning. So, I've got, you know, this nice rich interactive thing that my MCP server returned to Claude.ai. So, this allows us through MCP apps to add interactivity. In this case, we've just got fireworks, nothing too exciting for interactivity. But, great, we've got

our MCP server now able to do some exciting things within the context of in this case Claude.ai, but could be whatever agent that supports So, that there you saw authentication along with an MCP app. Um I'm going to walk through what the actual protocol looks like for authentication. So, with authentication, what we're going to do covers and not exactly what I just showed you is we're going

to try to make a request to the MCP server. The MCP server is going to respond and say 401 unauthorized, you're not allowed to call this MCP server until you're authenticated. Then the client side, the agent, will load this well-known uh OAuth protected resource page, which gives it some information about how to authenticate. It's going to then do this whole OAuth kind of dance. One of the

tricky parts is that in the world of MCP, my client could be anything, right? Like it like in this case, my client could be whatever agent, you know, is out there on the internet. And in OAuth, typically we what we need to do is identify the client, give it client credentials. But, in the in this case, we can't do that. Does that doesn't make sense. And so,

we need to do client registration. And so, the typical way uh today generally to do this is something called dynamic client registration. So, this is a weird part of OAuth servers that allow you to dynamically register a client. And this is this is what's available today in MCP for how we do authentication. I'll talk in a little bit about what's what's coming down the road. But, so

we do the dynamic uh client authorization. So, now we're authorized client. So, that's what Claude.ai did with my MCP server. And then we go talk to the authorization server to be able to have the user log in. And then once we're logged in, then we're able to get the jot. And then when the invocation to invoke a tool goes to the MCP server, it passes the jot

so that then it can the MCP server can validate the jot against the identity server. And great, now we're authenticated to our MCP server and able to make those calls. So, all works. Okay. So, now on the subsequent requests with the jot, now we don't anymore get the 401 unauthorized. We get the actual tool call response like you saw in the Okay. So, that is how authentication

works with MCP. But, there's a few nuances and different approaches here that I want to walk through cuz this is a important part of MCP is is auth. So, one of the things that we can do with MCP is provide static credentials to the MCP server. That's with OAuth. So, that's an option. The new thing that is coming is called SIM D. This is the client ID

metadata document. And this is kind of a replacement for the dynamic client registration. So, there are some challenges with dynamic client registration and so SIM D is becoming kind of the new way that we identify clients to the MCP server. So, this is a document that you put into a well-known location. And this allows you to This allows the MCP server to know who the client actually

is and validate So, then DCR as I mentioned is kind of being phased out. Which is what is currently supported and what we're using in the examples today. Okay. And then you can do things a few other ways. You There is kind of older MCP auth auth extension for client credentials. You can also just pass API keys in HTTP headers to authenticate. So, a few different ways.

And again, it depends on like, are you building the client? Are you building the agent? Or are are you relying on an external, somebody else's agent for what is going to happen here? If you control both the client and the server side, you can use whatever method you want for authentication, and you know, whatever makes sense for your use case. Um but uh if you don't control

the client, then you're going to have to go with what they support, which in this case on cloud.ai, for example, is and most AI code assistants is going to be the DCR, the dynamic registration. So, Okay. So, that's our run through on auth things. Um I The example that you saw, uh I've got code for that all on my GitHub and stuff, but uh this relies on

a project in the Spring AI community called MCP security. Uh it does allow us to take to like authenticate the user at the agent side, pass that authentication information down to the MCP call so that we can know who the actual user is both on the agent side and on the MCP call side. Um so, this is important because we don't want to open holes into our

data, uh allow users to access data that they shouldn't have access to. So, propagating that user identity down through the call chain is important, and uh this MCP security project enables Okay. So, that's you can do that in the application layer. You can also do it at the infrastructure layer. There's um there's proxies, and there's something from uh AWS called the agent core runtime and identity, which

allow you to manage those credentials and authentication at the infrastructure layer. So, up to you how you want to actually do that. Okay. So, now let's talk a little bit about scaling. So, uh so, we've seen that we can put these MCP servers, you know, up on the cloud. We can run them, access them over HTTP. Um that's all good, but there's some challenges. Once we have

this thing on the on the server, we can have many clients connected to it. I have an MCP server called javadocs.dev, which serves javadocs, and that thing has a ton of clients. So, how do we actually deal with the scaling challenges Uh here for MCP servers? So, let's talk about what those scaling challenges are. Maybe what we want to do is put a load balancer. So, we've

got our load balancer. That allows us then to route requests across multiple instances of our MCP server. All seems good, but there's some challenges because remember how we talked about MCP being a little stateful in if you've turned on that mode and want to use some of the more advanced features with elicitations, notifications, sampling, some of those things. Well, what's going to happen in that case? We've

got a message and even without some of those things, we have some other state with the initialize method. So, we're going to make our request through the load balancer and let's say that the load balancer sends that first initialize request to the remote MCP server number one, but then the client makes a second call to the MCP server for listing the tools, that list tools goes to

the server number three, that actually will actually be an error in the the stateful mode of MCP because there's an ID that is shared across both of those requests that needs to be correlated. And if we land on a different server, then there's no no way to make that correlation. So, so that becomes a challenge. So, what do we do about this scaling challenge? Well, one of

the things that you can do potentially if you if your load balancer supports it, is there is often a MCP ID header that is sent along with an MCP request. We can then partition we can do sticky sessions based on that particular header. So, that would allow us if a session is 1 2 3, then we can make sure that those requests get routed to the same

server. So, that's one possible way to deal with that. And you know, if your load balancer supports that, great, but there are some trade-offs to this. We're we're pinning a bunch of requests to a given server that may not allow us to actually balance the load correctly across all the different servers that we may have. And then what if we're doing serverless and have, you know, basically

a server per request, that becomes an even bigger challenge. So, what do we do about this? So, one thing we can do is we can we can turn our server into that stateless mode. And when we turn it into stateless mode, then as I said, we lose some of the features of MCP, but then we can scale horizontally without a having to do sticky sessions or any

of that. Um so, let me just show you real quick in the world of Spring AI, if I want to go over to my application properties, you'll see that there is the protocol that my server is serving and I can put this into stateless mode there. When I put it into stateless mode, let's actually give this a quick little try and see if this works. I'm going

to reload. Yeah, that's fine. Let's restart this thing. Uh when I switch this thing over to stateless and start this the server back up, let's go back to our MCP inspector and let's now hit reconnect on my server. So, the same actual like JSON RPC messages as before, but now if I go over to my tools, you'll see that a bunch of my tools have now dropped

out of the list of available things. Those were the tools that were using the stateful parts that are not available when I'm in a stateless mode. So, this may work fine for a lot of your use cases, but again, it does reduce the number of features of MCP that you can use when you run it in this way. Okay. Uh so, that's stateless. Let's go back here.

There are a few things happening on the MCP specification side uh to um to try to address some of these these issues with requiring the statefulness. So, um there if you're interested, the SEP 1442 uh in the MCP um specification group uh is currently looking at how to potentially come up with some better solutions to this problem. So, that's that is state. But, um let's see. We

said that already. We can go into stateless mode. Great. Okay. Then, you can also use hosted providers of MCP that manage the state for you. So, again, in the case of AWS Agent Core, it's going to actually do this routing to the same server to keep that to manage that state for you. You know, it's a smart load balancer that knows about stateful MCP. So, some different

options that you have available if you do really want to be stateful. And uh and I think there's some other open-source proxies and that Okay. Um for a couple minutes, I want to talk about one of the the things that you may have heard about MCP is MCP is dead. It uses so many tokens. Um MCP is definitely not dead. You know, it's uh being used um

very widely at this point. And there are ways to deal with the the token uh consumption thing. So, but there are some challenges. So, when we have our MCP servers, um we uh we potentially in your agent or in whatever agent, we potentially are loading many different tools. You know, it could be 10, hundreds of different tools into the context of our LLM. And there's some downsides

to that. One is that we uh we as we add more tools, we increase the number of tokens on every request to the LLM because there's a lot of metadata included in MCP. We've got the descriptions, we've got the schemas, potentially output schemas. So, there's a lot of metadata that's included. And by default, most of the systems out there today, most of the agents are going to

just load all the tools from all the MCP servers that you've configured in. And then the other downside is that when we uh do when we load in all these different tools, if we have potentially overlapping or seemingly overlapping tools, it gets harder for the LLM to reason about when to call which tool. And so, uh so to deal with these problems, what we can do is

a few different approaches. I'll talk a little bit about >> [clears throat] >> CLIs and skills later. But, here's one of the approaches that we can do and uh to to deal with this problem is tool filtering. So, there's different approaches to tool filtering, um but the basic idea is that instead of providing all of the tools on every LLM call, what if we only provide a

subset of those tools? And you can do that in a variety of different ways. You can manually filter your tool list. So, if you have a large agentic application, you likely have multiple kind of orchestrated parts of that application, and you probably don't want to give every single piece of that access to all the tools. So, you can use what's called tool groups or tool filtering to

say, all right, at this particular point in my workflow in the LLM, I only want want to give you access to these tools, and then at this point I'm going to give you access to these other tools. And so, this is one good strategy is break down your system into pieces of a workflow orchestrated, and then use, in the case of Spring AI, use a chat client

that's using a different tool sets across the different parts of that or tool tool filtering. So, that's one approach. Um you can use products called MCP Gateways. There's now a bunch of those that are out there in open source, and they can do a similar thing where, let's say we have a bunch of different tools available. Well, our MCP Gateway can decide how to do uh the

right kind of filtering so that the MCP client is only getting a subset of those tools. So, that's one of the common features of MCP Gateways. One of the strategies that gets used here is called semantic filtering. Uh so, what it happens is when the message uh is sent to the the MCP Gateway, or actually, before that, when we are giving the LLM the tools that it

has available to it, we can say, I actually have a tool that will allow me to search for tools. So, I have a lot of tools available, but I'm not going to tell you all of them right now. I'm only going to tell you some of them. And and then you can search for a different for all of the tools that are you can search across all

the tools that are available, and then it will include just the tools that are relevant to the user prompt. And so, that's semantic search for tools. Uh if you're doing that in the case of of Spring AI, there's a Spring AI community project that is the tool search tool, which does this in the case of of Spring AI. Uh and then the MCP gateways also often have

this built into them. So, another part another thing that you can potentially do is do progressive disclosure. So, if you've used agent skills, agent skills have built into them the concept of progressive disclosure, where the only thing that is included in the the initial call to an LLM is essentially the description of a skill. And in MCP, we have that same thing. We have descriptions. And so,

you can actually just say, instead of including all of the metadata every single time to the LLM, I'm just going to provide you just the descriptions, and then tell the LLM to call you back if it wants to get all of the details about that particular tool. So, this is progressive disclosure. Different clients do it different ways. And up to you how you would kind of architect

this and what libraries you may use to to actually implement that. But, certainly is possible to do progressive disclosure with MCP. Examples of that I mentioned tool search tool. Also, Kiro, which I showed, has something called Kiro powers, which allows progressive loading of MCP servers. Cloud Code has something very similar to the tool search tool that's default in it for for searching tools. So, and then you

can even provide skills that that give some instructions to the LLM to say when to use particular MCP servers. So, all sorts of different ways to actually reduce that that churn. So, we talked about searching tools and how tool search tool works. There's another approach which is possible. I'm not It feels a little too non-deterministic for my likes, but is some people are into that kind of

thing. And so, what you can do in this case is you can actually have your LLM basically write dynamic code that then decides when to call tools. So, you're like it's a a code mode that is writing the code to then call the MCP tools. And so, there's a few layers of indirection here for how that actually works, but it is one approach that people have used

to deal with some of the the the challenges with with MCP and the the large context usage there. So, that's one option. Okay. All of the code that you saw is available on my GitHub. So, there's the QR code if you want to grab that. thank you. I'm on LinkedIn. Connect with me if you'd like and thank you. I hope you learned something about MCP. And I'll

be around if you have other questions or any questions afterwards. So, thank you. >> [music]