Bootiful Spring AI by Josh Long / James Ward @ Spring I/O 2026

This talk, presented by James Ward, focuses on building AI applications using Spring AI, AWS Bedrock, and various integration techniques for Java and Kotlin developers. The speaker discusses creating an AI-powered dog adoption service called Pooch Palace, utilizing features like system prompts, memory management, and agent skills to facilitate user interaction. By leveraging the flexibility of Spring AI, the session covers the integration of different machine learning models and the importance of observability and security in AI applications. Additionally, the speaker explains how to use tools like MCP servers for external integrations, demonstrating both development and deployment considerations in building practical AI solutions.

Welcome everyone. Welcome. Good morning. Thanks for being here. I'm James Ward. I work at AWS. What do you do? I work on the Spring team. Good morning. Yeah. So, today we're going to What What are we going to do? Selfie. Oh, selfie. We got to take a selfie. Always. Always. Josh, why do you take these selfies? Always. Same reason as always. I'm going to show them to

my kid. I'm going to say, "See, they listen to me." Okay. I listen to you, buddy. Okay. What am I doing? With the selfie. It's got It's not a selfie if I don't take it. What? Oh, it's got to be Okay. There you go. Oh, there we go. It's going to give me time to turn the computer. Oh, yeah. You got to turn on your computer. [clears

throat] Yeah. Okay. So good. Okay. So, what we're going to do is talk about AI. Have you had enough AI yet this week? Okay. We're going to make it fun. This is going to be the fun AI session. So, what a great time to be a Spring and Java developer or Kotlin if you like Kotlin. I like Kotlin. And so, we're going to go through just some

great ways to build AI applications. There's our information. Oh, let's see. We got yours. Josh and I have a podcast together, Coffee and Software. Yeah, YouTube channel. Yeah, YouTube channel. That's what it is. It's like a pod video podcast. Vodcast, do you still call them those? Hey, in the back, can you all see this? Font size? Does it need to be bigger? They say bigger in back.

We could tell them to move forward. >> [laughter] >> Um Josh is writing our slides right now. >> Yeah, there you go. Sorry, we're a little behind on creating our slides. So, [clears throat] so we'll get those those created. Can you Okay, there you go. Uh that that wraps. That's a little unfortunate, but it wraps. Uh okay. Oh, here wait. Add one more URL to this. Okay.

Beautiful springai.com. Boo. Do you remember how to spell beautiful? Beautiful springai.com. Josh and I are working on a book about spring AI. It's not out yet, but you can go there, get information about it. But, that's actually where we want to start is we were like This is like yesterday, I don't know, after a couple drinks and we're like, "Hey, let's build an MCP server for our

book because then if you don't want to read our book, you can just have the AI read it and tell you about it." Just Yeah. And so, we're like, "Let's create an MCP server for our book." So, we did. So, go show them our MCP server. Okay, here. That we have set up here. So, this is in Claud. Here, you ask it. Okay. So, first click on

that little plus button. So, plus button. And then connectors. So, we added There it is, the beautiful spring AI. So, it's enabled. So, we added our MCP server and it has a login server. We're going to show you how to build all this stuff in a couple minutes. But now, you can just go ask it, "Tell me about the beautiful spring AI book." And that's going to

go off and actually call our MCP server. And then we're using this new thing called MCP apps. And you have to allow it. Yep. We're like, "Yep, sure. You can talk to our MCP server." >> know why? Cuz it's an iframe. >> That's right. And there we go. That's an MCP app. That came from our MCP book server. So, we're going to show you how to build

some of this stuff with spring AI, but >> And this is kind of this kind of underscores the point, right? The AI is We talk about in artificial intelligence these days, I think it's real value, as we kind of underscored in the keynote yesterday, is in the context of existing business applications and the and unveiling new and and deploying new interfaces, new There's like new interaction action

paradigms, new services that our users are going to interact with. Maybe they're going to use cloud.ai, maybe they're going to use some other AI system. Maybe we're talking about the world of code assistance. We want to provide integrations into those. So, yeah, great time to to like bring that world to the integration world that we've built so many things on in the Java community. I think Yeah.

Okay. So, with that established, let's build something. Here's the This is There we go. We have some questions we need to answer first and foremost. Wait, no. What are we building? Yeah. Oh, yeah. We're going to build this We're going to build something, uh as always, to help with the laudable goal of helping adopt uh everybody's favorite dog, Prancer. And by which we've spoken once or twice

before, right? And Prancer, who knows Prancer? Everybody knows him. The best dog, right? He's the best. So, this dog is again, the owner was looking for a new home back in 2020 to find this dog. She described this dog rather hilariously, "Okay, I've tried I've tried for the last several months this to post this dog for adoption and make him sound palatable. The problem is he's just

not There's not a very big market for neurotic, man-hating, animal-hating, children-hating dogs that look like Gremlins. We're going to give this dog a home, okay? I mean, if AI can do anything, it should be able to find Prancer a home. So, let's let's build that. Yeah. We are going to use AI. There are Spring AI supports integration with a number of different models. We talk about AI

models, obviously this is a good broad term. Most people are are familiar with chat models, but there's also image models, there's transcription models, there's embedding models, there's you know, what else? Yeah, model for everything at this point. So, what are we going to do? What's our What's our model de jure here? So, we need a model provider. In this case, we're going to use Bedrock, one of

the options available. Bedrock is an AWS service that provides many different models. There's the Claud models. We can go into the Bedrock UI and we can go to the model catalog and select from. There's like 250 different models that we can use. We're going to use the Nova models, which are Amazon models, but you could use the Claud models, whatever. So, lots of different options for that.

Yeah. Oh, yeah. Even JetBrains has, which it's under I think now the market. Oh, right. Okay. Is that right? Yeah, that's right. It's right in there. Like I don't Where is it? Somewhere in there is the JetBrains models. So, yeah. I mean, there's the So, the models in IntelliJ that do the like code completion in line, those are You can use those. Like it's crazy. Lots of

different models. Okay. So, we're going to use that We've already set up Josh's machine to be authenticated to AWS. And so now, we can start making model invocations, which is just like, "Hey." In this case, let's send some text and get a response back from the model. Right. And so, when you use Bedrock, you're you're talking to a network service that is in turn going to route

you to any number of different models behind the scenes. That's right. So, you have in Spring AI itself, you have portable service abstractions. You have the choice about which model provider you want. And then again, you have extra indirection in in and uh uh flexibility here Yeah. for your effort by using Bedrock, right? So, first things first, we go to start.spring.io. We're going to call this assistant

because we're going to assist people with building a new adoption thing, right? Um and then I guess we need some dependencies here. We need Postgres, but I need also a vector store, don't I? Yeah. So, we're going to be doing a number of different ways to do integrations with our AI. So, we're going to use RAG, which uses embeddings. We're going to also do auth. So, we're

going to need the auth pieces. We're going to do MCPs. We're going to need those. The vector store to store our vectors. So, and then what are we doing? JDBC? So, we got a bunch of different pieces that we're going to all pull together to build this dog adoption service. So, PG vector though is kind of a two-for-one. I need a Postgres access and I need a

vector store. In this case, Postgres can do double duty. Same network server, same process in a container on my machine, but two different data structures, two different data stores for the price of one. However, logically, they could be two different things. When you build this app yourself, you could choose Chroma, Weaviate, or Neo4j, or MongoDB, or Elastic, or or whatever as your vector >> different options for

vector stores in there. Just forever, right? Um we've got the web support, got the security support. We need actuator, obviously. Uh what else do we need? We need We need the models. So, let's talk about Bedrock and Bedrock Converse. What are What's the difference? Yeah. So, we're adding two different libraries there. Bedrock is going to be for our embeddings, which are going to be used with RAG.

And then the converse one is for our chat client so that we can talk to the model. Send it a text message, get text back. So, those are our two different libraries we're using there. So, okay. So, you got project all stubbed out, ready to go. And yep. Move that over there. There we >> Oh, yeah. Got to get everything all set up cuz we're going to

give everybody the code afterwards. So, >> Yep. That will be online. And so, you know exactly what not to do, you'll just see our terrible code and be like, "Oh, no. No, no. I'll pass. Thank you." Okay, here. So, the font. I've been told it's very hard to see. Can you see that in the back? How's that looking? It's okay? They say I think I think thumbs

up means good. Maybe not. We'll see. Okay. So, we're setting up some properties here to be able to connect to our database, which we're going to use to house our dogs. It's like a dog house, right? In this case. So, yeah. So, we got our dog house where we're going to house them. So, that's our database connection to Josh's machine. Dog house is the name of our

Oh, you're starting over with dog [laughter] house. Okay. back to setting up our properties. We're going to have to set up our database connection. Then we have to set up some different properties for which model that we're going to use on Bedrock. We have to set up some properties for our vector store. And so, Josh is going to set all those things up. Did you >> just

starting over again? Oh, there we go. So, we want the agent Yeah. It's We don't see Josh copy and paste much, but for this, we'll speed this up. Okay. So, we've got some security stuff that we'll get to in a little bit. And then what other configuration was in there? Did you miss Oh, chat memory. Forgot Shame. Man. Start over again. So, yeah. We'll get there. Okay.

Okay. What have you got? So, okay. We have database set up. Security set up, we'll get to. MCP set up. Do you want to do that initialize true for now or cuz that's for security. Virtual threads, why not? [sighs] >> threads, why not? Um then let's see, we've got uh our properties for which models we're going to use. There's a chat model, which is going to be

the uh Nova Pro, and then there's the >> Oh, right. And the system prompt. We also need a system prompt. So, okay. We've got this thing. Let's ask it about Like uh do we want to ask it about uh dogs, right? Do you have any neurotic dogs like friends here, right? So, okay. It's giving us a response, but it's not the response we want, because it doesn't

know we're trying to adopt a dog. It thinks we have one and we're trying to figure out how to deal with it. Um and that's just not the case, is it? So, and if you look at this, we actually have uh you know, we we have a goal here. We're going to help people adopt a dog from our fictitious dog adoption agency called Pooch Palace. We need

to focus this model on that mission, and one way we can do that is by giving it a system prompt, and I happen to have a system prompt pre-written. And so, we can just we're just going to copy and paste that. We're just going to cat that and then pipe it to the the clipboard, and then we'll paste Wait a minute. Uh of course. You got to

dog that file, man. Yeah, we're talking about dogs >> Much better. Come on. Come on. Yeah. var Okay. system So, let's get our system prompt in there. System prompt just like like put some gravity on this LLM call in a specific direction, gives it some personality. So, You are an AI-powered assistant to help people adopt a dog from the adoption agency named Pooch Palace with locations in

Antwerp, Seoul, Tokyo, Singapore, Paris, Mumbai, New Delhi, Barcelona, ¿adónde estamos? Uh and San Francisco, okay? Uh and London. Information about the dogs available will be presented below. If there's no information, we turn a polite response suggesting we don't have any dogs By the way, the AI doesn't care if you misspell. Yeah. If somebody asks you about animals and there's no information in the context, then feel free

to source the answer from other places. And if somebody asks for a time to pick up the dog, don't ask other questions. Simply provide a time by consulting the tools you have available, okay? Keep all that in your head. There's going to be a test. All right. So, I think we're good. Now try again. >> yeah. Let's let's check it out. See what Well, I would Ah,

there you go. So, it's better. So, it's like now like like yeah, knows that we're talking about Pooch Palace. We've given it this mission that the agent can now uh try to help us achieve. Right. >> So, the next thing that we want to do is provide an integration into this application, and one of the ways that we can do integrations with our AIs is through something

called agent skills or just skills. And skills are kind of like a wiki for the AI that allows us to progressively load in knowledge into our AI. And it also allows uh if you use skills in the way that we're going to use them, it allows this like progressive discovery of the actual knowledge. And so, we have uh a library that we're going to use, which is

the Spring AI Agent Utils library. It has support built into it for skills. And then there's a project called skills-jars that allow us uh the way a way to package up skills, put them into our Maven repository, Maven Central or your Artifact Nexus, whatever. And then we can then consume those skills as dependencies. So, a skill is basically a folder with a skill.md file, and that text

is weighted more heavy. It's weighted heavier than the knowledge that the AI model has. So, if you tell it, hey, you're an AI model, you probably know just about everything on the internet, but you got something wrong. I've got a skill file here. If somebody asks a question that can be answered by both what's in the skill versus what you already know, prefer what's in the skill.

Yeah. It weighs It has a precedence over the knowledge that they would give you otherwise. So, you you went to skills-jars. Well, you created skills-jars.com. Thank you for that. You're welcome. Then you went there and published a skill that we wrote Yep. for Pooch Palace. That's right. As a jar in Maven Central, and now we have that jar here, Pooch Palace. And this is nice cuz you

can version and audit and supply chain manage the skills that you use in your application in the normal way. We have that here. And if you go to GitHub, it's j you know, github.com James Ward, right? Pooch Palace, and there's this skill uh whoops. This I forget what it was. Yeah, there it is. skills Yep. And Yeah, let's go look at our I don't know, do you

want to look at our cat skill? Yeah, we got So, as Pooch Palace, we have some unique things about the pets that we have available for adoption, and so we wanted to encode that knowledge into a skill so that we can augment the AI with, you know, our very particular Pooch Palace knowledge. >> Facts. Facts. Yeah, absolutely. >> For example, most people don't know that cat whiskers

are actually high-gain antennae, right? They're natural Wi-Fi extenders. Absolutely. This is a fact. So, we're going to we're going to use these facts and answer questions about the animals. Now, let's set up the skill here, the skill integration by a via a tool. What's a tool? Yeah, so in order to provide this integration, really the AI has has two ways that we can provide things to it.

One way is through tool calling. Tool calling is the ability for us to tell the LLM that I have some tools available, and if you decide, LLM, that you want to use these tools, just let me know, and I'll invoke the tool on my side, on the agent side, and then tell you what the response was. And so, this is really the the primary toolbox for doing

integrations with LLMs. So, it turns out that Spring uh the Spring Agent Utils library, that has the skills tool in it that allows us to tell the LLM, hey, I've got these skills available. So, in our case, we've got a dog skill, we've got a cat skill for Pooch Palace. And so, then we we indicate to the LLM, hey, I've got these skills available that you can

use if you want to. Then if the LLM decides that it needs some additional knowledge, it can tell us, the agent side on Spring AI, to invoke that skills tool, go get the full file, and then give that to the LLM. And so, all that gets integrated with what Josh just wrote there, right? Yeah. >> Well, let me walk through this here. I've created a skill tool.

This is a way to teach the AI the Spring AI framework to, you know, it's a way to add tools to the tool belt of the model via Spring AI. And in order to make this work, there's a particularity for now that I think we'll probably get better into that, I don't know, where we also had to set up a tool call advisor. An advisor is a

Yeah, this is like an interceptor for our actual calls to the LLM. So, uh it allows us to intercept calls that go that are going to the LLM and the responses that come back. And the advisors just become in Spring the like standard way to build many different uh paradigms. So, we build rag on top of this. We build uh MCP uh tool calling, uh our skill

stuff all on top of this. What's the question? Uh should should I adopt a dog or a cat? And so, now with our uh our skill, what we should be seeing there, if we don't get an internal server error. Oh, try to I don't know, try it again. Turn it off, turn it back on again. Oh, no. What happened? >> What is going on? I don't know.

This is the first time we've tried this. We're just, you know, coming up with it Is this a skill? >> live. Did I do anything wrong? Skills tool? the tool call Let's just try removing >> Oh, try to turn on history. Oh, turn on the history. >> Yeah. >> Conversation history. What could go wrong? Don't answer that. We we already know. Okay. There we go. Okay. So,

>> worked. There we Just have to have the history enabled. So, So, getting the cats are natural Wi-Fi extenders. This makes them, uh, I think a plus That's a plus one in that in that column. Um, So, let's walk through what actually just happened there. We made a call to the LLM and we sent the user prompt We sent the system prompt. We sent the user prompt.

And then we also sent the list of tools that are available to the LLM, which included the skills that are available. And then the LLM decided, "Hey, it looks like you're asking about dogs and cats. Let me have you go fetch the skill, the full markdown file in our case, that has all of the contents with Pooch Palace specific knowledge about dogs and cats." And so then

we sent that back to the LLM. And all this happened in that one chat client call, this whole like back and forth with the LLM multiple times. And then finally the LLM responded, "Given the knowledge of Pooch Palace dogs and cats," it then responded back to us. So, that was the full cycle. >> Good stuff, but the problem is now this is a stateless API. So, if

I asked another question, it would forget the answer it just gave us, right? If I asked the same question again, it would be like, "Here you go. Here's the answer. Brand new. I've never seen you before." So, we need to enrich it with memory, which we're going to do via the prompt chat memory advisor, another advisor here that we're going to use to persist things in the

database here. And then this is going to take a rolling window of the latest and greatest things that you've, uh, said. It's going to be stored in the SQL SQL database here, okay? And we're going to specify that by specifying a repository, JDBC repository.builder.datasource, putting in the data source here. Voilà. And, uh, data source and dot build, okay? >> So, there's many different ways that we can

provide memory to our LLM. As Josh said, the LLM is a stateless thing by default. And so if we want something to be known across calls to the LLM, we have to manage that. And this is one of the more challenging parts of architecting these AI agents is how do we provide memory to it? And so, of course, Spring AI has built-in support for memory, and we

can do that in a variety of different ways. The way that we're using here is that that windowed memory. And so this is just going to roll through, uh, with a maximum of 20 or something messages is the default, and it's going to store the memories into our database as well. So, a really a pluggable system that you can use in many different ways to provide memory

to your application. There's now an auto memory thing that is part of the agent utils library as well, uh, which will just automatically manage, uh, memory compaction and some of these other techniques for you. >> That's in the Spring AI community as well, well worth a check out. Okay, here we go. I'm going to reload all that. I've enabled the schema generation here. I'm telling it to

enable to initialize a schema in which it's going to store memory. So, if I refresh this, you'll see Spring AI chat memory is there. Now, you know, everything goes to plan, uh, the next Before we do that, the next thing it's going to ask us is, "How do I ascribe the request to a particular user?" And the way you can pass in any arbitrary string, and you

specify that string, and you can specify that here as a parameter, right? Now, obviously, you can imagine in a more realistic scenario, such as the one we're going to have later, you would have a principal, which a Spring Security principal. So, let's do that later. So, Spring user, et voilà. And, oh, this font being this shot cartoonishly large is kind of fun. >> They can read it,

you can't. That's okay. Yeah. >> I'm not here to read, I'm here to write. Okay. Okay, so we've got our memory integrated into our chat client. And so now we it should remember things that we talk about. So, um, I don't know, what's what's the Oh, yeah, you got to put in your ID. Let's do a quick little test on So, great. It knows who you are.

And then if you ask it another question, that persistence should kick in and that advisor should be able to pull the memories out of the system, send those along in the next request. Did it work? >> Yep, it did. Next, we we need to give it access to the dogs, so we'll support rag. Yeah, so for rag, rag is our next technique that we're going to use

to build our AI agent that's going to be able to be connected to external data. And so in this case our external data is our database of dogs, and we need to have a way to provide dogs that the user may ask about into the LLM. And so rag is this approach where what we do is we take our database of dogs, we then create vector forms,

vector representations from a LLM, uh, usually an embedding model. We create these vectors that represent each of our dogs. We store those in our case in PG vector. And then when the user asks a question about a particular dog, we're going to do a similarity search on their prompt that they've put in with our list of dogs that are available. And what then happens is we if

we match on a dog, then we're going to take that record and put it into the prompt that's going to go to the LLM. And so that's all the whole rag approach. We're going to kind of cheat here and instead of like setting up our ETL pipeline that, uh, has create and update, um, procedures to create our embeddings on create and update, we're going to manually create

our embeddings. >> Right. So, that's what we're going to do >> When I inject the repository, I go through each record in the database. I'm going to create a Spring AI document with text. It's arbitrary. As long as you're consistent from one run to another, it doesn't matter. Normally, you'd take a large enough body of text and you'd tokenize it, you'd split it, and pass in the

collection of documents from that process into the vector store, and they'd be correlated in a one of five or whatever. But here I've just got one, and it's small enough to fit into a single document. Oh, wait, this is a document about dogs, so of course we shall call it a document. There you go. >> Of course. So, we want to tell the AI model to know

to consult that vector store when it loads up. So, we'll, uh, create another advisor here. So, the vector store is just our general API in Spring AI that we can use to talk to any vector store representation. And so this is in our case going to be the PG vector, uh, but many options that are available as vector store implementations in Spring AI. And it's going to

take time when it starts up this very first time. One of the reasons I love Bedrock's, uh, the embeddings the embeddings is so fast, okay? That's 5 seconds with all the embeddings. If you try it with the distinguished competition, it'll take two or three times longer. Don't ask me why. So, and again, we we're doing this on startup, creating those embeddings, cuz we already had our database

of dogs. In your case, you're going to do this on create or update when you get new records or updated records coming into your system, you're going to create those embeddings. But you do need to have the embeddings, cuz that is a vector that then we're going to use to do a similarity search, uh, with what the user has prompted. >> We found out buddy Prancer, but

it's a giant paragraph of text, and I can't really use that text to pass it to other layers of the API. So, what I'm going to do is I'm going to get something strongly typed. This is Java. We have an amazing opportunity by using a nice strongly typed language like Java. Uh, and we don't have to specify schema for this either and like we do in other

untyped languages. Uh, so we're just going to write a nice little Java object here, a little class. Okay? We'll say record dog adoption suggestion, int ID, string name. I'm going to take that type there. And then instead of asking for the content back as a string, I'll say entity, et voilà. And I'll return this. And ironically, I'm going to return a strongly typed object only to then

pass it back from Spring MVC, which will turn it back into a string again to send back over the client. But the point is I have access that data as a strongly typed integer, right? There it is, int 45, and I can use that for correlation and so on. And there's different ways I do want this. This is done is, yeah, we do want the string here.

So, there's different ways that this is done. Some models support what's called native structured outputs, in which case we're going to actually the LLM actually is going to be sure to return us the right JSON. We usually provide a schema that we want that to conform to, because we have the types and we have Jackson in Spring, all that schema information just gets passed automatically. Uh, in

this case we didn't turn on the native structured outputs, um, but we could. Right. So, now the next natural thing any red-blooded human being at this point would want to adopt Prancer. I think we found the dog of our dreams, uh, or nightmares. So, let's support that process by giving our model agency, the ability to talk to the world. We've already seen other people's tools being used

as Spring AI, uh, skills tool and things like that. We can write our own trivially. Yeah. So, there's different approaches for how we do these integrations with just our own tools or potentially third-party tools, where we can do tools in the same process as our AI agent, our AI application. And that's just using this @Tool annotation. We need that description. That's very important. >> That's your schema.

Yeah, so this is that description is what is really going to indicate to the LLM, "Here's when you should call this tool." So, remember how tools work is we provide all the tools that are available to the LLM, and then it decides when it wants us to call a tool. We call the tool on our side. And so, great, we've got our adoption tool. We'll get into

MCP, which is the other option for tools in a minute. So, great. So, we've got our tool. Now we should be able to adopt this dog, right? Do we still have our our memory? Is it going to send our memory along? >> Let's see. the Barcelona uh, Pooch Palace location. So, if everything goes correct, then that should be 3 days from now. Yeah, today is the 15th.

Yay. Okay. >> Oh, it's tax day. I just realized. Oh, oh boy. Pay your taxes. >> Okay. Um, in the states, not here. I'm sure it's fine. Don't Don't forget. Um, that Did we So, we got our tool calling. >> Did we know that worked? I didn't actually put a breakpoint or anything. Ah, that's so sad. Uh, here. Var I. Oh, you didn't print something. Yeah. >>

sad. Scheduling. If [laughter] it hallucinated, that that would be ironic. Amazing. Okay. Dog ID four. Mhm. Mhm. Mhm. Mhm. Mhm. Mhm. Okay. I wish you had string interpolation there, >> Seriously. And the also operator. Okay. Five. Wait, it might have It might consult its memory, by the way. These things are really smart. Great. April 18th, scheduling 45. Okay, great. Now, what if somebody else wants to repurpose

this business logic, Yeah, so the other option that we have for tools is we can use MCP servers. MCP is a great way to provide these types of integrations. You've probably used MCP with your AI coding assistant. Where I think MCP actually has a lot more value is in these enterprise integrations, cuz it's kind of like microservices for our AI agents. And so, what we can do

is we can take this tool call, move it out into an external server. We can also use MCP servers from third-party providers, and then we can wire our agent with these remote MCP servers. So, Josh has now created a new MCP server. Anything special we need to do? Refactor to Yeah. Yep. So, we're going to change the annotations here from at tool to MCP tool. We could

reuse actually the at tool annotations, but then we need to do one additional step to set things up. So, we'll just change it to the MCP tool. And now we need to wire change it out so our agent is going to talk to that MCP server instead of talking to that local tool. So, to do that, we need to uh Did you set the URL already in

the config? >> Okay, great. And then we're going to uh be able to tell our chat client to Hey, here's this MCP server that you have available to you. What actually happens underneath is when our Spring AI application initializes, it's going to go out to our MCP servers. It's going to get the list of tools that are available on that server, and then populate those tools into

the context into the LLM call that then we're going to make uh so that then it knows about the tools that are There is another approach that we can do for uh really kind of a more efficiency side of things where Let's say we've got a lot of tools. We've got hundreds of tools, and there's some challenges with having so many tools is that one, it gets

harder for the LLM to decide when to use a particular tool. And then also, there's quite a bit of metadata that it describes the tools. There's the descriptions, but then there's also the input and optionally the output schemas. And so, what we need to do is uh often times when we get a large number of tools is have a way to reduce that metadata information, because that

metadata gets sent on every LLM call. So, we can do a tool called the tool search tool, which provides semantic search using embeddings or Lucene to be able to first allow the LLM to search for tools, and then we provide in the subsequent call the actual tools. So, it all worked? That worked. MC We have a distributed system, but as soon as we did that, we have

created a hack, right? Like we have a distributed system. That's another port that people can knock on. We need to secure this thing quick. Yesterday, right? So, back to security. Uh, we're going to set up OAuth. Who loves OAuth? Don't all raise your hand at once. I love OAuth 2, actually. But >> [clears throat and laughter] >> Ah. But it is What does it do? So, we

need a uh standard way to be able to authenticate to our MCP server. Uh, we didn't show the auth step for our book MCP at the beginning, but there is actually an auth server, and that auth server requires that the user, in order to invoke that MCP server, is authenticated. And so, we can do the same thing with our pooch palace scheduling service, and we can actually

propagate user identity through from the agent call to the MCP server. So, that means we can authenticate both on the front door for the agent, but then also know their user identity when we call invoke that MCP call. So, you What do we need to set up for that? >> I'm going to create a authorization server. You can use any valid OAuth IDP as as so so

long as it supports dynamic client registration. In this case, I'm not even doing dynamic client registration, since it's a rather homogeneous setup here. I've got a a a client and service that know about each other in the same organization. They can afford to share and work with and trust the same tokens, right? Yeah. So, they have a mutual source of truth. That's the Spring auth server here,

but it could be Octa, it could be Keycloak, it could be Active Directory. It could be anything that produces and validates and bends OAuth tokens, okay? And I've also configured via some config here a OAuth client. Now, you might do this in a self-service form or whatever, but I'm just going to do that. And uh I'm also going to set up the MCP security, okay? So, we're

going to say uh security filter chain. Oh, oh. >> So, what we need to configure on this side is this this token server that's going to be able to create tokens and validate tokens. Uh, it needs to have some knowledge of how MCP actually works. There's a library in the Spring AI community that adds the this knowledge, this capability into the auth server, so that it will

be able to authorize MCP uh calls. So, we have to set up the the security chain um there, and this is always a tricky one, huh? It's the MCP authorization And then we need to uh allow uh uh uh OIDC. OIDC. Yep. Fun. All right, good. Easy. And then we It's actually more You know what the real the hardest boss in this level is? The font size.

That's It's throwing you off, for sure. There you go. So, I locked down everything. I did form login. I set up the I included an extra authen- an extra uh processor on the security filter chain coming from the Spring AI community MCP security project here, and I've said I want to set up an auth server with uh OIDC, but I'm going to lock it down for MCP

interactions on the auth server. Finally, I'm going to source my users from the SQL database, which we have configured and running in the background there. Okay. Oh, oh. Okay, here we are. Great. So, we'll have our users in our database. We've got our MCP auth set up. Anything else we need to do on the auth server side? Auth server, I think is all good. Now, let's go

back to the uh agent itself, and we have to restore the OAuth client connectivity, which I commented out earlier. So, here we're going to say as an OAuth client, the MCP client, when it tries to make a call, it's going to say, "Hey, do I have a token?" If not, it'll redirect to the auth server and and begin the OAuth dance to allow us to create a

token. Or if it's already got one, it'll promulgate it when it sends a request uh in in band, basically. So, here's the Here's the connectivity on the client side saying, "Hey, go to the auth server and get the token." And then because of this because we're no longer able to talk to the MCP server in a secure way, since it's not We're no longer able to talk

to it without a token, right? Right? We need to tell it to defer the initialization of what it has until we actually make the call with the token in band. So, that's what that property is. There is another alternative that we could go with here, which is you can say, "All right, allow users to get the list of tools unauthenticated, and then only require auth on the

actual tool call." Uh, that's another option, but in this case, we're just going with the easy one, which is deny access to even listing tools until the user is authenticated. I'm bringing back >> Bring back security. Love the security. Love it. >> What do you think the chances are this works? I mean, pretty great. [laughter] I I did it in like 5 minutes yesterday. It'll be fine.

What could go wrong? It's security. It's OAuth. It gets a little tricky, but >> Says nobody ever. I believe in you. >> Okay, thanks. Um, and then finally here, uh Oh, we have to configure the uh customizer in the client, right? That's also important. Oh, yeah. So, so the way that we need to actually have like an interceptor that's going to intercept the calls to the MCP

server and add the token from our web application side, so that then those tokens get passed to the MCP server call as well. So, we need to add that interceptor with our HTTP security customizer, and yeah, we're going to just configure that. Again, this comes from the Spring AI community library from our buddy Daniel, right? >> Ganjinyan level. Amazing work on that. >> and he's got a

Be careful, he's got a clipped wing uh this time. Okay, so we've got a We've got everything except for this. We have to specify the issuer, and we did that, actually. So, this should be fine, too. Let's go ahead and I got the security reload. Uh, there's just Yeah. a lot of risk here, buddy. Okay, I'm I'm root- I'm rooting for us. Ready? 127.000. Oh, we have

to Just to make life a little easier, one last thing Going to Instead of URL encoding the query in a browser, I'm going to just reify the the the question here. So, So, we need to use Josh was using his curl-like thing before, and that's great, but now that we're doing OAuth, it's a little challenging to do OAuth through curl. So, we're going to use a web

browser, and we are just going to hardcode our question in here, so that makes it easier. We don't have to URL encode that thing manually. So, um Yep, when can I pick up Prancer? Oh, and then you Did you do your principal? I'll do that in the server side. Yeah, good. >> Okay. Question. Get rid of the history, so that it doesn't cheat and look at its

own notes. >> Oh, yeah. Can't cheat with memory there. And then on the server side, yeah, we want to take advantage of the fact that we now have access to the current authenticated user. So, security context holder.getcontext.get authentication.getname. Rolls right off the tongue. Okay, easy. Four. Uh, and then it's this user right here, Hello. Take five. Let's see if it works. >> Ooh. Got Okay, I went

to {forward slash} ask. It redirected me to the auth server. So far, okay. So, now it should be calling back. 18 April So, okay, we got a response and now it knows. So, our MCP call then propagated that user identity through and security worked. First shot. >> First shot. No problem. No There's no doubt. Never any doubt. And we have this whole time also been sending data

to Grafana, which is nice. So, oops, wrong What What Here we go. Drill down. Metrics. Look at that. Very important. We got observability. This is important as well when you build product. This is the difference between AI and production-worthy AI. As you want to have observability. You want to know how much you're spending on tokens and how many how long operations are taking and and so on.

Here's your token usage. You want to know if you're bankrupt or not, right? This is important. Keep track of this stuff. Keep an eye on an eye on the single pane of glass. So, we have secure >> didn't have to do anything to add the observability. Just added the Spring Boot starter and pointed it to the right thing. We have now What do we What do we

have? >> We got Let's see. Yep, so we got our AI application. We've got skills. We've got rag. We've got system prompts. We've got MCP tool calling. We've got observability. We've got security. Did we get it all? Production-worthy AI. This is where things matter. I talked about this yesterday in the keynote. The 95 MIT did a thing last year. 95% of AI solutions fail and it's because

they don't integrate in the context of the organizations in which they in which they live. You and I and all of us here, we are the guardians of the business logic that runs the enterprise in Java and in Spring. We have an amazing opportunity here, right? We can build secure production-worthy systems that integrate well, deeply in the organization. Yeah. >> There's never been a better time to

be a Java and Spring developer, everybody. Thank you so much. You. Yeah. Thank you, buddy. Thanks for coming. I I have a question. Who learned something new? Something new, anybody? Fantastic. Who had fun besides us? Oh, very good. Thank you, everybody. Go see his talk if you want a deep dive on MCP. When's your talk? Oh, yeah. 11:30. 11:30. That's today, not like next Tuesday. That's today

in this conference. Deep dive on MCP from a just a roving tour of all things MCP. Thank you. And check out the Spring AI community. We have We've only begun to scratch that surface. Thanks, buddy.