Spring AI + MCP: Building the Missing Bridge Between Java Apps and AI Agents - Siva Prasad Reddy

In this session, the speaker discusses how to enhance existing enterprise applications with artificial intelligence capabilities using Java and Spring Boot. The emphasis is on integrating AI into legacy systems rather than developing new applications from scratch. The speaker highlights that many enterprise applications demand precise results, like banking systems, while others can tolerate some inaccuracies, as in recommendations for travel. He introduces Spring AI, a framework that simplifies the incorporation of AI into applications, enabling the use of various models and structured outputs. Additionally, the talk covers creating chat clients and managing chat memory, allowing for more interactive and user-friendly interfaces. The speaker explains how to build a Multi-Client Protocol (MCP) server to facilitate seamless interactions among different applications and databases, ensuring that AI tools can be leveraged across diverse systems without needing extensive code changes.

In this session, we're going to look at how we can build enterprise applications with AI capabilities. Even better, we're going to look at how we can take our existing enterprise applications built on Java and Spring Boot, and we can make these AI-powered application, because not many of us are going to start brand new greenfield projects, right? We already have a lot of systems, and we want to

leverage all these new AI capabilities. So, in this session, we're going to look at how we can take an existing application, and how we can gradually make it make it ready to use AI capabilities. Okay? my name is Siva. I am a developer advocate at JetBrains, and I have been using Spring from 2006, and I have written few books on Spring and few other technologies. And so,

let me ask you in the enterprises, not startups and all, in the big enterprise companies, so, we mostly build applications that decide deterministic and accurate results, right? Let's say we are building a banking domain application, and if you ask, "What is my current balance?" you mostly expect the correct answer. Like, not a ballpark, "Okay, you have close to 2 2 lakhs rupees." No, we want precise amount,

right? We don't want ballpark answers. But at the same time, there are many use cases where you don't need exact 100% accurate deterministic results. For example, if you are building a a TripAdvisor-like application, where if you ask, "Okay, I want to go for a vacation in Goa, and what are the top three places I must visit?" or something like that. And if you get slightly different results,

world is not going to end. It is fine. Just if you get uh 80% accurate results, that is good enough, right? You don't need 100% accurate results. So, like that, there are many use cases where you can leverage these new AI capabilities to build in much more uh user-friendly way, and even more easier way. If you think about this usual uh way how we communicate with the

systems, uh let's say many of us are using uh WhatsApp, right? How easy it is? Like, you just type, or even nowadays kids are not even typing. They are just recording the voice, and then sending it, right? Now, think of how easy or difficult it is to book a uh ticket in IRCTC. You need to have a little bit of uh PhD, like the tabs, and then

uh it is not so easy, right? So, how about if we can make the application so easy, even uh uneducated people also can easily interact with the systems. It would be uh better way of using technology and all. But nowadays, most of the times our enterprise systems, you have powerful back-end systems, but you have a fixed uh UI, like you select the drop-downs, you navigate through various

pages and all that. But how about if you have simple one big text box, and user can simply type whatever they are looking for, or even better, just speech. They can ask in whatever language they are familiar with, and then if they can get things done, that would open up a lot of possibilities, right? We can make our applications accessible to a lot of uh people. So,

how how can we do that? Uh we already have a lot of systems built, but how do we add this AI capabilities and make it more accessible? Luckily, we don't have to start from scratch. We already know how to build uh big enterprise applications. We already have a lot of experience in building scalable systems using Java and Spring Boot and all that. And we have a very

matured ecosystem in Java, especially. And now uh Spring folks realized that uh going forward, we are going to use more AI capabilities, and they started with Spring AI, which sits on top of Spring Boot. And uh they also uh provide higher level abstractions. Because now if you try to write something and uh maybe you start with uh integrating with cloud. Maybe tomorrow you find it is very

expensive. I want to switch to some other thing. Uh if you tie up your code base with a specific model and all, it is going to be very difficult. So, they uh made it as a higher level abstraction where uh using Spring AI, uh 99% you can make it uh replaceable. Like your back-end model, whatever you are using, you don't have to tie so strongly, uh so

that you can evaluate with the different models, and then you can switch to it. So, uh we are going to see how we can uh use Spring AI in a Spring Boot application. But before that, we should be familiar with some core concepts of Spring AI. I think by now you might have gone through this several times uh already. You might start getting irritated to repeat the

same thing. But I want to quickly uh take a recap, so that you don't get bored. Okay, so let's Uh so assume you have a Spring Boot application, and you want to uh start leveraging AI capabilities. So, first thing to do is you can choose whichever model you want to use, and just add the startup. Step one. Okay, once you have the necessary dependencies, you can inject

this chat client by injecting the chat client builder, and you have a lot of option to customize how you want chat client to behave. But, without any customization, this is how simple you can create a chat client. Once you have a chat client reference, you can say how you want to your LLM to behave. You can say, "Okay, you are a helpful assistant. Always be polite." and

all that. Or you you can even say nice things like uh Actually, I tried behave like a Chandler in Friends series, and then I start interacting with it. It is hilarious. You should definitely try that one. So, you can you can specify how you want your system to behave, and simply ask whatever question you have, and you get the answer. But, here you get the answer as

a string. You ask in a a string format, and then you get back a string. Which is nice. Uh If you are implementing a chatbot, that is mostly sufficient. But, in your enterprise application, you may ask about something, and you need to get data, but it is not exactly you just pass it to user. You may want to take that bits and pieces of the data, and

then act upon it. Let's say you want to get some order, and then try to see the status of it. So, something like that. If you get a textually a paragraph of data about the order status, it is not very useful uh if you want to process it in some other way. So, this is where structured output comes into picture. So, here what you do, instead of

getting the answer as a string, let's say you have a record or a class, whatever, and then you say, "Instead of string, I want to get the answer uh by populating all these details, ID, name, email, DOB, in a record, and then give it to me." So, when I ask LLM, it is going to return me a user profile object. So behind the scenes spring AI takes

care of chart length ask LLM. It also specifies, "Hey, I want the response to be in this JSON format where the fields are ID, name, email and DOB." So LLM takes care of okay, I understand this is the format you are expecting and then it returns a JSON response and spring knows that how to unmarshal that into user profile object. So that is how you can get

the data in a structured format not always as a string. But again, by default LLMs don't have any memory like if you ask okay, this is the standard you see everywhere right? Like first you say, "Hey, I am my name is Shiva." And it will greet you and then again simply by asking what's my name? It says, "I don't know." Okay, so they don't retain the context

across the calls. So how do we make it remember the conversation we are having? So this is where chat memory comes into picture. So chat memory is Let's say we start the dialogue with models and then first we start the conversation, it returns something and then for the next follow-up prompts, you not only send the your next query, you also send all the prior history of the

chart along with it so that they have a context of okay, we were talking about something in the last 5 minutes and these are the conversations we have. So based on all that context it is going to try to return the relevant response. And if you just try to play around, you can simply add this dependency and then by default it uses in memory chat memory which

is okay for POCs, but for production definitely you should use some JDBC, Redis and there are many other implementations where all these conversations are stored in certain persistence media. So, once you add these dependencies, after that you can configure this advisor message chat memory advisor. So, this retains This will persist all the conversations and also every time when you make a chat line request, it is going

to pull all the prior n number of conversations and add it to the prompt as a context. So, that automatically done by this chat not only just remembering the last 10 or 20 chat conversation, you should definitely know Let's say we are building an application where many people can parallelly ask, then we should retain who is talking to and we should only grab those people's chat, not

everybody's chat history, right? So, this is how you can do it. Let's say if you have a front-end chat line, you can take a unique conversation ID as a cookie and then if it doesn't exist, assume that this is a new conversation you are starting and you can add it as a advisor here by specifying this conversation ID. And once you get the response, set it as

a cookie whatever the conversation ID and then return the response. So, for the first time you are going to create a new conversation ID and all these conversation will be associated with this conversation ID. So, the second message, it already know that, okay, I need to pull all the conversations even multiple people can parallelly ask and then it will get only their associated chat history. So, that

is fine, but with LLMs, they can only answer the questions based on whatever the data they trained on. So, if you ask any recent information, they don't know because they are not trained on that. And also, if you ask it to do something, they don't know how to do it because they can only answer you based on the knowledge it has. So, this is where that is

fine. That is fine for just asking okay, random general things. But in the enterprise application development context, that is not what you really want to do. You want to do something with your application. You want to fetch some information from some backend system. You want to act something like place an order or cancel some order. You want to do some activity. So, this is where tool calling

comes into picture. So, what does tool calling do? You can define a set of tools, and it could be simply retrieving some information from some system. Be it a database, be it talking to some other REST API. And then you can return the details, or you can actually perform something. Okay, I think I need to change this. Cancel order. Wipe coding. So, okay. So, you can create

these tools, and once you create such tools where you can interact with any system, or you can return any data in the real time, and then you can add that as a default tools to this chart client. So, once you have that, now you can simply ask, "Hey, give me my order details." Or you can even say, "Cancel my order. This is my order number." And it

knows that by default it doesn't have what to do with this. But once you register these tools, it knows okay, this is the tool available, and I can use this tool to perform that action. So, this is how you can register the tools. So, this is very important. Usually, we know we ignore the descriptions, writing comments, and all that, right? We We are developers. We don't care

about warnings. We don't write Java docs. But in this case, it is very important to write a meaningful description so that based on that, LLM determines, "Okay, I need to call this tool because it seems to be performing whatever user is asking for." So, we need to pay a little bit more So, we can we can extend the LLM capabilities this way, simply by creating a lot

of tools, and you can make it do whatever you want to what if you want to do this same task in a different application also? Let's say, in your enterprise, there are many applications, and you want to perform these similar activities from a different application also. Of course, we have copy-paste superpower, but that is not not the best way. But, somehow, if we can reuse this, that

would be much better, right? So, instead of writing these tools and then making it part of one application, and especially if you just throw a bunch of different tools, that might not be a best way to organize your code base. Instead, if you put all these tools, related tools, let's say, order management tools, they are all related to some order related things. Similarly, you can bunch of

you can group all the catalog management tools in a separate group. So, if you group all these things, and then you can convert this as a MCP server. And if you build an MCP server, anybody can integrate with it, so that you don't they don't need to make all this code and then make it part of their application. No, they don't need to do. So, we have

a separate MCP server, and you can configure your application to talk to this MCP server, and then you make it available to anybody who wants to use it. So, this is where MCP comes So, what is MCP? MCP is the open standard protocol that enables any AI clients connect to any other resources. Resources in the sense, let's say we have all our databases, we have Slack, we

have Google Drive, we have Git. So, there are n number of tools available and you have chart clients like it can be a cloud desktop application. It can be ChatGPT or it can be an IDE also or it can be another Spring Boot application. The client can be anything and they want to talk to those components. So, instead of integrating directly from your application to that, let's

say if you try to do that, you will end up something like this. So, you might have written an application using Java and maybe there is some other component where they expose some Java API and you write integration okay talking to Java to Java. Or there could be other application which is written in Python and then you want to integrate with it and then again you write

some other way of integrating with it. So, you will end up having this m into n kind of integrations. So, this is where MCP comes into picture. Instead of asking every application to implement the integration to every component that you want to integrate with, you can make these tools as MCP servers and then anybody can simply configure the access to that MCP server and you can simply

talk to this. Because this is completely neutral part where you don't need to worry about whether this component is built using Java or Python, you don't care about it. You simply talk to it using natural language, not even as a API call or anything like that. So, this is the advantage with MCP. So, now let us take a look at how we can MCP server. Again, as

I said, we don't need to start from scratch, luckily, uh because we are building our Java Spring Boot applications. We don't need to throw away all the knowledge we have, all the systems we built. We don't need to do that. Instead, we can still use reuse the all the same existing systems and then make them as MCP servers. So, here in hotel service, it's a very simple

uh Spring Boot Uh it is simply exposing some APIs. You can search for hotel availability. Uh let's say in Bangalore, I want to I was looking for uh hotels from this date to that date. You can search for that. Or you can choose one hotel and then ask, "Okay, I want to book from this date to that date." So, something like that. We already have some application

which is implemented uh Java Spring Boot, Spring Data JPA, and you usual stuff. You know that, right? We can implement the REST APIs and all that. Uh we also implement uh REST APIs also. This is the application we already have. But now we want to make it as a server so that any client can talk to our service using natural language and be able to do all

this hotel booking and all those things. So, what we can do? We have this core domain logic here. I have added this hotel service tools, which is basically a uh Spring component, but I simply added this at MCP tool and give it a name and again a proper description what this tool does. And then here I'm simply delegating it hotel service, which is actually, uh, already implemented

talking to the database or whatever, right? So, here I'm just delegating. Actually, you can put this directly on hotel service itself, but we just want to separate things. And, uh, that's why I kept it in a separate class. So, like that, I can simply expose all these as MCP tools. Okay? So, here we have, uh, search hotels, we have, uh, book hotel, and we can get a,

uh, already booking information by the reference number. We can cancel booking. So, all these tools, I have exposed them as a MCP tools. So, what, uh, this will do, so, if you notice, there are different ways, uh, you can, uh, build a MCP server. There are different protocols. STDIO, standard IO, where the communication happens through standard input output channels, or you can make it as a HTTP,

uh, streamable, or stateless, those kind of servers where you run your MCP server on a centralized server and anybody can connect to it. Whereas, with STDIO, if you try to invoke any STDIO MCP tool, it is going to spin up the instance, whatever, MCP server you have, on your local machine. Let's say, from my laptop, if I try to invoke any whatever, if I use any Docker

container or a Java application, it is going to spin up that service on my laptop. Whereas, with HTTP, you can, uh, run the application somewhere and you can connect to it using by configuring this URL. Okay, this is host and this is the port. This is where MCP server is running. So, this is you just connect to it and then you can type. So, you don't have

to run the MCP server locally. So, in most of the enterprise systems, this is the model you will go for. STDIO is useful for kind of a local utilities if you want to play around something. But in the enterprise context, most likely you will go with some HTTP based exports. So, now let us check how we can build this application and then see how to check how

to invoke various tools we have and using natural language as well. Okay, so let me run So, I have started my MCP STDIO based MCP server and one thing if you are developing any MCP servers a easiest way to verify that using MCP inspector tool. So, this is a MCP inspector. If you are building any MCP server locally, this is the simplest way to check. So, here

you can check what is the protocol transport type you are using. In this case, we are using STDIO and how do you test? So, here we are going to make this as a JAR and then we can call it as Java minus JAR and specify the path to it. So, here did we build already? So, there is already a JAR file is created. You can simply call

mvn package and it will create a Spring Boot JAR file. And let's take the path. I'm giving the complete path to the JAR file. Now, let me connect to it. Okay. So, the connection is established. Now, there are other things also. You can create resources, prompts, and all that, but I hardly see anybody actually using them. But, most of the times we focus on the tools. And

here in this case, we can see there is four tools we have. Book hotel, cancel, all these things. Let's use search hotels. City Bangalore. And based on the test data I set up, I 0502. There are hotels available for this duration. So, I'm going When I invoke, it is able to talk to the MCP server and it get the results as Okay. Next, I can take Okay,

for hotel ID four, I want to book So, let's invoke. Uh check-in date, let's give it same. So, in the MCP tool, we also take more parameters here. Let's So, in the hotel booking, we have a command where it is asking for all these details. So, Spring AI takes care of scanning all these input parameters and then we are able to see uh here to provide all

these details. take this. So, we are able to successfully make a booking, and we also get the uh booking reference number. Now, let's say if I want to get the details of my booking, I can just give the ID, and I'm able to get this. So, in the same way I can also cancel. So, if you are building a MCP server, this is the easiest way you

can verify whether uh your server is working as expected or not. So, this is one approach. Also, here uh this is one client, right? But, usually you might have also seen uh let's say we have cloud code, and we want to invoke the same MCP server from cloud code as well. So, what we can do? There is a uh configuration file for cloud code. Based on your

operating system, you need to find where this configuration file is. And in my case, uh this is somewhere here. And here I mentioned, under MCP server, there is this hotel service MCP, and this is the command, basically the uh tool you want to use, in this case Java. And you can pass the other arguments. So, like this. So, once you have this, and if you make any

changes to the MCP servers, you need to restart your cloud uh code. So, here we post angle 2026 0504 18. >> With AI, you need to consider that things may go wrong, right? So, it is asking for the permission and behind the scenes it is going to execute Java minus jar and all this and then spin up the MCP server. And now we can see the results

here. So, here we have all the details and now we can ask uh go for this. Uh for Shiva and his email is uh Shiva at cloud apps.in. So, actually that uh hotel booking expects hotel ID, but let's see Cloud Code is smart enough to figure out okay, in this conversation we are talking about this we whose ID is uh it already should know and see if

it can Yeah. So, it is able to figure out the ID of the uh hotel that I mentioned as a name and then it made a booking also. uh you can test your MCP servers from Cloud Code as well. So, another >> Have fun figured out, right, Shiva? >> Yes. So, in Cloud Code configuration files under MCP server, I configured Okay. >> It didn't It didn't go

to the internet and what are the other files from the internet, okay? >> So, by default when we ask okay, book hotel, it doesn't know which hotel you are talking about, but in this context we already gave uh MCP server in the already shows we have all these tools available. So, it tries to But, the this is one other concern when you try to implement in the

production grade, there is no guarantee that it will first come to your MCP server. There is a chance that uh that is also one of the problem. If you combine too many MCP servers, it sees 120 tools, and there is a confusion like maybe if someone is asking to book a hotel, and there are multiple servers doing You never know what is going to happen. Is it

going to ask you for the confirmation, "Hey, which hotel you are trying to book?" Or is it going to assume that I know you are going to book some hotel and then make a booking? So, that is why in the production, you need to make as specific like in the handlers, "Okay, are you trying to book so-and-so hotel?" Or you need to ensure that it is not

going crazy, and you already seen some joke going on like on amazon.com, somebody went to comment section there he asked, "Okay, I want to build a Django application. Give me how to do it." And then it showed how to do in Amazon comments box. So, there are cases where people can misuse your AI integration. So, you need to lock it down to uh serve what your main

purpose is. You can't just plug the AI and then leave it to people and then hope they will use for good use. So, that is where every time when people say using AI is non-deterministic. So, you are hoping for 95% and then pray for the rest of the 5%. There is no when you ask something, nobody gives any guarantee. If If somebody is giving guarantee, they are

bluffing, okay? So, there is no guarantee. Here, simply we are not saying that, okay, if I give my name and with let's say when I book the hotel, if I simply Here are the details. [email protected]. And we are assuming that it knows that Shiva is name, [email protected] is email. And you are trying to assume that LM is going to do the right thing. There is no guarantee

because we are simply exposing these tools and then assume that it applies some brain and then try to figure out, okay, this looks like email, this looks like a name. So, this is where you can add more information like let's say So, in one of the tool, we can specify a tool parameter and then specify this so and so value and this looks like this. So, you

can give more tool parameter here. >> Can we do >> If you control the system prompt, system prompt is yeah, you can control through that like when you are building an MCP server to book hotels and all that, you can write a system prompt saying that your job is to only handle the queries regarding bookings. Don't answer how to implement Django applications or something like that. Your

job is to serve only this area. So, you can narrow it down, but still people try to find different ways to go around and then do crazy stuff. Yeah, description also helps. Here, I just simply put it as a cancel a hotel booking, but the more accurate detail you give, there is a better chance that it is going to use for the right purpose. No, there is

no guarantee. Let's say if If I I ask in a very generic fashion like here, if I Oh, yeah, we can ask. Okay, let's Uh let's go to Okay. So, how to make a fool of yourself like this, okay? In a live demo. Another? Find best beaches in Goa. uh usually it is not going to invoke hotel booking MCP. It goes to internet and get something. Yeah.

So, it it understood that this MCP server is not a good fit for this serving this request and went to internet and then try to find it. Is it? >> The MCP is main uh you know, it's part of the contact here will make it pure and physical contact to secure rights. So until unless you not feeling that force what I will use so and I won't

pitch from you and put that put that >> No, when you register these tools, you can use or or whatever to secure it and then only allow from the clients who already got the authentication token to make a successful call. But here, when you build an MCP server and a client configured to talk to your MCP server, they can see all the tools that you already exposed.

But it is not preventing But you what you can do on the client side itself, you can narrow it down like I think in James session you might have heard search tool search tool pretty set advisor where you can already narrow it down to okay, don't go all these places find and try to use these restricted number of tools. That way you can restrict from the client

itself. But all the tools that you expose, they are visible to the clients. >> But it started in case of client. Say for example, I don't have any I think it here I want to restrict only from my arc. >> Mhm. >> So in that case say I don't for example, I don't know from writing you know, I want to get something I don't know another technology

to where I can go and execute that >> will tell >> I can build an MCP and where I can access only from my application my arc can >> going to allow out >> Okay. So that restriction we should apply at the client level because what So here we are having an MCP server and we are trying to directly talk to this from core cloud code, right?

But if you have a Spring Boot application that is acting as a client, there you can let's say in your case it's say multi-tenant application. While registering the tools itself, you can say it is not a default tool that you use for all the interactions. Only in this particular case, add these additional tools and and then make the call. So, in that case, whoever the users that

belongs to organization one, they can only associate whatever the MCP servers you want to use and then make a call. So, let's see it's here. So, here we have tools, right? If you add them as a default tools, it is going to be available for everyone. But, you can narrow it down to only certain calls and then specify use these tools. So, in that way, you can

group those MCP servers and what not and then you can enable this. Let's say first you figure out who is the client that is accessing this endpoint. Figure out the tenant ID and then associate only the MCP servers that are accessible tenant ID. Based on that, you add the tools that are only allowed to call for that particular tenant. Like that, you can restrict it. As far

as I know, I mean these are the two sides, right? One, restrict on the other side where you can make a call, but unless you give me a valid access token, I will fail it. Or, you can restrict on the client side where you can restrict even showing these these are the tools available. So, these are the two things I am aware of, but there could be

other ways. That is also very similar to how we configure if you want to use let's say recently I saw this Excalidraw drawing tool we have, right? They created and made it as a open source where you can configure the MCP server. And then you can ask, "Okay, this is the concept. Draw me a diagram for Excalidraw." They are going to do it. You can simply configure

just like this and you will be able to interact with it. There. >> I think both of them need to handle in a communication within our MCP server any cloud. >> Uh-huh. >> Any cloud. >> That is very good. >> So, what these these tools expose it as a JSON RPC API. Once the request comes to here, within this it's up to you. You want to talk

to the database, you want to talk to another REST API, you want to talk to cloud service, you want to talk to Git. Whichever you want to do, you can do this because this is enables the RPC API. And then here it is a placeholder which our business logic you want to implement, it's up to you. okay, let's proceed and then later we can have conversation. So,

this is one way like building an MCP server as a JAR file and then you can invoke it. There is other ways as well like you can build a Docker image and let's say you build a Docker image using build packs and you have a Docker image with this name. Let's assume. Okay? So, in this case you can again try with standard IO protocol using Docker command

instead of Java. So, in this case we run it as a Java JAR file, but if you have a MCP server as a Docker container, you can invoke it as a Docker container here. So, let's go. I have already built this image. So, let's go. Now, standard IO use Docker. So, this is even better like you don't have to worry about uh So, in the previous case,

you need to have Java installed uh because it's going to run Java minus jar. But, if you uh make it a Docker container, you don't need to worry about uh whether uh the user has Java installed, Python installed, you don't have to worry about it. So, again here, uh we connected it through uh Now, you can see all the same tools are available. So, if you want

to create a Docker container version of your MCP server, it is as simple as that. So, this is all fine, but in the enterprise con- uh context, uh this is not how you usually build MCP servers. So, this is where we are going to use HTTP-based uh streaming uh protocol-based MCP server. So, let's change few things uh here. So, let me comment this out. Delete it. So,

to make it as a HTTP-streamable one, I configure this instead of standard IO protocol. And now, it is going to run on port 8081 as a web application. So, here let me start this. So, that I will do it by accident. this STDIO protocol, uh the server is going to run on the client machine. If I am the client, I am invoking the MCP server. That service

is going to run on uh the host application in this case. SSC So there are previously SSC was the default one. It's like a server sent events. That way they communicate back and forth. But recently they deprecated it and then now it is preferred to use HTTP streamable or stateless. So it is it is stuck. So now MCP server is started. Now let us see how we

can connect to it. So in this case we can instead of STDIO we're going to use this and by default {slash} MCP is the endpoint that exposes this and let's disconnect. Okay. Now we connected through MCP server HTTP streamable. Now we can see the same tools available we can perform. So the advantage is this you have the server and you decide where to deploy it and what

kind of a restrictions you want to apply. So you have a typical application and you want to run it in a whichever way you perform. And also you can apply access restrictions at the network level. So you have full control over how you want to expose this MCP server to the outside world. okay, we have the client sorry server MCP server running. But now let us see

how we can connect to it as a uh, a client. So, here we have configured So, here is one interesting thing. To build MCP server, you don't need any AI. In fact, if you, uh, see HTTP service here, we have not configured any, uh, OpenAI keys or cloud keys, nothing. Because it is not using AI. It is simply using MCP server and exposing all the tools over

JSON, uh, RPC. So, you don't need to have any, uh, OpenAI account or anything like that. But, from the client side, if you want to consume those services, there you need, uh, you need to have this account. So, in this case, I'm using OpenAI here. And I configure, okay, so, I have some streamable HTTP servers, and this is the connection. This is where the MCP server is

running. Or, if you are, uh, using STDIO, so, this is how you can configure. Like, you can specify all these properties. You can configure. And another option is, you can, uh, in fact, use this cloud code format. Uh, this is exactly same as how you configure, uh, in the cloud. So, you can create a configuration like this, and you can simply specify use the configuration class. So,

so, either way, you can, uh, you can use it. So, now we have the, uh, client configured. Let's run this client application. And if you take a look at uh, chart controller, again, we simply have chart client, and we have So, this is one thing we need to remember when you connect any MCP servers, you need to inject this tools here as default tool callbacks are per

call basis you need to hook up. So, all those MCP server has these tools, right? They're all coming back to here, and then you can hook it up. So, we have the client application also Nice. It should be better. It shouldn't look like this. It's the first live debugging. Why CDN? Why CDN is not accessible? Okay, I else let's put aside styling for a minute. You need

to pretend this UI looks good. Okay, something is >> You also need the non-data break. >> Sir? Yeah? >> We are in a smart. >> Okay, how about invoking this as a HTTP rest client? >> It should have been nice seeing that in the chat UI, but let's see. We are developers, right? We should be liking this weird thing. Uh of course, here are available hotels in

Bangalore so and so. Uh yeah, you get the response, right? Okay. So, the client integration is working. UI thing we can ignore back as a back-end developers at that thing. We're good to go with uh REST API that is how simple it is to introduce AI capabilities. You don't need to start from scratch or write in a different language or anything like that. You can take your

existing applications, and then you can uh add the MCP server capabilities, and then expose it, and anybody can connect and uh use this uh So, there is uh advisor concept. Uh just see here. So, here for a chat client, you can add the advisor. Advertisers are nothing but a filter. Uh we all know servlet filters are AOP advisors, right? So, you can simply write a advisor that

just uh uh take the start time and then let it flow and then once the responses came back, and then you can write, "Okay, how much time it is going to take for interacting with this one?" So, it is simply implementing your custom advisor here. So, in fact, Spring AI already provides advisors. And if you are using actuator, you can go and see how many tokens are

consumed and all these details is already captured. So, if you want to measure how much time it took for LLM calls, you can add one more advisor and then see it. that is that and I hope you at least learned how to enable this Take an existing Spring Boot application and then expose them as MCP servers and then how to connect to it and all that. But

beyond that, what is next? Like there is Spring AI community project and naturally AI space is evolving so rapidly, it is hard and almost impossible to build something concrete. So, in this Spring AI community, they are adding more and more utilities and then maturing it and then once they kind of a good, then they are merging back into the score Spring AI things. So, if you want

to explore how to secure your MCP servers using what, there is a project for it. And also, if you want to use some tools like file utilities or shell script utilities and all that, there is a tool library for it. Utils or something like that. So, there is a lot of things going on in the AI community GitHub repository. If you want to check out, check out

that. And then there is Embodiment framework which built on top of Spring AI, which is more like a building a agents where you ask it to achieve some goal and it figures out how to orchestrate various things. So, you write all these methods and then you specify what each method uh performs and then you have a goal and you don't need to orchestrate by yourself like if

you want to place an order uh first search and then get the results and find the cheapest one and then uh book it, you don't need to write all that code. You can simply say find hotels in so and so and then find the cheapest hotel and then book uh this hotel and uh return the response. So, you can write different methods and then you simply ask

it okay all this procedure. Amble figures out okay for this first I need to call that method and then once I get the response, I can use the information for the next method call. So, it figures out. You don't need to write the orchestration logic by yourself. So, that is even more powerful and uh if you are trying to build any AI agents, I definitely ask you

to check out that uh Amble framework. >> Okay, then I'll one feature >> Yeah. Yes, in fact it has even much better support. So, here in a application there are uh scenarios where you want to use different LLM models for different purposes. For example, if you want to do some research, you may want to use some powerful model. And you can configure cheap models uh costly models

and then for some purposes you can configure and then for cheap model let's say you want to use some open AI oldest model and for uh heavy research you want to use some uh more advanced model. And then you can say for this use case use a uh high reasoning model. You don't need to specify use open AI 5.5 or something like you just say use a

uh high reasoning model. It figures out okay, what are the available high reasoning models here. So that kind of a indirection we have so that you have complete decoupling like you don't tie your application logic to specific models or anything like that. >> decide what which model you need to pick then during >> configuration. You just configure in the application properties saying that these are the three

cheap models we have. These are the three research models we have. And then you just say use research model. And also in embeddable framework you can actually for each LLM call you will incur some cost, right? So it also gives for this total for this total task completion this took this much of amount. Okay, it will give that and also you can restrict that for this task

don't spend more than X dollars. So you can configure that as >> It's totally dynamic, but though that's dynamic. >> static >> I mean like a static background >> Huh? >> Can be that sort of session or kind of based on the user I have a separate topic. Right, yeah. So all the AI capabilities are wrapped in a class called AI in embeddable. And you can leverage

all the platform features like if you want to find out what is the information we already have in the blackboard. So there is a concept called blackboard where when you ask it to perform some action and it figures out okay, first I will call this and I get this information and I will use this information to perform next step and the next step. So in each of

these steps it needs to extract some information, put it put this information in a blackboard. Now at any point of time you can ask okay, what is what is the information we have on the blackboard and if you want to proceed further, you can do it you have full control over it. >> They want to register how we can add the fall backs. >> Fall back strategy.

So in they are building this agentic patterns and all of them follow this advisor approach. So for example, if you ask LLM to return some response in JSON format where you should have ID, name, email. This is the format you are expecting. Some models don't respect that properly and then return somewhere where when you try to unmarshal into the object, it fails. So in that case, if

you retry it again, it will return properly and then you will succeed. So instead of writing that logic by ourselves, we can use the advisor that says it's already there provided in one of the agent utilities where it will try to marshal it. If it fails, it will ask okay, we are getting this error. Correct it and properly return in this format. It will perform again. So

we don't need to write that custom logic by ourselves. >> No. So. >> Yes. So check out this Spring AI community. There is agent util library which has all these utilities.