Building production-ready AI Agents with Spring AI and Amazon Bedrock AgentCore, Vadym Kazulkin

In this talk, Vadim Kazokin discusses building AI agents using Spring AI and Amazon Bedrock. He highlights the integration of operational aspects for scalable agent performance focusing on the latest features of Spring AI, including the MCP protocol, which allows language models to access specific information. Kazokin explains how to develop applications that interact with AI through natural language, showcasing the conference search application as an example. He elaborates on the Spring AI agent core, which supports various cloud environments and emphasizes the importance of security and observability in production-ready AI solutions. The presentation also touches upon the use of memory management and the Embel framework for optimizing AI interactions by breaking down complex tasks. The speaker encourages developers to pay attention to operational considerations to ensure successful implementation of AI agents.

Okay guys, uh without further delay, uh let's start with the next talk. Uh Vadim Kazokin will talk with us about Spring AI and yeah, I have used Spring AI not to build agents but for MCP and it was pretty great. So I guess that this is even even better. So thank you for coming and enjoy the talk. >> Yeah. Uh thank you very much for having me.

Yeah. So the topic of the talk is building AI agent with Spring AI and as an Amazon bedrock agent core. But as I applied for the talk several months ago, there have been some stuff that has been released since then like in Babel but also spring um AI agent course. So I will try to talk about this topic as as well because they provide values. Yeah, my

name is Vadim. I'm Ukrainian native but live in Germany since many years and Java and AWS cloud are my two passions that's why we will be talking about um kind of both so um yeah we will be talking about springi people who visited the talk of Arno in hall a there will be some overlap of course but there will be different things but we'll be also heavily

talking about operational stuff of the agents because I see a lot of demos agents it's running locally and it belongs the the operational stuff scaling of the agents belongs there. Of course, it requires maybe some knowledge and understanding of AWS cloud services. Even if I will explain especially this agent core service, but basically if you use other cloud providers, you will find similar services. Yeah, this is

nothing here um that that you won't find. Maybe you will you can find open source stuff on Kubernetes to run this the things. But it's important to to pay attention about the operational aspects. So we'll be talking about SpringKI. I will be using Amazon Bedrock as well. But Spring AI you can use everything and we'll be deep diving into certain services. certain things I will be leaving

to you because I would like to talk especially about Spring AI agent core that has been released recently and if we have a bit of time I will briefly talk about embable framework which builds upon Spring AI and provides additional value okay I think everybody knows all these AI terms AI terms so we don't need to repeat I heard today all of them but we will be

heavily using MCP um we won't be using any agent to agent in my example. So that's important that that we you understand MCP. This is basically the way to provide LLM's access to some information that's specific to us so that the information doesn't have any knowledge like and we can extract expose those things through MCP um and and so on. So there will be artificial sample application

which will be extending. So once again it's artificial. It's nothing. It's nothing uh rocket science. Um so we'll be conference search application and we'll be like would like to provide certain functionality to outside of the world like search for all conferences search for conference by topic. Additionally also by date range for example the conference is happening in June and there will be for certain reasons one so

the first three are for visitors like who are searching for the conference and the four the fourth method is yeah is there any call for papers often this is for somebody who would like to submit something and the question is I I would like just to to talk to this type of application through the natural language so what should we do and basically yeah the idea will

be that we those methods, those functionality to be exposed as MCP as MCP tools and yeah, so we need the MCP server and then we need the MCP client and both are also provided by SpringI but like yeah everybody already heard and and knows that Spring AI is the framework exactly for this. Yeah. to have access to the different launch language model and if you will be

only talking talking to the LLM through Amazon bedrock service springi is completely undependent and you can use entropic openi directly you can use even local models like colama so that's not the point but I'm heavily on AWS it's for me easier to experiment that's why I use Amazon bedrock so springi it has much more that we can talk about this are chat models we'll be using that

there are lots of chat models we'll be using exactly this green one Amazon bedrock there are embedding models if you would like to uh to use semantic search and all that stuff there there are things there are image models so this multimodality is there images audio and moderational models as far as I know there is no video currently might be in development yes uh spring provides MCP

support mcp is a protocol like http yeah there so that's why there are so many providers and spring provides the server side of it and provides this client side of it which we'll be using and there are also racks and vector databases we will not be touching up all in our application. So yeah I will be using Amazon bedrock because it's easy for me. Amazon bedrock is

basically the container where I provide the model ID which Amazon bedrock supports and they support different ones. They have their own Novas from AWS, but they also support um entropic and the models from meta rich number of models. And yeah, I can I can go to the bedrock console of AWS and simply search for supported things. Yeah. And as you see here, this is the model ID

um that that is required and this will be basically the ID that I will be providing uh through Spring AI Amazon Bedrock. So once again now coming back to this application how we will de develop this yeah as we would like to talk this through to this type of we need to expose that these operations as MCP tools there will be different story if you already have

the API yeah like rest API which does the same we can convert that to MCP I will be talking about that later but let's imagine we would like to implement it from scratch and don't expose this as a as a rest API but only to the LLM as a tool tools and for this yeah once again springi has MCP server support. So we I have in this

application you by the way see here the links I have the articles which I published not about all topics but there is also link to GitHub repository and I currently was lazy to implement the proper back end in terms of the persistence. So I persisted this conferences the artificial data in JSON. Yeah, if I will have time I will do it in the database but for the

sake of this uh demos it's not relevant. So what's happening here? What we need here is um yeah we add dependency to springi and we we need the spring spring boot starter web and what we need here is the um spring AI starter MCP MCP server web MVC. Yeah, this is the MCP server implementation provided by Spring Eyee. Uh MC WebMCP says we have the we will

talk to this server synchronously but you can use the web flux dependency instead on this and then the type as synchronous. If you would like to stream from the server then both are supported. So now let's expose those tools. Um and how we will we will do this basically it's only one example for one tool uh which provides the search conference search by topic and date. Everything

else is the same. Basically we need to declare our class which exposed that as a component and then each such method that we expose as a tool we need to annotate with MCP tool annotation. Provide the name which is more or less like ID and provide the description. description is important because LM will use that description to find out if the LLM cannot answer the question is

there any tool capable of answering that and it will search for the description and the prompt and and and do the comparisons is there any match so that's that's why the description is important and of course we have parameters in our tool in our case it's conference topic start day and end day of the conference to answer the question search for the conference by topic and day

like by topic Java and day from the first uh to the third 3 1st of July. So all others are the same MCP2 MC tool MCP2 uh parameters. Um yeah this is all others will be the same. I have four of them as I showed you all conferences also by CFP and then this is in spring AI 1.1 which is now the current version. we need in

this uh to write this bin to expose these tools. So basically we say here yeah this is conference search tools will be more or less exposed as a tool callback so called this is the way how you can we can access the tools is kind of by the callbacks because this is the the server side with springy 2.0 zero which should even be released this week. The

declaration of the bean is obsolete. I always ask myself why should I declare it? Spring can figure out whether I have components having MCP tool annotations to do that for me. This is exactly what will happen. So we can skip declaration of that bin. So basically now we have the uh MCP server. You can start it as a spring boot application. Spring run spring boot run or

java minus jar and it works but it's the local one. Yeah, the thing is you probably don't want to operate something locally. Yeah, we will be talking about the client later and yeah, there are challenges around this agent because uh yeah, proof of concept is is one thing but addressing the scalability, security, observability and operational concerns is something that's important especially if enterprises using LLMs and tools

and so on and that's why I will introduce once again this is Amazon cloud service with the name bedrock agent core or short agent core which has certain set of features that we will be going through and this is exactly the service once again as Azure has similar yeah and probably Google and all that stuff but I'm not familiar with those cloud provider and services yeah and

the goal of agent core is exactly to enable uh us to deploy and operate the agents effectively yeah and uh it provides certain features or subservices we will be looking only into some of them with the runtime to run the agents For example, MCP server will be gateway. We will be looking to this and there is observability memory stuff and all these important pillars that that help

us to operate this agents. So let's first start with the agent core runtime. So once again you can run this agent which is this MCP server that we developed this normal spring boot application. You can run wherever you want. These are the possibilities on AWS and you can use servers you can use all types of decorization. You can if it is short living leaving living agent you

can use e function as a service as AWS lambda nothing wrong with that but um what this agent core provides once again it's framework agnostic so you can run all models it's also um it's it's model you have model flexibility all models are supported not only those from AWS like bedrock and it's also framework agnostic you can run all more or less all agents there. Yeah. So

it's not only spring you can use uh Python based things. So this this flexibility there and this agent core supports different protocols. What is important for us it supports MCP protocol with that you can run MCP servers on agent core runtime and I will show you how it also supports HTTP protocol. We will be talking later agent to agent we will not be touching here. So what

what are these benefits? The runtime supports real-time interaction also for the workloads which run up to eight hours. For example, if you will run it on AWS Lambda, Lambda only has the timeout of 15 minutes. So if you have long long running conversation, it's maybe not exactly what you would like. It also this agent core provides session isolation. So basically each user session runs on dedicated microVM

with isolated CPU and memory and file system and then the session is completed everything will be sanitized. This sounds obvious for security reason but if you are running locally I don't know if that's obvious. So people knowing AWS maybe know what's what is powered by this. This is the firecracker VM. Yeah, I think it's like Kubernetes orchestrator but made for a on AWS cloud and if you

use AWS Lambda fire lambda runs on firecracker and it's kind of a good choice for agent core runtime because agents may vary from the duration they can be short they can be long lived with we put different prompts there and the answer might take depending on what agent needs to do it also the content context can vary from zero to many gigabytes and the this the firecracker

has the ability to grow shrink CPU and memory just in time depending on the context on on the on the output and that's kind of important basically you don't even provide how much memory or CPU you would like for this runtime AWS will handle it for you yeah because if you will use the server you will probably need to size it yeah it might not be enough

depending on the prompt that you send so this is kind of very good thing and once again agent core runtime support there are stateful and stateless MCP protocols because it may be beyond the topic. The default mode is stateless, but now also stateful protocols come into action. And what's important is the the state-of-the-art protocol for MCP is now streamable HTTP. Yeah, if you use MCP server, then

the default protocol it will be streamable HTTP. The one was before server side event is now deprecated. So the specification of MFCP now requires that all implementations support streamable HTTP. And this is what Spring AI does and what agent core does. So basically if you would like to deploy our application to search for the conferences on agent core runtime we need to configure certain things like you

see here the protocol is stateless. That's fine for our case and we can do assume and sync both is fine. I have infrastructure as a code. This is how you deploy on on on cloud providers. I don't want to show you that. I would like to show you how it looks like if you have deployed that already on agent core runtime. The most important are two things.

If it's deployed, you have the resource IRN. This is the full resource ID and runtime ID. This you will need later. And basically behind each agent core, there is so-called endpoint. In the default, it's always kind of default endpoint, but it points to the version of the deployed application. Yeah, we can make changes and redeploy the stuff. And you see here you you see below the the

full history of the versions that we used and what is needed to deploy such a version. we need to dockerize our application and AWS has its own service ECRI for deploying Docker containers and behind this version this there is an im um the ID the resource URI to the docker image of that and basically if you deploy once again there will be the next version pointing to

the newer image in the docker registry powered by AWS this is basically what you need you see here the protocol is MCP there is also HTTP there is also the way we will be using it I will jump over it how to secure communic communication to this um MCP server. Basically, it happens with JSON web talking. Uh this is beyond the topic how to set up it.

I will only briefly mention it because uh you cannot simply assume that everybody can talk to this. This is kind of the identity part. So what we now need to talk to this server is of course client. Yeah. Now we deploy this. We need some client to talk to this. And this is where Springy starter MCP client that was a server. Now client comes into play and

once again web flux is a sync variant and here we can declare the programmatically you probably know how um springi works as we gave the dependency here to the springi starter model bedrock converse that means that we will be using automatically amazon bedrock and we can provide certain configuration like timeout like maximum number of tokens in response the chat model this is Amazon on one we can

use entropic so we can provide certain things and this is what once again this runtime ID of the deployed agent core runtime this is just to for the security let's not touch about it and yes you can use synchronous one and then you need this MVC client instead so it depends on you I personally would like to stream and so on so you probably saw this how

you will talk to this model uh the main interface is here chat client from spring AI And we we need to build the options. So the default one are from this property file I showed you. But you can override if you would like. You can override the model. You can override number of tokens. You can override temperature whatever you want. And um springi is clever enough to

understand that we are talking about bedrock options because we provided this bedrock starter. Yeah. If we can provide open AAI starter, we can provide entropic starter and then then behind the scenes the other options will be built using other other APIs. Yeah. And we can provide here default system prompt. I didn't provide it here but basically you can say you are the intelligent agent capable of answering

the question about the conference search. And if you don't know the answer simply like this. So the chat client is kind of the entry point. Now we need to talk to this MCP server which is running on agent core runtime. Once again we provided this ID basically um which is more or less than enough just to talk to this. Now let's go through what's happening here because

it's quite a logic like just to talk to the to the model. So the first one let's forget it. This is we once again the security point here. Um and it's uh you see why it's required uh because we would like to obtain this JSON web token and put it into the HTTP header as a beer token. This is how we secure with JSON web token. Yeah.

So this is yeah we we get this token. This is the Amazon service behind it but you can use others. We we add the beer here and we add it to the header. Yeah. And what's happening now? We would like to create the MCP client here. Yeah, this is the asynchronous one and I get this from this MCP client transport with talking and here what's happening here

is this web client streamable HTTP transport will be instantiated once again it's streamable implementation of MCP protocol. Yeah. And I give here basically the URL of of the server which speaks MCP and certain headers. That's basically it. And with that you see here the MCP client can be instantiated. The first operation to talk to MCP server is to client initialize and then we can optionally ask for

the tools available and there will be these four tools like all conferences conferences by topic and so on. So we can print them that they are available. Once again this was springi 1.1. In springi 1 0.2 there will be small changes. They renamed the classes and here we used the web client builder. There was the web flux uh client from springi itself. Now they switch there is

one http client or client builder from java itself. So they say okay we can use that and they basically rename the class with uh 2.0. So you see http client streamable http protocol and that's basically it. So now when we have this MCP client we can basically convert it into so-called uh MCP tool callable provider. So with that the MCP client call back MCP server call back

basically will be done. This is more or less the wrapper around the client and then we can basically talk using this chat client. this this entry point to the um basically here we're providing the user prompt like search for all Java conferences and now we can provide those tools with this tool call back we provide those tools that coming from the MCP server they are more or

less this call back get tool callbacks with that we will get these four servers from the uh four tools from the MCP server we can define also local tools for example to answer the question provide me the list of the conferences uh happening with call for papers happening today the current date should be known llm doesn't know the current date yeah that's with that I can define

the local tool I don't need to deploy it in MCP server because there is no application behind it you see I simply defined the class defined as a component have the tool annotation not MCP tool because it's a local tool get the current date and basically it's only one method and I can provide it with the tools additionally. Yeah. So there will be tools coming from ECP

server. There will be tools also adding to this locally and with that I can already talk through chat chat client to the MCP server. You can also filter if you say oh I have a lot of um MCP uh tools here and I definitely know I don't need all of them. You can provide the tool filter and filter all the tools by name if you need. Yeah.

So that there is less chance for LLM to hallucinate. If it sees too many tools, it might. So if you see that there is endpoint providing you the tools for orders and for users and you say I don't need any user tools here, I need only orders tools. You can filter all the tools starting with user by name and then you increase the chances. So once again

we I saw how I showed you how to programmatically create this MCP client. If you only if you don't secure it and you have simply the URL of the MCP server, you can more or less configure it like you're doing the spring uh with the spring. So basically with one you prof you do here the URL stuff and then this assume MCP tool to tool call back

will be um injected for you automatically. Yeah we cannot inject because we need to provide this authentication ber token that's why we need to obtain the token and create everything programmatically. If it's a public MCP server, you can simply use Spring declaratively and then inject this MCP tool holdback provider and skip everything before that and simply talk um to the model. Yeah, there will be changes how

to use that tools start with Spring AI 1 0 2.0 if nothing will be changed. So this is how it is in spring 11 and here they use tool spec basically the same but uh they change it with instances you provide the local tools and with callbacks you provide the tools coming from the MCP server I think it's kind of a good idea because nobody understood how

to use those methods now this is the consumer of tool spec and it's I think it's a good thing so now there is another possibility if you don't personally know or There are so many tools and you spend so many tokens. There is the community project springi tool search tool and with that you can provide lm or with one tool which is the search tool and provide

the semantic search string and then uh the search for the uh the the the tool that is exactly fit will be done through the semantic search. So basically you don't require to to to fetch all those tools from the MCP server but you provide the search tools and the search tool will search for you um with that. So now let's test we implemented this MCP client. Let's

run it first time locally using curl. So I I don't have UI here. This is basically web interface and I can ask something. Please provide me with a list of conferences including the ids with Java topics happening in 2027. Once again I have artificial conferences in JSON and this is what basically will be returned which is uh the the right response and the um LLM was figured

out that it doesn't have any information how to do to deal with that but it found this tool provide the conferences by topic and by dates and this is exactly what has been called you you will see then the string here this is another example basically the same ah Here I was asking also for conferences with a call for papers opening today and LLM says okay let

me first get the today's date and search for the conferences okay I have a date today this was the day when I experimented April 14 so it used this local date tool to to first grab the date because that's the information that is required and was date topic and then the MCP tool was invoked on the server side so you will see all this information very powerful

with that MCP and tooling. How much time? Okay. [snorts] Okay. Now um let's imagine we would like to extend our application not only by providing all these things but also let's assume we have another part for applying for creating and fetching the talks and applying those talks to the conferences that we can select with the previous let's imagine we already have an API for this yeah artificial

example let's imagine we already have that so we have the business logic basically doing the stuff and now I want to add those things so I don't only search for the conferences but I can ask LLM create the talks and apply for that conferences yeah with human in the loop or not it's not possible how to do that yeah we have MCP server now we have existing

API yeah how I can mix that and the best way to do is is to use another agent core service with the name gateway and it provides more or less the way how we can expose MCP functionality Yeah, this is more or less managed MCP server with the different targets. Yeah, and we can convert existing APIs not only hosted on Amazon API gateway which I did but

you can also use the API anywhere else. The precondition is that you can get open API specification of it. That's the the important thing. You can use lambda function AWS specific and you can also have access to MCP servers running on agent core runtime that we did. So basically this is how it works. We can define gateway and defined many targets and what we can do here

as I said we have already existing API gateway for creating the tools and so on and it supports also as a target time FCP server. So we can define basically there are also integrations to Jira if you would like. So basically what I do here I create the agent core gateway by providing two targets API gateway target where I have existing API and MCP target that I

developed. So with that we have here two targets the first one has these three methods create talk get talk and apply talk and this MCP target has four for the search for the conferences. So we'll be if we will be talking to the gateway will we have seven tools returned and the gateway is also kind of the managed MCP and it's really powerful thing. Yeah. Because with

that you have one entry point to talk to different managed things. Yeah. And you don't need to rewrite your business logic if you have one that you exposed as an API. So why to copy the logic and annotate it with the MCP tool? you can simply chain it as where it's MCP. Yeah. Through this gateway without any changes. Um yeah, this is kind of uh one of

the important things. And basically if you have the API, you can also select um that you expose not everything from the API. For example, you don't want to expose uh read methods. So puts and so on. You can exclude that in the agent core. You can say you will see everything. You can say only read operations. I would like to expose this MCP server uh SMCP tools.

Everything else not. Okay. Yeah, this is exactly what happened here. I I select I see these three operations apply for the talk, create talk and get talk by substring like by title substring like related to Java on the gateway and I can select and say I don't want to apply for the talks for example. Yeah. So this is very flexible and possible. I will skip through the

identity. It's important how you secure how agent core talks to your API. You need to secure it or to the MCP server. The two keywords are O02 and API is sometimes secured by putting some secret into the header or into the query string and you can do both. Yeah, you can secure that and that's the I simply would like to jump over but yeah with O key

you can use Amazon services you can use octa Google everything that is possible to issue their token for you so basically when you create this gateway with these two targets that I showed you the thing you will get this gateway URL it will end with MCP and it speaks streamable MCP protocol as the agent core runtime so for the client we don't need to change anything This

is basically the the the URL of the service. We need to put this URL. MCP is a standard. So nothing has changed. We talk simply to through the different URL. But now we will have seven tools. Yeah. And with now we can put such a prompts like provide me with the list of conferences including the ids with Java happening 2027 with the call for papers open today

but also provide me with the list of the talks with this topic. This is Java topic and finally for each conference and talk retrieve apply individually for this conference. So now we have basically with such a huge prompt uh LLM will be figuring out that there are two or three tools that it needs to use you see here that needs to to fetch the conferences. It needs

to fetch the talks with the topic Java and apply with the the fetch talks for each conference. I don't have here human in the loop. Normally you would like to review and say it's too far the company will not cover but you got the idea. Yeah. Basically LLM is capable. There are seven tools and it will pick all the right tools with one prompt basically. Yeah. This

is how it works. Okay. Here's another prompt to create the talk. Okay. Um the thing is here you saw this is the MCP client and running locally basically and it's a huge client. So as you see a lot of logic happening here. So what happens if I would like to change the dependencies? I would like to just spring fix something and [clears throat] would like to update

[cough] [laughter] sorry. So if you have only one user of the client then exchanging it it's it's [snorts] easy. But if you have multiple users of the client then how you would like to patch it to that that there is a possibility also to deploy the client on agent core runtime. So you put this logic also to the cloud and the AWS supports that um also with

the HTTP interface. We put MCP server with MCP interface there. We can put this client with HTTP interface here. There is some some requirements there how to do this. We need to expose invocation endpoint as a post and ping for the health service. So basically we need to rewrite a bit our client basically that we have now post invocation thing and that's it and pink method that

that returns something with that we can more or less and pink we can use spring actuator by the way and with that we can now use HTTP protocol of agent core runtime to deploy that client there. So the whole client logic now moved to the cl to the cloud and can be scaled but of course we need some kind of client to talk to this. Yeah we

need something to put the prompt but the this logic is much simpler. So now how it looks like if you deploy it here you have that IRN this is resource identifier um you put it here this is the prompt and then you use AWS API basically you create bedrock agent core client and then we invoke do invoke uh runtime request but providing this payload which is a

prompt and putting this runtime IRN and that's it and then we can stream the results so in case the client needs to be updated. We don't and we redeploy it on agent core with HTTP protocol and we don't need to change here anything because behind the this default endpoint there will be a new version of what we deployed and the client renames the same here. Yeah. And

with that there is no need to patch it frequently because there is no information. We don't get the token. We don't get the tools. we only talk to this agent core runtime client which in turns talks to the agent core gateway and this is how it like gives you additional capabilities. Okay, 16 minutes. Uh, so let's talk about things in between. So, uh, Springi there are frameworks

or I would say that support this integration of Springki and agent core provided by AWS team and basically AWS team released Springki agent core SDK. It has basically it's a library which eases us to talk to agent core and I will only show you two features of this. The first is the runtime starter and the second is the memory. And the basically what we are doing here

this runtime starter has more or less two functionality. If we uh the two functions if we uh include it the first of all we can annotate each method that we would like more or less to expose as its invocation endpoint and it pre-builds the pink endpoint with the sprint actuator. So we don't need to implement this pink endpoint. We simply add uh actuator and it does and

it also gives us the possibility to do the rate limiting using bucket forj. If you would like we can define that that the things will be throttled in case there will be too many requests against agent core. So basically that's not too much happening here. And then another feature which which I really like the the the agents are stateless by default. So there is no memory. If

I will apply for the talk and then use uh the next prompt give me the talks information about the talks and conferences where I applied the um the lamb will not be answering that. Yeah, there's by default stateless springi implements things like uh there are memory uh implementations in memory you can use or you can use GDBC stuff like posgress database but likely you will probably need

to manage something this database where you store the memory and so on. Yeah, this is uh something that might happen and there is this springi memory thing where you build this chat memory and so on. Um, agent core provides also memory capabilities of two types. The first one is the short-term memory which basically dumps the whole conversation like an MD file and stores it somewhere. You don't

need to know this manage component. You create short-term memory that's it. Yeah, this is not very effective because everything will be dumped and this will require then lots of tokens and there is a long-term memory implementation which uses semantic facts or sum summaries. So with our conversation, the summaries will be created and stored in the long-term memory for us. Yeah, they don't exactly explain what is behind

this long-term memory. So what technologies, but it just maybe doesn't matter. And this is what this springi agent core bridge gives us. We can add this memory and we can create memory. If we go to this agent core, there is a memory. You can create short and long-term memories. You can say when it expires and basically if you create it you have this memory ID and this

is what is required only by configuration by providing the memory ID. Springi agent core memory is capable of dealing with memory by saving the events but also retrieving the the the events in the agent core memory. This is for the long term. So also there is an interface chat memory for short-term memory and for long-term memory. This is the um springi implementation. But basically if we now

executed the prompt to apply for the conference with the talk. Now we can ask you recently apply for some conferences. Can you provide me with the details? And now with the memory applied this all information about what we did can be pulled. Yeah. And now you can see kind of the whole history. So I personally don't know any agent which doesn't require memory. It's very rough. But

with that with agent core memory and with spring keyi bridge to that you can do this. [snorts] This is the last feature of agent core. I have 11 minutes. That's fine. Observability. Yeah, we talked about managed MCP server gateway which is agent core gateway runtime for running the MCP server or HTTP based client. We talked about different memories. We also need observability. Luckily what's happened AWS decided

to to use open telemetry from the beginning. Yes. Which doesn't bound you exactly to AWS tools but you can use that and AWS released their own AWS DRO for open telemetry. This is basically um um open telemetry prepared for for AWS just to to easily put the things there and you have agent core observability um there in the cloudatch. This is the service of AWS for this

observability where you can have the metrics by default. You see here input output tokenage token usage also you can select the model if you use different you can see that uh uh for for all the models time to first token end to end invocation invocation count error rate everything is there but also by using AD dot you will get the other observability things like uh traces basically

spans and traces and with that if you simply do certain things you will see here like all sessions, all traces, all spends and you will have all the things. What you will need to do is basically to adjust the docker file to download this AWS open telemetry and put it as a as a agent Java agent into this tool options and provide certain values. This these all

parameters are open telemetry parameters. So you say here what is the service name? What is the service ID which is agent core and basically you put the endpoint for traces and logs and here this is the cloud watch AWS endpoints but if you would like to to to export it for example for to data do you will need to provide the data do urls for traces and

logs and you will have that there yeah what AWS gives you I already showed you can see all the sessions all the communication yeah if I start to communicate it's a session for all sessions I can see all traces spent ends and for all session I can see the the traces themselves and I will jump over it quickly but you have different views there is a tree

view where they can see the whole tree invocation what happens how we started that what tools were pulled security memory you will see everything there is a timeline basically which is similar there is a trajectory that you can see that in other form how that that invocation went basically observability how you how we would like Okay, 9 minutes [sighs] maybe several words that's kind of enough and

it's really powerful thing without observability it's probably even more complicated but now imagine this is this was the prompt that we used previously please create the talk for example then please provide me the list of the conferences and finally apply with each talk for the conference you see here there are at least three different tasks to do yeah create the talk provide the list and then to

apply there is a huge chance if we provide it as a one prompt and give all seven tools that LLM will hallucinate here because because of the nature it's sometimes too much. So what embel framework gives you is built upon springki is more or less isolate different goals from the huge uh prompt provide the boundary and the boundary is in terms of Java is a record more

or less. Yeah, the input is that record. The output is that record. Yeah, don't put other parameters here if you don't find that. And path toward towards the goal and the goal was here to apply the talk for the And by the way, beel is created by Rod Johnson who created spring uh 20 years ago. Yeah. Um so these are actions, goals, conditions and so on. And

let's see here for example I provide the prompt this huge prompt and I say I would like to split that into different actions and I split the action here the first action extract the talk creation request from the user input and how I can do this I can use LLM not very powerful because there is no need that and say create a talk creation request from the

user input extracting the criteria like title and description from this huge and create this uh this talk request and this is talk creation request is once again Java record that only has title and description. The second action is create the talk from this as an input. This is exactly that object with title and description and I only give here tools tool that is required for this because

I now split it into the small problem that to the small action which only creates the talk and I can give say okay you have seven tools but filter here the only that tool and use only this tool and and then the output is the domain object for talks with all talks. So basically I shrink that yeah using I decompose the big task into several uh smaller

tasks and here is the same extract conferences. First of all create the search request which basically contains topic date and so on and then search for the conference uses the this search request and only provide those tools and not everything else. So we don't need create talk for the for the agent which is searching for the for the and the the last path is which we annotate

with achieve goals not only with action achieve goals is to apply uh with the talks that we selected and the input is the result of the invocation of the first two actions. So the the conferences and talks and we with that basically we split it into this and this is the final destination to achieve the goal to apply to the conference and then if we run that

embel will be able to find the path because it knows the input of the agent if it's a user input the prompt or it's the result of this and it can build automatically the path. Extract talk request, create the talks, extract conference, search request, search for the conference and then in the end apply for the conference. So it's able to analyze the signature of the methods where

is the user input what is the domain object of the output and how to provide the path to the kind of the thing that we annotated with the achieve goals. Yeah. And this is with that I we have here we will using we'll probably using more tokens because we use LLM to decompose our prompt but it will probably we can use weaker domains because it's only analyzing

the prompt models are capable of that but we will shrink the possibility that something will completely go wrong or we will see it in the exact step where it went wrong. So look into this. It's very interesting. We have another competitor from Jet Brains Cook which basically does the same. This they are all built provide other abstractions because Springi is only basic thing. There is a chat

model and there is a tool but they yeah they cannot decompose currently. Maybe it will come in the later versions. So now 4 minutes enough for um wrap up. Yeah, we we looked into how to build MCP server and clients with Spring AI, but we also looked into the operational stuff like runtime once again for running the MCP server or agent with the HCP protocol gateway which

is kind of the unified platform to talk to the things like uh to the MCP server and we can expose APIs as MCP which is very powerful. So we don't need to repeat the logic and we can secure it uh basically um easily. We looked briefly into memory short and long-term memory and how to give it to springi. For that we need the springi agent core u

module and we look into the observability provided out of the box by agent core. So the the cool thing the services are composable. You can run your A once again there is nothing that that that is bound you here to use certain models or certain frameworks. You can use Python based whatever you would like. Yeah, this is framework agnostic, model agnostic. But the thing is also you

can run for example your agent on agent core runtime and if you have better service to provide you managed MCP like on Azure use that what's the problem? Yeah. In the end you are talking to the URL capable of speaking MCP protocol. That's it. Yeah. Where it's located doesn't matter. Yeah. The important thing is just it provides the capabilities and yes the observability stuff if you are

not happy with cloudatch AWS observability and you have contract to data do open telemetry helps you. You can push it there. It's even other way around. If you run the agent not on the runtime, there is a possibility to put the observability into AWS. I don't know if somebody will run the agent not at AWS but use the observability there. It's probably the rare case but basically

yeah you can use everything you can use only um certain things there are certain resources because you probably know how the things work there is a war what is better Python or Java and probably the answer is for data scientist Python is unbeatable but for writing enterprise application Java has a very strong foundation yeah and you can use spring AI there is length chain for J it's

up to you but for enterprise application with a strong typing of language. I think that uh the Java is very well positioned and uh all these things how we can talk to MCP are coming. There are certain metrics that have been released the latency how to talk to MCP but also throughput. Yeah, people using Java were pretty happy especially then they saw how it worked for Python

and so on. I don't know if it's opinionated uh test or not but it was heavily discussed several months ago on different networks. So you see Java is very well positioned for writing agentic applications. Uh yeah so I wrote I I continue writing the series where I explain that step by step. I will release the the slides so you can basically look into this. I released five

series uh five articles and we'll be doing that. Um we'll continue doing it especially when springi 2.0 will will be released so I can update this. Yeah, these are GitHub examples. Once again all the things that I talked to you you can check out there are even more things now with 2.0 and there is infrastructure as a code. So if you're familiar with AWS, you can run

it there or somewhere else, but you will need to change infrastructure as code. [snorts] And that basically it I have 27 seconds. So that means the time management worked. I know it might be a bit heavy especially the operational side if you not familiar with AWS. But basically my message is don't ignore the production side. The operational side. Yeah. With all that stuff otherwise it's not production

ready agent. I would say it like this. Other way that thank you very much and have a good rest of the conference. [applause]