Production-ready Agentic AI: Building enterprise-grade Java systems with Jakarta EE and MicroProfile

In this talk, Kenji Hasunuma explores the intersection of Java and artificial intelligence, specifically focusing on how Jakarta and MicroProfile technologies can be utilized to create AI-infused applications. He discusses the types of AI, including predictive and generative AI, while emphasizing the rapid pace of AI development that often does not align with traditional Java strategies. The speaker proposes a balanced approach that incorporates existing Java assets and skilled engineers in building AI systems. He introduces a pragmatic three-step strategy for integrating AI with enterprise systems using Jakarta EE's robust architecture and standards for security, transaction management, and observability. A demonstration showcases the implementation of banking transactions using Jakarta EE, LangChain4j, and MicroProfile, illustrating the effective integration of AI functionalities within a reliable enterprise framework.

Okay. Thank you for joining my session. I'm Kenji Hasunuma. And today I will talk about something about related to the AI. But I'm not going to tell you that fancy new technology. But instead, I will talk about uh secure, reliable, trust, Jakarta, and MicroProfile technology to build AI infused application. So, today's agenda. First, I'll start the uh looking at the relationship between Java and AI. And after

that, I we will dive into how Jakarta and MicroProfile technology solve the problem when creating So, let me briefly intro- introduce myself. So, uh I served as some uh members of several uh Java related groups such as Jakarta Adopt Team and JCP EC. And I'm also working for Fujitsu, and Fujitsu provides some Jakarta EE compatible implementation. Okay. So, let's get start. First topic. Java and AI. So,

let's classify quickly the uh types of AI. So, AI is a broad field with the machine learning and deep learning as subset. And most of AI used in real world is predictive AI, which predict future outcome based on the historical data. On the other hand, the uh generative AI is uh emergence. That is to create something new, like uh ChatGPT or uh diffusion. And the generative AI

is expectation is very high, but actually these are not used so much in the real world at the enterprise area. So, today's topic is right to hand. So, how we can use generative AI into your your Java application. This is a theme of this topics. So, looking at the AI and OSS landscape, as you may know, the main stream of AI development are not happen at open

source. very limited number of big tech company develop the AI uh core part. But, some of surrounding uh technology happened at open source. For example, the Linux Foundation compiled the uh landscape as a data and AI. Like this. But, you cannot see the detail, but it's contains Java related product. So, that's mean this landscape is targeted toward data science, but not application developer, uh specifically Java application

developer. So, why this situation is happened? The reason is the pace of AI technology is very fast, does not align with the traditional Java strategy. That is uh standardization and compatibility. But, on the On the the company want to when creating AI infused application because they have a lot of Java asset and Java skilled engineer. So, the key point to success of the AI infused application is

to leverage their company's Java asset and Java engineers. So, to do this, uh we need to have another strategy. That is balanced strategy of uh innovation and compatibility. So, here is a a pragmatic uh three-step uh strategy. Uh step one, we provide a framework to create uh AI system integrating with the current existing system. Okay? And then step two is we validate these integration with the real

world use case and systems. And for the long term goal, we will provide the Java specification and This gives the companies the confidence to commit and invest this technology for a long term uh period. So, now maybe we are uh between step one and step two or around step two. And so, I I mentioned uh simple type AI infused application, but what is AI infused application? AI

infused application is a uh program software program with uh intelligent functionality by leverage AI models. So, up to here, it's not big deal, but the difficult part start here. It is not only infusing AI, but also ensuring the enterprise grade quality equivalent to the existing system, mission critical system. For example, security, observability, transaction, and so on. These are the difficult part of creating And looking at the

architecture of AI around the application, centering around the application, it's interact with sub- several components like vector data or a model and so on. So, but if you look at this image, you'll be you'll feel like you've seen somewhere before. Okay? So, it's exactly the traditional distributed component system. So, where exactly Java excels. we will connect the need of AI infused application and the solution that Jakarta

and MicroProfile provides. As I mentioned before, the uh architecture of AI infused application is a traditional distributed uh component systems. So, which is exactly the Jakarta EE And further, Jakarta EE has a long-term history of being used in enterprise system, which require a high uh reliability reliability required. And further, Jakarta EE also standardize the specification in a vendor neutral way, so that the company safely use Jakarta

EE specification. So, because of this Jakarta EE is the best choice for the first citizen of building AI infused application in enterprise system. So, this landscape maps the current Java technology in AI domains. uh it contained my uh some biases, but in particular the uh right-hand side or uh center bottom there are a lot of Jakarta and MicroProfile technology. So, these are exactly the uh technology to

be used creating the AI infused And in addition to the Jakarta EE, uh in later demo, I use LangChain4j, so I I will mention a uh little bit about LangChain4j. So, the detail of LangChain4j is uh uh described at the previous uh e bus talk, but so uh here I just briefly uh explain. LangChain4j is a toolkit for building AI infused application, but AI infused So, you

can easily uh create AI infused application on top of the Jakarta EE. And one more project is the LangChain4j CDI. It's a subproject of LangChain4j. And it's a uh toolkit uh creating AI infused Jakarta EE application. So, these uh LangChain4j project are very uh strong library and combined with Jakarta EE. So, I used uh LangChain4j, Jakarta EE, and microprofile to demonstration. And this site summarize the our

argument. Every enterprise grade quality has a solution in Jakarta and microprofile community. For example, authentication and authorization, we have Jakarta security, microprofile.authentication. For observability, we have microprofile telemetry or remote call, we have Jakarta receivable web service or microprofile rest client and so on. So, pass to the AI infused application is already here. So, we can just use this technology. And from now on, I show some demos,

but there are a lot of those things to of that technology. So, I today I just focus three things. A security, transaction, and observability. And this is a demonstration we will show you. We have two servers. One is agent server. It's a blue one. And the other one is a MCP server, red one. And when the user makes a request, the supervisor agent receive this [snorts] request

and delegate some task to sub agent. And sub agent interact with external tools which are hosted by MCP servers. So, this is a demo structure. And the scenario of today's demo is a simple banking transaction, like a credit some money to some account or withdraw some money from some account. Very simple uh banking transaction. Okay. Uh the first topic is security. Uh we I use uh for

security, I use OpenID Connect and JSON Web Token uh to secure these two servers. Let's get uh look at the code. We have two projects. One is for the here, and one the other one is uh MCB server. And look at agent server first. Actually, the agent server is a Jakarta EE application. Uh more precisely, Jakarta Jakarta RESTful web service, JAX-RS application. And we have uh two

entry point for Jakarta JAX-RS application. One is post method, and path is ask. Actually, secure path, secure ask. And the other one is uh for get method, and path is top. And this entry point provide HTML like this with the form for the user input to request. And user click submit button, then uh the control is transferred to uh this entry point, the secure ask is uh

right here. And here I want to talk about how to secure this entry point. The point is uh this uh annotation, and Connect to secure this endpoint. OpenID Connect can be used with the Jakarta Security since 3.0. Jakarta EE 10. And the usage of the OpenID Connect is very simple. Uh you can just put some parameters to this annotation. But uh some of the parameter are sensitive

data secret. So I create uh Jakarta Expression Language like this and I create OIDC config like this and OIDC config is and I use MicroProfile Config to externalize the parameters. So that uh you cannot uh specify the sensitive data into the source code. So let's build and build and run this program. This is a Maven project the package. I created a uh a usual war file, okay?

This is a JAX-RS program. And I use GlassFish server to run this war file. GlassFish 8 is uh Jakarta EE 11 And run this script. Okay, it's uh this server listen on 8080 port. So I access this from the browser. Local host So, it's redirect to the Google page. Because I use the Google as a ID provider. So, I put the ID for this demo. And password.

And this is the Maybe you often seen this screen. This is a consent agreement screen. And click continue. And again, redirect to our application. This is the flow of all OpenID Connect. So far, it is a standard Jakarta EE from now, I connect this AI systems. Okay. Let's look at the uh MCP server code. So, this this is a central uh MCP [snorts] server code. And I

use MCP SDK to create MCP server. Uh these class are provided by MCP SDK. And I don't want to uh tell you the detail of how to create MC server uh because it is out of the scope of this session, but I briefly uh explain this code. Actually, MCP server provide a feature to create the uh MCP server on top of the Jakarta EE servlet. These are

the code using MCP MCP servlet is a actual instance of Jakarta servlet. And I create three tool credit tool, withdraw tool, rollback tool. This is these sub three sub Sorry. This is the three tool, bottom right. So, I create three tools. Okay. this portion is a standard Jakarta servlet API. Nothing special. And here I add the this MCP servlet to the Jakarta context. And also I add

servlet mapping. Okay. So, you can use MCP server as a servlet. And this portion is a security constraint, but temporary I comment out. I will come back later, but but the definition of the exact tool is here, but very complicated, so I I don't explain, but instead I will show you the MCP inspector tool how this server looks like. So, it's also the project, so first build

And it also uh creates the war file, and I also use uh GlassFish 8 to run MCP server. This MCP server listens on port 99. And I use that tool that is called MCP Inspector, which inspects any MCP uh I inspect my server with this tool. >> [clears throat] >> and specify 99 and MCP. This is a servlet mapping. connect. And I I told that we I

created three tools. So, let's look at the uh tools uh tab. And click list tool button. So, we can there are three transaction tools credit, withdrawal, and rollback. This is a corresponding that I created uh Credit, withdrawal, and rollback tool. And you can simple test in this window. For example, uh select credit credit transaction and specify some parameter in this Alice or account name amount, 10. And

click run tool. Then uh there are some response. So, now you can uh make sure this code function working as a MCP server. Okay, now debug this comment. I use OpenID connector to secure the agent server, but in this time for the MCP server, I use MicroProfile job to secure the MCP Okay, let's build again. change the code. And run again. Yes, 99 port. if we reconnect

this server again, it happened error I put security constraint. That's means that the I secure the MC So MCP inspector tool cannot connect to the secure MCP server because I didn't provide the valid JSON web token. So it's mechanism. So how would can we pass valid JSON web token to the MCP server? Again, so back to the So this is a code to create the agent. I

use LangChain 4j to create agent, and bank agent. Credit agent and withdrawal agent and rollback agent. So, we create three sub agent. Each agent uh corresponding to its tools. It I make a one-to-one mapping, but it's not necessary to make a one-to-one mapping, but I I here I make one-to-one And uh Okay, the point is how to pass the JSON Here is uh uh creating the uh

transport uh layer of MCP. Uh the communication between MC agent server and MCP server. I put the HT HTTP header uh authorization and bearer. And the bearer is here, and what is context? Context is OpenID Connect. So, what uh what I'm doing is sub agent connect ID provider and get And sub agent extract the validation web token from OpenID Connect and pass the JSON web token to

MCP server. So, this security mechanism I use. So, this is a uh very secure way and lightweight uh to secure both agent server and MC server. Okay, let's request something. >> And here is withdraw 100 euros from Bob and euros to Alice. A little bit smaller, but uh this is a typical uh simple banking transaction. Withdraw 100 and credit 100. ask button. And it takes a little

bit of time because uh agent server could interact with AVM. And it's successfully, so this screen [snorts] This is a terminal of MCB server and you can see the two requests. One is withdraw transaction and one is credit transaction. Uh these are expected. Okay. And let's try the other and 300. And click ask. Yeah, it takes time. I have withdraw use all Something strange. Sorry, try again.

Expectation is uh failing the that this transaction. Yeah, and I apologize, but we don't is not proceed, cancelled. Okay. And this again terminal. So, we have two transaction here. One is withdrawal transaction, and one is rollback And what the reason why this request is failed is I put the initial deposit uh very little. Yes, I put only 200 euros as initial deposit. So, you cannot withdraw 300

uh euros from both. So, this is the reason why this request is failed, but uh this is not point. So, I want to say about the transaction uh here. This is a code uh that implement the uh transaction in the MCP server. I use Jakarta transaction like this, user transaction, begin, and some some work, and commit, and something error, rollback. This is a typical acid pattern. So,

you can use uh Jakarta transaction as uh before. You can use Jakarta transaction for the local transaction, but how about the uh remote transaction agent and MC server between agent and MC server. So, you can use of course Jakarta transaction or acid pattern only if the resource resources implement the XA interface. But, typically, we are recommended to use a Saga pattern or remote transaction. But, the point

is uh somehow sub-agent can submit the a new uh compensation transaction to revert the previous So, through that, I create rollback So, I If you uh provide some rollback transaction, uh to the MCP server, so the agent can access this tool and rollback or revert the previous transaction. This is a uh uh typical case to use remote transaction with agent and Okay. Uh next is observability. It

is very easy. You can use MicroProfile Telemetry or OpenTelemetry. And you can also any uh visualization tool you like that that is Jaeger or Zipkin or Grafana, whatever. But, uh today's demo, I use a simple uh console visualization that is called Otel TUI. And the use of the OpenTelemetry is very simple. Just provide the uh system property or environment variable. For this demo, I uh provide Jacarta

MCP as a uh service name for the MCP server. I I put the Jacarta agent as the service name of Open Telemetry. Yes, server up. And again, I access from And I put same And create it. And push as product. here is a hotel TV console. So, you can see some traces. It happened this demo. And you can click more, you can see more detail of the

trace, like post method and what duration it takes. yeah, you can also find some metrics both MCB server and agent. Like this. And skip rows, and you can also see the topology. observability very easy. Okay? So, this is the end of them. Uh it's a next topics, Jakarta agent AI. So, this is also uh uh provided some detail from that uh previous eBus talk. But I uh

just briefly uh introduce uh Jakarta EE AI. So, as you see the demo, LangChain4j have already the uh feature to create agentic flow. But uh it's a little bit complicated and uh it just provide a low-level And the goal of Jakarta EE AI is uh more simple. It's a defined common usage pattern and life cycle for uh running Jakarta EE runtimes and uh integrate the existing Jakarta

EE technologies uh seamlessly. This is a uh scope of Jakarta EE AI. It's not It has not released yet. But uh uh it will be a standard stand- stand-alone uh specification. Not not included in the platform or or So, uh let's wrap up. The most important key takeaway from this session is uh enterprise grade quality is the key to to building the uh agent uh AI-infused application.

And Java and Jakarta EE uh the first citizen in this new era. Uh because they have a uh long proven history of being used in enterprise uh high-quality system. And of course, provide uh strong compatibility, but at the same time, Jakarta EE community uh also advancing AI technology for the enterprise people. So, you can use this existing technology and you you can uh combine with the new

AI So, this is uh my uh presentation. Thank you for taking time. So, if you have Yeah. Thank you so much for the presentation. How do you see um Jakarta EE um deploying models or applications on the edge devices, basically local devices, if we can see with where we have the limited number of memory, and JVM itself takes time and memory. In my personal uh opinion, it's

difficult to fully uh Jakarta EE application to that uh device, but uh CDI has a uh some uh body variation of uh versions. For example, uh standalone CDI can use just Java SE. So, if you can use uh this kind of uh CDI, portable CDI, so, you can uh deploy CDI and some Jakarta technology to that local device. Sure. Thank you. >> [music]