Orchestrating Intelligence: Multi-Agentic Design Patterns for Production AI - Mary Grygleski

This talk explores the development of multi-agent systems in the context of AI, focusing on architectural patterns and challenges related to event-driven workflows. The speaker discusses the evolution of AI agents, particularly in enterprise applications, emphasizing the distinction between single-agent and multi-agent systems. They highlight design patterns for building production-level AI environments, addressing the importance of memory, reasoning, and planning within these agents. Additionally, the session touches on practical applications using Java, such as integrating AI-driven alerts within distributed systems. The speaker also raises awareness of ethical considerations and the limitations of current generative AI technologies.

May I actually actually ask you how many of you are already working with the AI agents? Let's see. Um Oh, okay. Oh, wonderful. Are you mostly Java developers here? Are you guys mostly um who are not not Java? Oh, okay. Oh, so still quite a few, but do you work have with Java as well? Are you familiar? It's just that currently Okay, that's great because my examples

too um mostly right now Java because this conference started off more for Java developers. So, I've been assuming it's more for Java. So, but yeah. Anything I'm going to speak to will be applicable for any Yeah, anything. So, okay. So, the talk to this This one I sort of treated like sort of an extension from what my talk from yesterday. I I'm doing two talks here, which

I already did the first one yesterday. That was more introductory event streaming event you know what you call it event-driven approach for more multi-agentic type of workflow kind of systems. And I do have to admit too it's a bit exploratory at this point um because I don't feel the current state of the agent type of usages or development work, right? That that's being used I think in

real production. It seems to be more about uh some single agent dev defining some steps and then you know you do certain things maybe just uh have some tool calling in the back and doing something. Uh don't get me wrong. I think it's still very good some of these agentic way of doing things. It The ideal situation is for the autonomy autonomous type of scenario automation essentially

um and leveraging large language models, but what we're trying to address here is more about multi-agentic, which is more for I I kind of in my mind I'm more thinking of the true next generation, next phase of agents development. That's actually more catered for enterprise level type of application, which I don't think is there yet. What we're doing now is again more kind of single step, I

mean, single agent at the way I look at it. So, so then this particular talk is a bit more exploring some design patterns for more production AI. That's how I named this title. So, okay, because I I actually proposed another topic and then at the last minute the organizer at Git here was asking, "Oh, I saw you doing this one." And so I have to admit this

a bit newer topic and it's going to be new to. And I also got asked by a gentleman from my talk yesterday afterwards. He said, "Have you actually done anything with, you know, implementing multi-agentic?" And I do have to admit at this point I have not have not done any serious type of multi-agentic applications yet, but I do feel there's a need for us to talk about

it and explore it. So, that that's why I'm going to talk about this. So, okay, so enough of that and this is the the plan for today. It is just I'm going to have some introduction talking some fundamentals first about AI agents just to make sure all of us are, you know, coming from the same point for this discussion of this talk. And then some agentic architecture

and patterns and some code samples as well. And then also I'll discuss some challenges of GenAI. Sorry, seems like there's a bit of a static. So, do we know like why? It seems like it keeps having some static, right? Does it bother you? Yeah. Okay. Yeah, okay. But I think it's okay. All right, so and then that that's the plan. Okay, so who is Mary? And I

our MC already introduced some so thank you very much and but again picture is worth a thousand words and how many of you actually came to my talk yesterday? I'm just curious too. Also a few of you. Okay, so I may have something that I'm repeating so pardon me for doing that but right now too is just a quick introduction just picture is worth a thousand words

and so I'm also Java Champion and also Oracle Ace too and but at the heart of myself I'm still passionate advocate meaning I like to go and speak and also you know I will reach work and meet developers like yourselves like that and but right now too I'm also involved with this group called AI Collective and how many of you I'm curious to have heard of AI

Collective or maybe attended any of their meetups. No, not yet. Okay, but anyway, just to give you an idea, you know, how Java we have Java users group. This one AI is not strictly speaking it's not really like users group so to speak but it's more a collective of people that are can be developers can be technical and also non-technical folks such as you know people who

are investors, who are founders. Some founders are developers themselves too but also sometimes a lot of founders could be non-technical. They are marketing sales in that you know in that kind of fashion and also kind of the business strategies kind of people and so as such you know the the meetups too that we organize to around the world now is global and there's also a Bangalore chapter

by the way and in India we have Delhi and also Pune and we're going to open up more maybe in Mumbai as well in Chennai all of these cities too. So anyway, I'm like responsible for the Western Hemisphere. So we do meetups like regular meetups like technology but we also do what we call pitch night and these are like for founders. So if you How many of

you, I'm curious, are founders actually? You may be like a you know a small startup founders, no? Oh, one. Okay, that's cool. Yeah. So, yeah, I I think maybe some cities is like San Francisco and in the USA is a lot of founders, as you may be aware. So, founders, too, they like they need to bring their idea business idea because you want to raise funds or

whatever, you know, reason you need to raise the visibility. So, we also offer um these kind of meetups that's called pitch night. So, these are founders uh coming together to pitch. So, usually it's like a high-level product pitch. So, so nothing like deep down technical dive, but still interesting in the product sense. So, people can do pitch pitches for like 5 minutes, and then we also have

investors that come and try to evaluate, maybe will attract some of investors to invest in some of these uh startup companies. So, so that's the kind of thing we do. Um so, essentially getting into founders community as well. So, I'm the VP of the global uh for Western Hemisphere, but I also travel everywhere. So, that's why I'm in India as well, and I also was in Europe

as okay, and I also help the Chicago chapter um too um as well cuz that's where I live. I'm from Chicago. I'm also I also run the Java Users Group in Chicago, too. So, I I'm for Ja- Java Users Group C jug. I've been doing it for over 10 years, just just so you know. So, okay, so enough of myself, but my interest, too, is really into

distributed systems. How many of you, I'm just curious, work kind of call yourself more of a distributed systems type of person? You handling streaming Yeah, okay, quite a few, too. So, that's cool. So, as as you know, if you're doing work on distributed systems, they tend to be more complicated, too. I mean, because we're now talking about data that's going to flow through different uh network boundaries,

so to speak, right? And also, too, in some cases. And that's what we want to kind of investigate into is like agent system currently are pretty kind of single-minded so to speak and that's maybe using tool calling using MCP whatever it is conforming to MCP type of thing but it doesn't maybe necessarily deal with multiple you know your data flowing through different network boundaries so there's a

need for making sure that if you have a transaction that's open and the transaction is going to flow through different sub systems and then you need to make sure the data comes out correct and consistent so that's kind of like the big challenges one of the big challenges of distributed system but anyways so that's what I'm kind of interested into okay so let me kind of quickly

get into fundamentals of agents first so just so get some understanding so basically to agent right the way it we are understanding it is is in today's world is a bit overloaded I I should say right it's like this big truck kind of everything's overloaded meaning you know it from a marketing point of view or frameworks or research I think the non-technical folks tend to think of

agent is a bit more like magic you know that and then it's interesting when I'm doing AI collective is like when you talk about agents and people are fascinated because they don't quite understand how you know programming works you know that and they're more thinking of maybe product like that so it's a bit on the marketing level maybe thinking of that and then they're also like framework

level two there's also I'm sorry I'm sorry just push it there push it out okay I noticed yeah it's yeah it's better right okay all right thank you maybe that's right sorry okay so oops can you still hear me right okay okay that I think that's better if I pull it away okay so okay so agent again back to it agent can kind of be confusing in

that sense but let's kind of get back into some fundamentals too of agents. I feel there's a need to kind of understand it and also too I wanted to point out right from computing point of view actually it's not new. I'd say you know if you are familiar or working with Unix or Linux systems it's pretty much is also using a lot of agents but I call

those agents demons too. They are process that that's running right. So it's essentially they usually are focused on certain goals doing something very well. So let's kind of take a little little quick look. Fundamentals of agents so why do we use agent at all? And because some tasks you know work better when the whole I call it shebang are handled by some single party. So for example

you know I'm going to use some examples right of an agent. It can be travel agent or insurance agent right? And and it's kind of likely same thing like for us here like without you know kind of mentioning AI if we are you know living in today's world it can be very complicated. So if we need to get a car insurance or house insurance or something like

that. So we can actually go and purchase the the insurance ourselves right? Like travel especially like say travel agent. We want to travel here let's say from Bangalore to Delhi. You want to make reservation and you can book it yourself of course but then what about you know sometimes you could run into problems as the flights get cancelled a flight get you know moved or updated all

these things or you need to try to find the best route and you don't have enough time to search for it. So things like that right? It takes all takes up so much time. So it may maybe worthwhile to kind of delegate that's that's the keyword to delegate to some other party to handle the whole thing. That's why I call it whole shebang and it's basically it

takes care of the goal of taking you safely from Bangalore to New Delhi something like that. So that's an example of an agent and some things are better handled by agent. But, it doesn't mean everything is good using agents, too. So, that's also a bit of a takes a bit of like experience and skills to figure out, you know, agents are not for all situations, but in

a lot of cases maybe it makes sense. So, I then you know, ideally what the duties for an agent is is basically first to understand the goal, you know, what are you trying to achieve first the goal and then essentially to once you know what it is then you need to break it into different steps, especially we as developers and engineers, we we don't just like say

it's not magic. It's not like raising a wand and oh, go do this. It's more like we need to break down, see what is needed and into different sub steps, right? Sub kind of sub parts of that. And uh so, from there, too, you also need to make decisions, too. And so, when there are conditions change, um for example, like you're booking travel and you know, there's

a weather uh problem coming, you need to rebook. Things like that, you need to immediately make some decisions, cancel your flight or update whatever is needed. Um and then also there are exceptions, too. Maybe like you know, completely a flight got canceled. Now, you're going to change your plans, you need need to fly from Bangalore maybe to Mumbai, Mumbai to Delhi, something like that. Handle the exceptions

and then also to keep track of the progress, too, in a lot of cases. So, that's kind of like basic abstract sense what the duties of an agent is. So, examples already kind of quoted travel agent or insurance agent. Okay, so now then get in getting bringing AI into this conversation is basically AI is essentially autonomous. This autonomous um way of performing task and also specifically is

working with language large language models. These are like the premier kind of you know, name for the kind of models that's working with the current state of AI, generative AI that we're working with, right? At this core of this AI agent. So, sometimes these agents are also called LLM agents, too. Um but, as we all know, LLMs are very restrictive in the sense because it's kind of

bounded. It's depending on what data you use for training it. And that's all it knows. That that's, you know, that's the LLM knows about. So, the scope is actually as large as the data set is, it's still kind of uh bounded, you know, by what it's being trained on. And the reasoning is as well. Right now, as you kind of as we all understand, it there isn't

any true AGI, you know, artificial general intelligence, which actually is about true reasoning in which, you know, then agent able to figure something out. As we all know right now, you know, as we have worked actually with LLMs, you know, since ChatGPT came out in 2023, it's not actually magic. Again, it's like uh Neil Ford, uh one of the speakers, right? The architect speaker from ThoughtWorks, he

talked about uh at Venkat's conference actually at last week. And he said, "They are actually LLMs are like recipe finders." And think of yourself as like trying to cook food, and you need to like how do I cook food? And you ask the agent, it needs to look for some recipe for you. But, it in using an analogy, that's what LLM is, right? And that's what it

is. So, basically, agentic technology will extend um the capability of LLM cuz because of this limitation of LLM. And the way it does it is, you know, via tool calling in the back end. And also, too, um it should be able to obtain some up-to-date information because LLMs are trained with old data because you it needs a lot of time to train it. So, usually, too, the

LLM that comes out would contain data from, you know, couple at least couple months back, like for example, like that. And then also, the workflows, too, will have to be optimized. And also, agents ought to be able to create subtasks to all of these things and being able to be flexible. So, now let's take time to take a look for single agent, you know, and what is

an AI agent? It's clear definition is basically AI agent, it knows it should be able to perceive the environment, too, and then make some decisions and take actions on it, too. So, it's not like a chatbot, you know, chatbot usually well, they are also you can think of them sort of an agent, but specialized case in which it is responding to some prompts. Uh usually a prompt

will kind of guide what the chatbot should do, but agent sometimes is could be some background task, you know, like think of again, going back to Linux um kind of environment is you have cron jobs that does stuff, and those are kind of is actually agents to that handle handle some of the cron jobs, all of the steps, too, but this is kind of like the agent

definition of agent. And then also some building blocks of agents, these are core components of an AI agent. It's very often it needs memory because it again, you know, LM is stateless, doesn't have memory, but that's what agent will make possible to be able to remember what it has been asked or some, you know, some like the context essentially. With the memory, you can build up a

context, so then, you know, you kind of maintain the state for you your interaction with an LM. And of course, too, to a certain extent we call it reasoning, but it's not true reasoning, but in some ways it's like agent will rely on external tools, for example, to help with reasoning. And then also planning, too. Planning is essentially you know, we give it a problem and agent

can should can and should be able to break down into sub-steps to to kind of tell how to do something, and also pro pro kind of provide a bit of an action interface, too. So, essentially AI agent is composed of multiple layers, too, that which are like these kind of four building blocks here. Okay, so we'll get into a bit of agentic architecture now. So, the current

definition of agent is basically a system that first receives a goal and then two will decides what to do next, right? So, that part is the planning part. And then also then it needs to keep uh the context relevant. Keeps these relevant contexts and then it is essentially does all the execution and then repeats until it reaches a certain stopping condition, what you're trying to achieve, like

that. And then what's kind of interesting, the planning uh when I was looking up IBM's definition is, you know, formal definition is planning. It's like determining a sequence of actions toward a goal alongside modules like, you know, memory, reasoning, and action. So, that's thing. And our current current understanding too, especially when we work with AI agents and it deals with like prompt chaining. As we all know,

you know, you if you're working with prompts, these are essentially instead of using one single prompt, you could be breaking, you know, your prompts up into like different uh steps. So, then you guide guide it so then it's not just one chunk of prompt going in. It's more like you chained it up so the input, you know, from one or the output from one the first prompt

can become input to the next prompt. So, and so on and so forth. So, it kind of makes you can kind of guide it like this and makes your what you know, achieve what you're trying to achieve essentially better, right? It kind of reducing hallucination in some sense. And then of course there are also tool calling assistants and these are usually um a kind of like tools

uh working with any kind of external tools in the back end too. As well as also handling stateful type of workflows and these are essentially two um We we probably won't get into all of these yet, you know, like when we talk about stateful workflows and very often, I think if we think of enterprise level then it's probably dealing with more transactional type of scenario. But at

least too, like workflows could be there are different workflows that carries with it data that you want to make sure the data is is being consistent between different steps as it step through the different, you know, flow type of procedure. Okay. So, now I kind of get into some agentic architecture. In this case too, like just now I talked more a bit more single agent, but let's

see, you know, in in kind of more enterprise level of execution and agentic type of ways of doing things and basically, you know, having single agents probably not going to help as much as basically having multiple agents kind of scenario. We are still using some multi-agent scenario now to do things too, but then the the fact is, you know, it's basically say for example, we're going to

design some kind of airline reservation system completely guided by agents. Then in that case, you probably will need multi- multiple agents because you need maybe an agent first to interact with you and then from there, it actually, you know, the goal is to book tickets, you know, and and manage your flights and and probably then you have like different steps of booking your tickets and then eventually

too, you also have to see if there are, you know, you have to interact with another booking system with all of the maybe another system that aggregates all of the flight information. You have to then, you know, figure out if the time would work with your personal calendar, things like that. And then later, you also have the agent interact, you know, with your payment system as well.

And and so, all of these things will involve multiple subsystems along the way. So, it's it's going to be always going to be more complicated than one single agent doing everything too. You have multiple agents and some are doing certain task. So, basically in agentic architecture sense, then you can have like a vertical architecture, essentially having one agent be the leader too. And then, um, they're are

also like what is called like horizontal architecture, too. So, then this case you can have multiple agents, each collaborating, but they are of equal kind of, um, what you call equal power, so to speak, that it doesn't it isn't quite a leader, but they all kind of collaborate together, and depending on situation, maybe one could be voted as a leader, like that. And then, also, too, then,

um, in more complicated scenario, you also need someone essentially another agent be the orchestrator. So, the one orchestrator is the one that kind of overseeing, making sure all of the agents, whoever is supposed to do do some task, they are doing it in the proper sequence. As we all know, sometimes, especially say with financial type of situations, for example, you need to somebody needs to with I'm

just citing a kind of a straightforward example, somebody needs to withdraw money, but it actually needs to have the transaction of him putting in money first before they can withdraw, otherwise you'll have a negative balance, for example, things like that. So, you also need, for example, in this case, maybe an overseer type of agents to be orchestrated to orchestrate all of these things, but of course, that's

a simple kind of cases. In really complicated type of financial transactions, for example, making buying and selling of stocks and options and trade trading situation, maybe you really need a better orchestrator to orchestrate everything. Um, so and also I like to kind of make a the compare an analogy, too. Orchestration is necessary. It's like if you kind of think of it if you are like musically inclined

and you listen to music and you know, in an orchestra or any kind of an ensemble of music, you know, kind of a performance, there can be multiple, you know, instruments playing, but you want to make sure the music comes out correct, too. So, very often, right, in symphony orchestra, you need an orchestrator, then that's their job is to conduct and making sure all of the instruments

are playing music at the right time. So, then the music comes out correct to kind of a pleasant to the ear, so to speak. But it's kind of in that agent should work like that. And of course, too, then it should also help with, you know, complex kind of task execution. It should kind of manage that and also the autonomous decision-making side. And right now, again, you

know, agents are not fully, you know, 100% like able to reason, but we can kind of maybe have external tooling, that type of stuff to help to make it sort of like helping human in the loop type of reasoning as well and planning as well. And then also, too, one key kind of capability agent should have is basically a continuous learning and adaptability kind of capability capability,

capability, too. It needs to be able to adapt to the different kind of situations. And essentially, too, we want true agentic architectures to have no human in the loop or human intervention. Basically, currently or I should say, currently, too, a lot of cases in which we're seeing all these systems still require humans to be in the loop. You know, you still need to check it. And the

way the LLMs are today are still not quite You can't fully fully be trusting it. And I'd like to suggest let's not fully trust LLMs because they are, again, just recipe finder. They are based on fuzzy logic and they're non-deterministic. So, don't always trust it and always have a second pair of eyes looking at the results that it gets generated and verify. Either you you use another

agent to eval, like eval agent, to make sure it's correct or better, you know, kind of closer to accuracy than it is, you know, with completely trusting it. And but also, too, while I'm here, I think I should also mention because as as developers, we tend to be we're more rational in our thinking. We we do things and I think we tend to be better in terms

of not being fooled by any system. But let me tell you a story because I'm I also organize events, you know, so the AI collective in Chicago, we had an event that celebrating Women's Day, International Women's Day in March. And then I have some you know, mothers coming to a a meet-up. And not just mothers too, there are fathers too, other people, but it's basically celebrating women,

but I just want to share this story and it's telling about how, you know, AI, I know it's not developer specific, but I think it's worth mentioning because the mom said that her son has autism and then was taking medicine medicine. And so the son actually quite smart, 15-year-old teenager, just like any teenagers, you know, used ChatGPT and then ask, you know, he even he was asking

question about the medicine he was taking for his autism. And eventually, what he got back from the ChatGPT, I think through many conversations, he was asking different things, the ChatGPT came out and told him, "Do not trust your parents. They do not know what they're doing, you know, this medicine and blah blah blah." So the kid was like thinking, "Oh, that's what ChatGPT told me to do,

so I'm not going to trust my mom." And so thankfully, because he did it at school, at the school computer, so the staff caught him kind of like asking ChatGPT these things and he was check she was checking, so she immediately notified the parents. So thankfully, so the mom was telling the story. So as you can tell, all of these you know, AI thing, they are just

being trained with some certain set of data. If the data is bad, it is whatever it is, it's just spill out what it's being trained on. So it can be telling you not true things, things aren't true. So do not trust it right away. Always be skeptical, not to say don't trust it to the point of not being productive, but always kind of with the skeptical mind

saying that, "Well, this is just AI as a tool, it can be wrong too." and all these things. So anyway, I just want to tell that story. I like to kind of mention that because even I have other Java developers who was talking to me that they were kind of using it to generate code, and they were looking at things, and they were like, "Oh, this looks

like it's it it looks good." And they keep doing the the, you know, the code generation, the code assistant, and he this this friend of mine also was saying that, "Oh, I'm starting to believe that it is true." And so easy, right? That we feel like we can start to trust it, but just remind ourselves do not trust it that is the key. So, not all the

way. Okay, so let let me kind of move forward. So, some fundamentals to of agents again, you know, basically agentic and basically the the agent is one who is authorized to act for on the place of another. And so, always remember this not trustable. It's just somebody you can delegate to do some work for you. And also agentic is basically use systems that use AI to pursue

the goals and complete tasks on behalf of users. That should show reasoning and planning and memory and have a high level of autonomy to make decisions to and and adapt and all these things. So, that's kind of the general thing. Generally, I agents are basically software entities that orchestrate complex workflows and coordinate the activities of multiple agents, process logic, and evaluate answers. Multi-agentic systems are when multiple

autonomous work agents working together, and they each specialize in some task, and they also will have intelligence to from agent-to-agent kind of interactions, too. And it needs to be able to divide complex tasks up into manageable subtasks and leverage the strengths of different AI models to increase robustness and fault tolerance, Okay, so now I just quickly, too, there are also protocol that's being used from Anthropic is

model context protocol, MCP, and that one is more of a lower level that interacts with resources, you know, infrastructure, that kind of level. And then there's also A2A protocols, agent-to-agent protocol, and that one is from Google, but now part of Linux found governed by Linux Foundation, too. But to kind of look at the two kind of relationship, too, the next diagram will show you that MCP is

on the again, more on the infrastructural lower level, interacts with enterprise apps and doing all these things interacting with database, for example. Agent-to-agent is the higher level that's like between agents. They they can communicate among themselves. Okay. So here and and just a bunch of things, but I actually won't kind of going through all of it because I said we'll talk more about the patterns, too. So

I just want to jump to this talking about agentic design patterns. And those of you, maybe just a few who came to my talk yesterday, so these are some of the repeats too that I talk about, but I wanted to talk about agentic design patterns. There are essentially like four things that are you know, kind of can think in AI specific for agents, too. This one is

a reflection and then tool use and planning and multi-agent collaboration. So and also actually orchestrated to our going next, but first let's talk about reflection first. And so this one I also talk about yesterday, so bear with me if you already heard it. So basically, too, we want to design these agents let's say, right? These days where it's common we're using coding agent. But at the same

time, again, we want to be critical skeptical, right? About what what comes back from the LLM. We need to be skeptical in knowing that it isn't always trust you shouldn't trust it right away. So in this case we use a critic agent, which is also called an eval agent, to kind of be there kind of like looking over the back essentially with the coder agent. Coder agent

will interact with your LLM, ask question and maybe you know, kind of first ask your LLM to say, you know, perform some task, do something, and generate me the code. And so, the critic agent will be there and immediately like criticize, you know, looking at the output and will find some problems with it. With that, it asks the coder agent to kind of re- re- send this

information back to the LLM and ask the LLM to regenerate better code, like that. So, kind of so on so forth. So, it's like reflection is kind of a reflection kind of pattern, right? You just kind of have a feedback kind of loop. And eventually, you kind of do it enough times to kind of achieve that condition in which you know the the final generated set of

code is the most accurate. And so, that's a reflection. Okay. And then, the next pattern is called tool use. So, tool use too. Now again, I didn't kind of mention this all these patterns I'm actually essentially to describing what Dr. Andrew Ng, he's like a well-known AI person, right? He started the deeplearning.ai. So, very knowledgeable, obviously, person in AI. So, I kind of am quoting what he

says in here. And he has a lab that also does a lot of research, too. So, so this is what I'm just repeating. So, reflection is the first one and tool use is the second one in which this pattern, meaning that we know that sometimes LLM may not have the answer. So, in that particular case, you want to configure it, basically giving the agent some external help.

And so, these are essentially using tools in the back end to kind of extend, you know, what the LLM can't give you back or it doesn't have an answer, then the agents should go and reach out to these tools in the back end and ask for, you know, additional help. And that's what is tool tool use, this pattern. So, example is that it could be, you know,

you you giving it help and saying that, you know, if the agent doesn't get a proper answer, you can always use for example, go to Wikipedia, go to some sites and other website or another tool, whatever it is, to look for additional help, you know, to get better answers from that. So, that's essentially what it is. And then this one too, I'm using an example. I thought

just mention it because there's a quite a popular agent development SDK called Crew AI. Maybe some of you have worked with this Python essentially. And but Crew AI too, their model is is more of a rule-based agent agentic kind of approach of doing things. And so, they actually have this particular way of doing things and they have Crew and you can have different agents and and you

kind of define the tools for example, and then they're also process that helps, you know, define how the agents will actually work together to how tasks are designed, all these things. Because because it's That's weird that sounded me. And um So, okay. So, they they're also the Crew AI again, it's essentially the same kind of idea using tool uses reaching out to get external help. And then

essentially too, then they're also tasked and task can actually essentially override the agent tool with specific ones too. And so, they're also specific like agent that can tackle it too. So, it's a it gives a little bit of a different flavor to this tool use type of interface to in Crew AI. And then it will generate you the outcome. So, I have also a link in here

too. So, if I get to the point, you can if you like I'm sharing this slide deck, so you can also have access to this information. Okay. Now, this third pattern here is basically very interesting. It's called planning. So, in this particular case, you actually want the agent to do a bit more work, a kind of more thinking. So, essentially too, planning is, you know, using an

example like this Um, is you can just say give give a picture an image, right, to the LLM and saying that, well, you know, this is a little boy on the left-hand side is riding a scooter. And what I want to achieve is to get a another picture with a little girl in the same posture, but the little girl is reading a book. So, in this case,

yeah, you're not really giving too much kind of hint to the agents, really letting the agent be the driver to figure this out. So, so in this particular case, you know, being an agent, you know, I would kind of look at it, what is the goal? Let me kind of start breaking up the steps, and then it has to look, you know, and use techniques to kind

of figure out the image and all of these positioning, whatever it is using, that's what the power of LLM can come in. Looking, you know, using a visual based kind of LLM, look at this, and then it determine the posture, and then using some model to kind of determine the posture, and then essentially, too, using the posture to look for an image of a little girl, and

then in this particular example, maybe using a Google VIT kind of model, for example, and search for a girl, and then also look for another image that's reading a book instead. So, then it should then generate you that output on the right side like that. So, so as you can see, I'm skipping over all of it, but essentially, the agents will have to determine all the posture

and then figuring out the steps in order to achieve the final image in this particular case. But I want to also point out is that from this article that I read and which I'm sharing in here, Dr. Ing was saying that his lab at the time, maybe 8 months ago, it it isn't as it wasn't as accurate at that point as as, you know, they tried different

ways of using this approach or kind of trying to use images and generate the end result, but it's not not very good yet at that point. But I believe, you know, 8 months later, maybe these days the models are better at a better state, and also agents are better in terms of able to figure things out. You know, this kind of abstract input, it should be able

to do a better job. So, Does that planning? And then also then the fourth pattern in here that it describes is called multi-agentic collaboration. So, this particular one is a bit kind of not in real life yet, but there's there is actually a GitHub too that they are developing this. It's sort of like a game game-like kind of scenario. So, if you're interested, I also have shared

link to to its article. But, it's very interesting as you can see too. It's kind of mimicking an IT project using a waterfall model. And this agentic design patterns is essentially multi-agentic collaborative collaborations too. So, you can have different roles, you know, in your in in your code or in your in your particular system that they are different roles that are handling design designing things or coding,

testing, or documenting all these things. And there are different, you know, interaction between the different kind of roles among themselves. And then it should be able to then help to produce a project. But, that's again, you know, it's not real life yet. But, I think it's an interesting kind of concept that they are experimenting on. So, if you want to, you can look into that too. Okay.

So, these are examples of a few multi-agentic libraries like AutoGen from Microsoft or CrewAI. And I also mentioned to and on Java side there's LangChain, FourChain, of course. And then of course LangChain by itself is Python. LangGraph is also is graph-based and LangGraph 4J. And there's also some code example I'll show you shortly too. It's using LangGraph to doing better position the you know, the to to

do like orchestration type of work. And there's also Strands from AWS or Agent SDK from Google. And on Java side of your Java Quarkus LangChain 4j as which is based on CDI also has some examples too which in I've given in here. Okay, there just a bit this page is just giving you some additional code examples and there are also more too. So, yeah, so if you

if you miss it I think so I see some people taking picture but you can also yeah, get the link to this code. Okay, so just a quick example then. So, here LangGraph 4j some example here is just a quick one. It's not like you know, I'm not really showing working so to speak but just to show you using LangGraph 4j you can use a state graph

and essentially too it's basically help because you can then using graph and you can actually add the node and add add the edge. So, node is more like the entities and then edge is essentially describing transitions right between between nodes for example and then you can then use a state graph and do the compile and in this particular case it's just simply print out these items to

in this graph for example. So, so this is just again an example but wanting to kind of kind of point out to you is basically helps with a bit more explicit like state and orchestration to using LangGraph 4j. Again, you know, this might be something you can consider using if you're interested because especially if you're thinking of thinking of your code in terms of nodes and transitions

and deterministic flow. So, this state object help with making like memory and intermediate outputs more explicit and then nodes will represent the units of work or decisions and the edges define like all the transitions too. So, essentially too if you're doing orchestration you can maybe consider using LangGraph 4j to help you with it too. And LangChain 4j also has some higher level building blocks too for tool

calling and using chat memory rag, and agents as well. So, Okay. So, here are some challenges. I thought I have more too. Let me see. Okay, so challenges again before I get to the example, just wanted to quickly point out to, you know, using still, you know, with agents and doing like AI, I think it's exciting and everything, but just always remember that there are always some

challenges, especially, you know, these are complex and we have to deal with agent memory, context management, and you have to ensure the consistency among, especially if they're dealing with multiple distributed agents to the data consistency. And there are also failure situation, partial results, and also latency and resource usage, you know, how do you store certain data, all of these things. Yesterday I talked a bit about data

engineering side too that that type of stuff, the pipeline, the boundaries. And also too, ultimately too, the LLMs too are quite expensive as we all know, uses a lot of tokens. Sometimes, you know, when when you use it, you know, if you don't think about using it you know, before you use it, it can actually use up a lot of tokens like that. But there are also

Yesterday I was actually went to a DevRel Jam. It's called DevRel Jam meetup too. That's offsite. And and Brian Benz, who is from Microsoft, he talked about there's a I I'm less of paying attention to some of these chip maker, all these things, but I think there are new ways of kind of chip that enables you to use less tokens to do certain things too. So, those

kind of things, yeah, there are certainly there are advances to make in terms of, you know, reducing pollution, so to speak, and all that stuff. But still, you know, as we all know, you know, it can use up a lot of tokens, which means the data center is generating a lot of heat too. From what I know is that I think the the demand is quite high

that we're using all of these AI agents and tools and they're burning up a lot of energy. That that's the you know, that challenge is still there. I think we're advancing, but it's just not quite there to make it more kind of commoditized to make the cost lower in that sense. And of course too, LLMs too, one thing again is is It's a fuzzy logic, so there

are still hallucinations. We can't just trust it right away. And also one more thing, right, is basically the very important is AI safety and responsibility to the ethics side of things. Um and we just have to make sure, especially if we're builders, developers, developing code, we also have to be aware of the fact that what we're going to do, what we're going to produce is not going

to be misleading. And also we also as developers, I feel we are all advocates too. We should go out and educate people, right? Especially sometimes the non-technical people, they get a bit nervous. Oh, what is this? What is that? They just don't have the idea of how to do coding. They thought this is really like magic that can that can cause the world to end or something,

right? We we know it's not going to be that simple of just able able to you know, end the world. I actually read about something about this famous computer scientist, I think Yoshua Bengio, a Canadian computer scientist. And he made a comment that oh, this is very scary. AI you cannot even shut it down or something. And I was thinking, why is it not able to shut

down? Anyway, I saw that like a interview. And I don't know why they're saying stuff like that because I'm thinking it can have danger, but it's not about not able to get shut down. It's it's more about people misunderstanding and being like deceived by things that they think is real. But somehow he was saying that because I was immediately thinking, why can't you shut it down? You

can turn off the power, the power supply and everything will shut down. So, I don't know. But anyway, I got into some more talking with some colleagues and stuff and they were saying that, you know, sometimes these is so-called like celebrity computer scientist, they think they are the ones who they are the only ones who know the secret hold the secret key to how things work. Maybe

they are just thinking, you know, like a little bit like too arrogant to think like that. So, they are like scaring everybody. Maybe, yeah, I don't know, but I don't know if you've heard of that. But, anyway, to me I feel like, you know, nothing to scare about. Although, there are also things in which we are I I'm sure everybody maybe in India is not as much

an issue, but in the US I feel sorry for some of the junior developers. They are not They just graduated or about to graduate. They are having trouble looking for jobs and because right now a lot of companies, too, they are cutting costs. they're basically saying that, well, we don't need as many programmers because we can five code and get the job done. And while five coding

can help you, but I've done it before, but I feel that I still need to spend quite a bit of time to to make sure my code work correctly, right? Or doing the what I wanted to do. So, it's not completely and plus I'm I'm just generating something quite simple. If I kind of go back into doing enterprise level of work and interacting with database, that's no

way. You can five You can't five code it. And I don't know why. Maybe the whole society, maybe especially in the US, there are lots of companies in San Francisco, you know, Silicon Valley as we all know innovation is going crazy. They are just trying to make it sound like, "Oh, this is, you know, we're going to take away jobs and creating a sense of fear maybe

among people." And then the rest of the companies are responding, "Oh, we better do something to so that we are not losing money." And we kind of show the world that we can do better without you know, without being so expensive by not hiring people, then we can we can be ahead because we get value valuation higher, something, I think. So, anyway, but my thing I feel

is that what I tell the junior people who come to my meet meet up and and their concern I'll say, "You know, just continue." And I think there's a lot of valid reason why as developers we need to keep these skills going and there's so much you can also say there's also a lot of fun too, you know, that how I got into programming is because when

I was younger, right? When I was going to universities because I really enjoy coding. There's also the joy in doing coding is a good good for your mind and of course we get into working. We need to do things faster, but that's when we can use AI code assistant to help us to achieve results faster in certain cases, right? But it's not going to replace us is

what I think. So, anyway, but that's just what I want to share too. I feel that we are, you know, while we're talking about tech stuff we need to be aware of some of these other things too, I think. these are resources, but I I'm going to show some code too, but before I go I kind of share with you this slide deck first and so otherwise

sometimes if I show code and then folks started leaving and maybe you won't get to it. So So, okay. Yeah. Okay, cool. Yeah. And if you miss it don't worry because or I'll also show some of the thing, but there are also some links too, but I again will show some more stuff and I also by the way have a live streaming Twitch too on Twitch and

then on YouTube as well. So, I suggest if you're interested you feel free to join or even if you can join it live I'll try to schedule it so the timing would work for your time zone because I'm based in Chicago, but let's see what to do because if you're doing it live if you respond back, right? I can actually invite you to join my stream if

you want to, you know, kind of like speak too. And I I have done it before. When I was at IBM I I did it and it's kind of fun too. There are people talking. I said, "Why don't you come on to my stream?" People will be chatting and for a while too. And and it's kind of fun because we're doing live coding and trying different things

out. So, yeah, so that's that. And then there's also the the link of all of my stuff. Although, I I don't know why there isn't that one uh slide, but I will then go back to uh showing some code at this point, too. Anyway, you can find me, too, on LinkedIn of these things. So, so over here, I just again, you know, I have to say my

examples isn't so much about the multi um multi-agentic uh pattern. Oh, I actually forgot to talk about one thing before I get here, of course, yeah. Because I talk about the I think I remember I skipped this one thing, which is uh about orchestrator. Uh just one more thing before I get to that. It's basically agentic orchestrator design patterns, too. Yeah, because I just talked about the

agent design pattern, but orchestrator design pattern. So, orchestrator again, you know, it's a dealing more with multi-agents kind of scenario, and it's basically there essentially like four things I just quickly kind of talk about it. Orchestrator worker. So, in this particular case then, it essentially having a central agent that would decomposes a task and delegates these subtasks to specialized worker agents, and then it aggregates and validate

results, too. And of course, it's easier to set than done, you know, it's it's but the pattern is how it works, you know, orchestrator worker kind of pattern, a bit more like the vertical type of architecture. And then there are also hierarchical agent, too. They These are organizing layers, too. So, it it depending on your example, you know, of your uh um your uh system, your application,

I should say. So, you can have like, for example, a manager, a specialist, and ex- executor, something. And they will enable abstraction, delegation, and error handling across different levels, too. So, these are like using hierarchical type of agents, too. And then there's also one is called blackboard. It's basically agents will contribute to and react from a shared blackboard kind of workspace. We have separate workspaces. Basically, it's

a bit more loosely driven, a bit more like the event-driven type of collaboration I talked about yesterday, too. It's they can have a workspace and then talk about what needs to be done and you can use a messaging kind of infrastructure to help too for messaging like pops up type of architecture. So, that's kind of what it is. And then also another one is interesting is called

market base. It's a bit more competitive because you are allowing agents. They are autonomous participants in this system and they will actually be negotiating and bidding and competing for tasks too. So, that's kind of an an interesting kind of concept in here as well. So, okay. So, that was the one thing I forgot about that before I go to the example. So, in my example here less

about all of these patterns yet because I do have to admit I didn't have enough time. I I came from This is a actually a new talk and then I became from Frankfurt too. I was in the another conference called J Con. It's a Java conference and then I was doing mostly my event-driven talk, but I also was developing this and then I was running out of

time. I have to admit because I was kind of travel a lot of travel for me past few weeks. So, pardon me, but I wanted to quickly show to this particular example. Okay, let me see. Why is it doesn't Okay. So, over here, but this is the the read me. I I will publish my GitHub short shortly too, but I just haven't check in yet, but For

some reason it's not uh Yeah, dependent Oops. Up. There There we go. Okay. So, basically I I just have a simple example. It's just event-driven alert intelligence demo. So, right now, can you see okay then or maybe I can make it a little bigger. You can see okay, right The the fonts? Okay. Okay. Okay. Yeah. All right. Okay, good. So, okay. So, this one too again, I

just wanted to experiment, right? Like event-driven with agents. And this one again, not [clears throat] like too many agents in in this particular case, but essentially using an alert sender. So, less of calling it an agent, but you can call it an agent. So, this one will be the producer that actually will send out information. I'll act as um uh let's say a problem or a reporter,

like a feedback reporter that that comes comes in and publish some request. And then I actually will then have an analyzer. The So, the analyzer is the one that interacts with LLM is a AI enrichment service. And then also I have a notification service. So, that's another is basically will notify whoever needs to to get the notification, too. And so, essentially that's just how simple it is.

And basically it it is a Java 25 project, event-driven demo using Spring Boot services, Kafka, Pub/Sub, and a local Ollama model in this particular And so, the thing is because I for demo purpose I didn't I don't make it too complicated. So, I'm using local Ollama. If you're familiar with Ollama is local, you can run your model locally. And then also with Kafka in this particular case

I'm using Kafka Kafdrop. And that one is also like a I have Docker image and then can bring it up Kafka local, too. So, but essentially too, if I run it is basically I can do Docker compose and bring up my Kafka topic and it will listen on 9000. It will be the the one that's, you know, receiving messages. And then on my Ollama side I can

just start up my Ollama serve and then pull in the Llama 3 in this particular case. And then I can then run Now, again there are three components to this. I have a alert sender. So, I have a Spring Boot app that publishes messages and these are just text messages and they will describe a problem. For example, like in this particular case, I can test it. I

call it water leak, you know, some is found in a building in certain building B and all that and near whatever like that. So, in this particular case, I'm using then I can actually use an alert and analyzer that will actually work with LLM because it's natural language is a spoken language, so it can analyze as an LLM to figure out oh, what is the complaint in

here? So, then it will then come back to me and tell me, "Okay, there's water leak." So, then it should know to notify the facility you know, coordinator so they can look at the problem. So, it's just very simple. And then the notification service is the one that is another Spring Boot app that will actually be running the notification to send to who needs to be. So,

so that's kind of just how simple it is to and the alert center basically send the messages to this topic called alert raw and then the analyzer again consumes this event, right? And analyze it and figure out what it is and basically using Ollama interact with my LLM and then the model will will then return, you know, invalid JSON or if the analyzer fails to fails back

all of these things too. And essentially too, the notification service will then consume this alert and rich you enrich the messages and then get sent to the center whoever needs to be notified. So, that's kind of just simple very simple kind of cases, but it makes yourself event-driven kind of techniques too. So, we'll kind of just really quickly look into this example. The First of all, I'm

I talked about why is it not here? Okay, here. The sender. Yeah, can you Actually, I I want to make it a little bigger. I think there's some some ways of view. Let me make it a little bigger. How can I make it bigger? Um this is IntelliJ, but I thought I could just number Huh? Command plus. Plus, right? That's what I thought. That's what I was

doing. But for some reason, it's not doing. But I just thought I should make it a little bigger for you. But I'm sorry if you can't see it too good. But I will share with this I'll I'll be checking in. So, over here, but I can quickly kind of do it, too. I have my Java file um that basically you can look at a tell Yeah, again,

I'm so sorry. I wanted to try to make it bigger. But anyway, so it's a Spring Boot app and is again just standard alert sender application. And then I have my model that basically my topic. I I named it raw alert.raw. And then the raw alert is basically the another record that just kind of you know, have information about the event ID, the correlation ID, what is

it created at, and source system message, and reported by some kind of basic thing. And I look at that, right? And then the request that comes in. The request will have the source system, the message, and the report by. and then the alert provider in here, the producer, I should say. So, it's a Kafka template and I basically will produce and then essentially just get this Kafka

template and I publish this and publishes to the topic called raw. And then it gives the alert ID and the the information and the controller. Same in here. It is basically too. It's very simple, right? It just have that message like that. And so so this is my sender. And then over here is my uh Okay, sorry. And example here, I have my analyzer. So, my I

have my analyzer in here and then I also have my service, too. So, service in here as you can see it's just have an enrich alert that will analyze it. So, this in this particular case I have my client that takes in the prompt and you are an alert triage resistant and read the raw alert and return only valid JSON from these exact fields, right? All of

these fields are severity, category, location, crucial info, audience recommendations. Severity, which can be must be one of low, medium, high, and critical. Critical info and audience must be a raise of strings and keep category uppercase with underscore and do not wrap the the JSON in mark markdown. That's what my message is. And basically over here to the this one and is basically, let me see, over here.

And and then yeah, so some of these are just the different helper messages in here, too. But, I also then will kind of identify the severity based on the type, you know, the text if it's contained. I'm identified, too, if if the text has electrical or fire that's severe. If there's leak and water, they're high, but it's not, you know, it's a just an incident and like

that and you know, so on so forth. So, so again, this is just my um you know, kind of my my service element in here. And then I have my model, too, and the messaging and the controller and the config, too. So, my Kafka, too, will already been in here that I have configured my topic alert raw, but there's also alert enrich is after I analyze the

the input, then I basically determine, you know, from my LLM that what the problem is and then it will then generate that notification over to send to the consumer, too, at that point. So, so that's what that's what the this this works and I do realize I'm running out of time, but I do have to say I haven't got time get to the point of actually uh

run it yet. So, but I again will share the GitHub so then you can all get to it, but I just want to point out just three simple services and then of course there's also the notification service in here as well that you can see as again these are Spring Boot application. I have notification service that uh handles the routing part um in here and the subscriber

registry, for example. It is true. Sometimes when we do Java, we tend to have a lot of classes and handle different things, right? The types of um all of the messages, but it's very organized, too, you know, I I do have to say. So, there's also the enrich alert consumer, that type of stuff. So, okay. So, over here, too, if you look at your Docker um kind

of uh example, the YAML file that will just start up with my broker on here on listening at 1992 and uh so all of these things and uh yeah. So, so that's what it is and so you can actually basically try to run it and let me kind of go go back to my readme file. Where is it? My readme? Nope. Uh set up here. Yeah, so

if you kind of do the readme, then you should be able to then run it and then start up your Olama and then uh basically then have your alert sender send run run that to send your messages and use the alert analyzer to analyze the input and then use the notification to basically notify your those who are interested in it. And if you need to test it,

just use a curl command to send the messages over here and see what happens. And so, that's essentially this particular example making use of uh event notification and and uh Kafka type type of topic to illustrate how it would work, too. And it's not fully like all agent driven, right? In some sense, but still we are combining different technology together to kind of do what we need

to do. So, they they um the LM part to help to analyze things and we still use Kafka topic to handle the events. So, so that's the idea. But again, from here I will I'll put it onto the slide deck that particularly the GitHub once I check it in then then you can take a look at the code and we can talk about it too and my

information is there if you want if you're interested yeah, contact me and we can maybe do a twitch stream together something like that. So, yeah. Okay. So, yes and I know it's time is up too. So, but thank you very much. I do appreciate you attending my call and my calls my presentation that he sometimes is doing too much of virtual thing. So, thank you so much

and I appreciate and please let me know what you'd like to hear more and what I can help you with in in your work too. So, thank you. Yeah, thank you. Yeah. >> [music]