Harnessing Event-Driven and Multi-Agentic Approaches for Efficient AI Data Flows - Mary Grygleski

This talk explores the potential of event-driven multi-agent systems in the context of artificial intelligence. The speaker discusses the current state of AI agents and highlights the limitations of large language models, such as their lack of memory and statefulness. Focusing on the need for a robust agentic architecture, the session addresses challenges in deploying AI agents for complex enterprise applications, emphasizing scalability, resilience, and task execution. It also covers various frameworks and tools for building these systems, including generative AI, retrieval-augmented generation, and event-driven architectures such as Apache Kafka. Additionally, the speaker touches on ethical considerations and the importance of security in AI implementations.

[music] >> My talk is about more event-driven multi-agent. It's more exploration. As we all know right now with agent AI agents, it's still in a very early stage. I mean, there are plenty of things going on happening now. But maybe before we start, can I ask how many of you are already with AI agents coding and doing all that? Wow, it's actually most of you. So, are

you like mostly doing like using some sort of framework? Is is that correct? Yeah, you're doing that, right? Yeah. Are you also like say using event-driven type of techniques or doing something with it as well? Yeah, okay, very cool. How many of you are doing like say Java? For example, Java and then they're non-Java, right? I'm asking because yeah. Okay, so maybe because too, I understand this

conference too probably started off a bit more for the Java side, I think. So, I'm just assuming still carries with it some kind of Java flavor. So, I'm asking. But then as such with agents with you know, machine learning AI seems to be mostly Python, I believe. Python or JavaScript, right? Or any other languages. So, so okay, so I'm going to start. So, this is my agenda.

I'll do a quick introduction and then do some demystifying of GenAI and agentic architecture. And then then get into like event-driven world in AI. And I I do feel is less has less been talked about, I think from an event-driven kind of perspective. Primarily because right now I think with AI agents, we mostly I feel is still we are working with maybe like single, so to speak,

like single agent or doing some task that's kind of achieving one goal that's kind of maybe broken up into multi-steps. And then and then it goes through, you know, figuring out what to do and then achieving a goal and doing all the execution. But then it as far as us, you know, going into the next step of generative AI, applying it for complex enterprise type of applications,

then no doubt too we're going to get into for example like transactional type of problems which which actually will require more of computing type of techniques in involved too. So, There's still folks coming in. Thank you. Okay. Anyway, so then I will then get into distributed computing because that's the area that's actually quite new. I think it's still being worked on in by the whole industry as

such the whole AI all these things are happening so fast. I'm sure, you know, it is it's just going to take some time to become a bit more stabilizing too so to speak. And so now then I'll in you know, get into rag and agents workflows all these things and some challenges too. Uh Okay, so first of all I'll just quick introduction of myself and thank you

to you already did some. So I won't go kind of go through a lot. It's just a picture is worth a thousand words. I'm currently I'm helping out the AI collective. How many of you I'm curious too are have heard of AI collective or member or attending any of the events here in Bangalore for example? No? Not yet? Yeah. Okay, well, just a quick thing. So AI

collective is is a grassroot organization. Right now we're glowing growing really fast like globally too. So it's not just you know, not just in the US. It started off in San Francisco but it's also not only technology focused. It's not just for builders but it's as such with AI we're as we can see right AI now kind of attracting attention of everybody non-technical folks too. So the

group too itself is focusing also for non-programmers too. So they are also like investors, um, business people, or founders, too. Startup founders, a lot of them. And so, it started off as in San Francisco, and uh, now it's spreading all throughout the world. So, I'm like with the Western Hemisphere, all of the Americas, but I also travel. So, I also talk to other chapters, too. We have

chapters all over the world. Here in Bangalore, I believe we're very big, too. And tomorrow, or actually Saturday, there's going to be a a talk, uh, an event, uh, at Harness. If you If you're interested, let me know. I can share with you this link on Luma that you can register, and this will be on DevOps, this particular meetup. So, so that's that. And then, um, if

you're interested, you're from other areas of your country, I think we also have it in Mumbai and New Delhi, too. But they're also we're intending to expand, further expand, too. And really like helping not just with builders, you know, doing some single task thing, it's mainly expanding and kind of really, bringing out the human, the human side of working with AI. So, it's kind of more than

just coding. There are more more things about AI. So, anyway, so that's the group I'm with. I'm the VP now, and I'm also helped to manage the Chicago chapter, too. And then, there are other things that just were pictures worth a thousand words. I'm also Java Champion and also a Oracle ACE, and uh, I'm a Actually, I started off as a developer 25-plus years as a software

engineer, too. And I came here, too, uh, back then when I was with IBM. Maybe some of you have come to my talk in the past, too. I was talking more about event streaming, event approach things, reactive systems. So, it naturally naturally brings me, when I start dealing with AI, I also look at it from that perspective, too. Anyway, so so that's about me, and my really

big interest, though, is kind of into distributed systems. That's kind of my area. But I also like, you know, that distributed systems very applicable for AI is what I'm looking at. So, Okay, so just really quickly, too, like introduction, demystifying GenAI and agentic architecture. As we all know, right? AI, yeah, it is can do a lot of things, but as builders ourselves, developers, this conference is for

most of you builders, we are kind of really looking at data. So data is really what is making AI happen, making, you know, machine learning, all of these things. And as such, GenAI is becoming the things because it's the first time like NLP, natural language processing, like I have like both both things too, but this is simplified kind of high-level description of AI, machine learning, and deep

learning. But as you look inside, right? There's the GenAI kind of within the deep learning side and large language models. And as such, large language models too is really the essentially it's it's what it it is like for a agent to be working with as well. But it's largely enabled by NLP, natural language processing, because as we all know, now you can interact with AI and talking,

you know, kind of basically the command is human a human communication way rather than in the old ways. So as traditional developers, we have to follow certain rules, you have to send in your input a certain way, otherwise, you know, then your program won't understand. But now with GenAI is really revolutionary because you can talk to it like talking to another person. So So that's that, but

again, these are high-level stuff too. Basically, we are all producer, but they are also the consumer side too that we're using tools to kind of further our goals. But as builders, we have different ways of looking at things and how do we leverage, you know, agents, for example, and and like leveraging like autonomous uh way of doing things, you know, how much, you know, is it going

to take away our jobs? I think that actually is even more important issue that we are all thinking of too, Okay, so this I won't need to kind of get into all the details. There are generative models that are kind of like power all the engines, you know, underneath this whole phenomenon. Um and then of course there are platforms and apps and there are also like Amazon,

Azure from Microsoft, or Google. These are more like optimized type of AI platform. And then of course there are tons of other, you know, if I have to lay out all those apps that are in the world that can't even fill this screen, you know, there's just so many things. But let's kind of get back into it a little bit about agents. I why agents kind of

become the thing is because there are limitations to an LLM, large language models. As we all know, you know, large language models are essentially they're just algorithms to kind of do pattern matching. I mean if you strictly speaking it's not really search, you know, we always say okay search for your answers, but it's not the traditional way of doing search. It's really um again it is you

know, about super super way super fast way of doing pattern matching. And the input to what you wanted is through the prompts. And so it has to decipher what you need all of these things. So that's one of the large language model's strength is. But the thing is though it has also a lot of limitations. There's no memory with it. It's stateless as we all know. So

like right now we can say oh you know, we can use chat GPT, we can talk to Claude all of these things. Yes, it's because there are tools, libraries, and programs behind the scenes that help you to form a context as you are asking question. But otherwise by itself if you work with an LLM directly it's stateless. No memory. Also the results too are highly like probability

base rather than being deterministic. So that's actually the problem as why too why why as to the current, you know, gen AI this whole AI seems powerful, but actually as you kind of look around a lot of especially major companies that deal with financial data they're not using it yet because of this non-deterministic aspect, This thing of this current state of the AI is powerful but just

not, you know, giving you deterministic answers. You can ask the same questions many times, 10 times, 100 times. Every times it come back with slightly different answers because of this, you know, kind of like very fuzzy logic type of approach to things. And also, too, like the answers it comes back, you can't really say add any attributions to it, too. It's opaque, this one is called. Not

only that, it's very expensive. If you kind of think deeper, too, all the data centers now are working hard to try to, if you send a prompt in, it actually uses a lot of processing kind of going over it, too. So, very expensive, right? And all of these things as we know. And again, it's a one-shot deal. But the thing is so, I mean, don't get me

wrong. I think as we all know right now, AI agents, you can go do stuff. And for example, how many of you actually using Open Claw? I'm just curious, too. Yeah, Open Claw, anybody heard of it? Okay, just a quick thing. That's That's kind of becoming the thing. I don't I don't know in here if it is getting the same kind of attention. But over to where

I am in Chicago, too, and I remember somebody says Open Claw in February. Then all of a sudden, everybody is doing Open Claw. But the thing is so, Open Claw is actually just a personal assistant. And you can essentially set up like personal tasks. And through maybe through your Slack, through your WhatsApp, through some kind of messaging, you can actually set up all these commands and using

Open Claw the interface, and it will do things for you. They're essentially agents, AI agents, and does things. Now, that's the thing. It becomes very dangerous because there are no guardrails. It's basically you can open up your own system, open up all the ports, whatever it is, and it can send email on your behalf, do different business transaction. It is very powerful, but it's just that very

dangerous, too. You know, if you don't know about anything, your your personal data that be stolen, all of these privacy things. So, it's also at the same time very dangerous, very all these things. But anyway, I'm just trying to kind of bring this up. And as such, it is powerful, but also this security aspect we need to pay attention to. Also, AI ethical AI, too, right? What

you're going to expose is going to be you know, the data will become like not secure. You can have other people getting to it, all of these things. And not only that, right? If we are a lot of us, too, I'm assume you're working also with companies that are There are actually systems that are enterprise level. Meaning that you're, you know, dealing with financial data, important company

data, all of these things. And, you know, all those things, too, tends to be, right? Enterprise systems require kind of responsiveness. Also, it needs to handle maybe very scalable kind of situation. Maybe like an order entry system. Actually, I don't even think there's like one agent can handle, but, you know, kind of one like ordering flight, you know, if you want to say, "I want to fly,

you know, here from Bangalore to Delhi. I want to make one want the agent to make reservation for me." If you want it a little bit more complex, I don't think it can truly do it yet. Maybe simple stuff you can say, "Okay, just go find me a flight." You can. But if you say, "Manage the whole thing." Because that's the purpose of agent is to help

you to do something, you know, on that level, like project manage essentially. But it's just cannot handle it yet, right? The the way this the agents are. There's no true AGI, and artificial general intelligence yet, which is to deal with reasoning. The reasoning part is missing. So, anyway, so going back, scalability is an issue. Complex tasks, there are workflows that, for example, order entry I mentioned about.

You will be ordering flights, and the flight reservation is not just with the flight, with, you know, with the with the airline. You are dealing maybe with a system that's kind of aggregating all of the flights information, and you need to make reservation. Is it the you know, the is the price correct? And do you have enough money? So, it needs to go into your bank account

and check if you have money and all these other things that you need to bring into concern if you need to make, you know, make your agent being able to do a a flight reservation from end to end. That can That kind of stuff that the way it is going is not quite, you know, safe yet. A lot of things. And not only that, too. With agents,

too, one aspect is the memory. So, memory, too, is to keep all the contact all of these things. I think they're still very complex at this point. It just By by itself, LLM is not able to do it. So, that's why, too, we're relying on agent to be there. Because as such, you know, if if you know, the the definition of agent is basically it has a

goal and then it will need tools, you know, the different things you need to rely on to kind of make reservation, for example, and then achieve that goal. Another thing, too, with agent is that it also needs to know how to handle if things go wrong, you know, as we all know, things can't always be right. Like you have to be prepared for occasions in which there

are exceptions, that type of stuff. So, so, with all that said, and that's what it is. So, we get into like agentic architecture. Let's take a look at it. So, the structure and design of AI systems that leverages AI agents to perform tasks needs to have like it needs to be able to do autonomous kind of execution. So, some of the things you don't even need to

supposedly explicitly tell it what to do, you know, how to execute, what to do, all of these things. And also the tool integration part that needs to also rely on tools in the back end, too. Um and also the the seamless scalability, as well as I also mentioned about it also should have the resiliency and recoverability to it, too. So, these are kind of the the high-level

kind of architectural requirement for agentic So, some more key concepts, too, is basically with the architecture, you can have vertical architecture in which is like one agent is being the leader of deciding what to do. Think of it just like a human run project so I will work without AI. You can have vertical architecture or you can have horizontal architecture in which you can have multiple agents

too, each taking care of specific task, but they are all collaborating with one another, which leads to the fact that we may need an orchestrator too, like a agentic orchestration in which you know, agents are doing different things, but you need another overseeing agent to coordinate making sure the sequence, the steps, you know, are in line with what is supposed to be. And you know, you need

to have this performed first before you can do the next thing. That type of scenario. and then there also essentially too, agentic architecture also needs to handle complex task execution and also that autonomous again autonomous decision making is like the reasoning, the planning part too, but the reasoning part I think will still get some ways again. AGI isn't isn't really ready yet as we all know or

there's also new terms they call it ASI artificial super intelligence too. All these are not quite ready as we all Um but they can do certain things very well. Don't get me wrong. Something repeatable that type of stuff. So but ultimately too for agents to one also property important regardless of whether you are using AI or not AI is the fact that agents needs to be adaptable

meaning that needs to continuously learn. You know, it needs to to be continuously being trained to. It needs to have that kind of a property. And ultimately too is basically we don't want human intervention essentially having the agents take do things from beginning to end. So if you think of agents right with the with without AI that concern is basically can be like a travel agent like

a insurance agent. As we all know, we do things. Why do we need agent? Because this will eliminate some of these um things in which, you know, you will need to make decision if a condition change, you need to make changes, all of these things. So, you need a an agent like a single party to help you to do stuff from end to end. This that's what

it is. So. Okay, so that's that. Let's get into the event-driven world. So, at the heart of AI, again, I mentioned about data. And also not just right not just the data itself, it's not static, we know. Data flows to in many directions. And by that, I really mean is that, you know, let's look look in little further in that. So, on the architectural level, we can

have event-driven type of architecture to design systems. So, essentially too is basically deal deal with, you know, there there can be like, you know, your your data signing systems. It's just like any other event-driven systems, but we can at this point in time, we can kind of look at a system and determine what part of your you know, component of your flow can make use of AI

agents to do things. AI agents will be there to interact with LLMs, for example. But the thing is too, let's look back into the data uh the data itself. So, as we all know, data too can be in different states too. So, it can be at rest, meaning, you know, they are in storage, in a database or in, you know, file systems or whatever it is. It's

a static kind of, you know, at rest state being stored in some device, for example. Or the data itself can be in in trans transit too. So, it's like you know, in a network kind of distributed data kind of scenario. You need to have data maybe selected from a the certain database and then that data needs to be sent over to another system for verification or updates,

all of these things. and they're traveling, you know, between different nodes too. It's in transit. Or data can be in use. It's basically, you know, of course, they're order entry system. You kind of have have to have your data being stored. You initially, you know, create an entry of an of an order and the order too will need to be updated as it goes on. But maybe

because in in the in some physical objects in an order kind of system, maybe like if you check the inventory system, there's there there isn't anything available. Then you may need to update the order and and kind of market as, you know, I'm kind of pending or just waiting for something, whatever the state it is, right? So, that type of stuff is basically the data is in

use, too. And then there are also techniques to deal with like event driven type of systems in which, you know, you we're talking about event streaming. These are like when data being sent at any time. In and here too, I quote example of using Apache Kafka or like Apache Pulsar. I came and talk about Pulsar before. It's an event streaming broker kind of system. It's essentially also

like event logging system, too. You can think of it much like Pulsar is much like Kafka in that sense, but it's a bit more designed for cloud native type of usages. And then there's Red Panda, Solace. All of these, they have event broker kind of functionality. And there's also event messaging. So, we're kind of talking about patterns, event patterns like broker, using a broker in a pub/sub

type of system. If you're familiar with publish/subscribe system, yeah, so in which in this case too, you are separating all the sender of messages and receiver of messages and then using broker to be there to be the broker, essentially distribute these messages accordingly. So, in this kind of like decouple kind of system, you can actually kind of enable your system to be more efficient in terms of

scalability kind of scenario. You can add more brokers to to kind of manage all of your messages if it gets there to be very, very busy, for example. That type of stuff. So, and then there's also one more technique too that's involved. It's called event sourcing and CQRS. I kind of pair them together. I know it's a not doing justice explaining too much because by themselves, they

are topic to that deals with like how do you manage data even sourcing? Let me explain quickly. It's basically if you know you let's kind of use the same order entry kind of example. So normally in traditional a system you tend to be you you kind of create an order then you have one entry of the order in your database and order table and then you have

a column called status table. Status is status is what captures the state of this order at any given point in time. However, if you just keep it like that then what happened if you want to go back in time and see how that order the state has changed over time. So you can't do it if you kind of design it like that one table one column just

indicating. So it will only give you the status at the you know the most recent time. So there's this new technique called event sourcing. So what you do is essentially keep track of the state changes. So you basically have another table that's essentially capturing this the order changes. So whenever there's state changes let's say order comes in as new that's one entry and then it becomes like

somebody you know a clerk kind of processing then you update it to pending or be in process something like that and then you capture that too and then they're basically saying that okay order is waiting for something whatever it is then so on and so forth and then it might the whole order can be manipulated in any given point in time. So it's that's what is described

as event sourcing because you're like capturing the state changes. So the good thing about that technique is that you can always go back in time. Let's say you know for example I work for Chicago Mercantile Exchange as a options to futures trading. So those places they are very sensitive right? You need to be able to track if something somebody enters an order everything is in millisecond. You

want to kind of figure out well how come this order doesn't come? You need to kind of clarify you can always go back in time using this technique to kind of say, "Okay, yeah, between 3:00 and 3:15, what's happening?" Then I can kind of look at the table, look at the the records, you know, for that particular order during that time. Then you can really minute, very

minute detail in granular level, you can see how it changes. So, that's kind of like technique you can work with, too, basically. So, I'm just kind of citing that example because I believe eventually, if we're going to make agents to be more useful, it has to be able to manage things on that level as And then there's also CQRS. I just want to quickly mention that's it

stands for command query responsibility segregation. It's basically a pattern. So, when you're dealing with event sourcing, as you can see, if we need to track the change of state of the order, then there will be many records, right? That any order can, you know, and not only that, you have, let's say, Amazon, you get, you know, million orders all the time. You have many records, too. So,

using CQRS RS is essentially separate out the read and the write. And so, that's important, too, because there are, you know, you have many reads and many writes, it can actually create a lot of load on your processing, too. So, if you separate out read and write, actually, it can make it more efficient, too. So, that's kind of, long story short, and that's what these are. So,

so that said, let me kind of go into talk about next thing, which is uh generative AI kind of streaming usages now. So, and and in a generative AI type of uh input, right? We're talking about, okay, can be chat, uh dialogue, or any kind of um applications in which you need to actually get the prompts, get the the input request. So, the prompts can be on

command line or from the browser, different can be different devices, mobile apps, for example. So, all of these, too, if you kind of think of it, all of the prompts can be is basically being streamed to your apps to running somewhere, maybe on the cloud, or even if you're testing, can be running locally, all these things. And then generally I too, it goes into that intermediary steps.

For example, in a rag application like retrieval augmented generation, that you're using that techniques basically. And so rag operations essentially too will take in sources, data sources, right? Because you're augmenting what the LLM doesn't have. LLM essentially is has a fixed set of data because that's what it's being trained on and usually the data is older because it takes a long time to train a a general

purpose LLM. Then you need to, for example, not not only that, you need up-to-date information. Maybe you can get it from another database with more up-to-date information. But in some cases too, it can be a your your company. If you're working for a law firm, working for a private hospital, then data too has to be kept, you know, within the firewall, all these things. And then you

can use rag, this particular technique, to help you to feed those data in. And so with that too, there there are also like different sources that can actually you can also leverage if you need up-to-date data. That's actually most up-to-date too. You can actually use streaming type of event streaming to get real-time data into your system too. And then that also includes a Kafka topic or message

queues, all of these things as well that you can use for rag too. And then from there, then the rag will then update, you know, take pulling your real-time information or whatever data you feed it in, then it can actually update your prompt, right? Enhance your prompt and then send that over to the LLM. So that's also another kind of streaming type of usages that you can

think of applying, you know, this event approach to it too. And then of course too, then now at this point the LLM is processing the information, then it will eventually need to send the response back to the the requester. In this case is a rag application, for example, then that's also can be think of thought of as using streaming type of way of handling that, >> [snorts]

>> Okay. Um now, let me kind of quickly kind of talk about the benefits of this event-driven approach. It's basically, as we can see, right? The data itself is very dynamic, you know, it never stays in one state, you know, a lot of times, especially with modern-day apps. Um and also the flow pattern, they are very dynamic as well. And also, too, in this case, if you're

using event streams, then it helps with commit the communication in in a more asynchronous fashion, too. Um and so, now it should be able to then address issue with scalability and flexibility because again, you are separated out all of your your you know, your components, they are more loosely organized. They are not tightly coupled. So, they can help better with scalability, especially if you're using pub-sub type

of approach to do things, it will make it even more scalable or easier, too. Then again, you can add brokers to kind of handle if, you know, a messaging situation becoming kind of little bit out of hand. Okay, so then there's also the case you know, agents, too. Maybe like you can have different agents that are handling different aspect of your system that actually you can be,

you know, and because different LLMs are built for different purpose as well, right? And they can be handling text, you handling video kind of information or you know, what are the thing like 3Ds, images, or you can handle code. You know, there LLMs that are specific, like Codex is handling code, for example. So, you can have different agents, too, that designed it so then your agents are

working with different types of models within your larger application, as well. Okay, so you can kind of think of using event-driven approach, but the thing is, too, I want to reminding, you know, of anything that's kind of powerful in any kind of situation, You it you know, there are some challenges too. The challenge is that it can become more difficult to debug, you know, and as such,

you know, event-driven systems tend to be harder because you can have events kind of being sent, you know, anytime. It's it's, you know, not as easy to track that type of stuff. So, but the thing is though is it's not impossible. It just means your observer observability side of things, you need to make sure you build better better ways of tracking and being able to kind of

go back in time if something goes wrong. All this. So, just wanted to point that out. Okay. So, now I get into distributed computing and agentic AI kind of aspect. It's basically I want to quickly kind of talk about this CAP theorem, too. Um I know get I get into a bit of this theory thing, but I think it's still important to kind of understand it, too,

because as as such we're again talking about data and then distributed data. That's the difference because now we're talking about data that's in different network domain as well. You want to make sure if you're handling agentic systems that are complex data, then the data itself flowing through from one, you know, boundary to another, that overall they the you know, you won't get problems with data being out

of sync, so to speak, right? So, okay. So, CAP theorem. It's So, CAP stands stands for consistency, availability, and partitions tolerance. So, basically consistency means that every read, you know, any kind of when you work with data is basically all of the read will you make sure that it comes back being correct, you know, this this from the most recent write, right? Otherwise, there will be an

error. Um but in that case, too, um okay, let me kind of go to the next thing, which is availability. It means that every read, [clears throat] you know, will get back a response, you know, whatever read you need you need some request you send and you want always some response coming back. But it not necessarily the most recent write. So, which means um in this case

goes to the third one, P, which stands for partitions tolerance. What it stands for is basically pa partitions refers to network kind of issues or network is essentially network connection. So, as we all know, you know, we live in a world that things are not perfect. So, sometimes we have network failures and connectivity issues, all these problems. We know it, right? Sometimes we're waiting for things is

like, "Oh, this isn't right. That isn't right." It always have something wrong because of the network and we all know. So, the the theories the theorem basically states that only out of these three aspects, you know, four fundamental aspects of data distributed data, only always only two things are guaranteed, you know, which means, you know, let's say if there's a there there's no network let's let's let's

say, right? Or actually, I should say if, you know, there there are problems with network connection, then what do you choose, right? There there network connections. So, you know that if you go and get fetch the data, then the consistency it won't be good because there's no network connection. So, in the meantime, maybe your data is being updated somewhere, but from your perspective, your app doesn't have

the connection, then what you get back is not going to be the most recent, right? But in that case, too, do you suffer and say that, "Well, I don't really care. I can wait." So, I if I wait, then I know I'll when the network comes back, I'll get the most accurate data. Whereas in some cases, too, you can say, "Well, I don't actually care if I'm

getting back stale data, but I just want availability because it is very important. I need to get a response back." So, that's availability. So, that's kind of basically is explaining that any given point in time, you know, only two can be guaranteed, but of course, you can have all three, but sometimes if there's no network or this slightly glitch, too, you could be getting back stale data,

for example, things like that. So, that's kind of the theories theorem part. So, here, too, like you may be asking, "Well, why is it kind of important to know about these things?" And because well, this is yeah, kind of comes back interesting the color, but anyway, there should be some color, too. Um but basically, too, there are, you know, if you look at all three aspects, and

some you just have to be aware that some databases, they are each kind of vendor, how they design the databases, they basically have to kind of decide on what they want to put their emphasis on, right? So, say for example, um over there is on consistency and availability is basically Oracle, Oracle, MySQL, all of these um the RDBMS is basically they they pay more attention to to

uh consistency, meaning that if there's network problem, it will just wait. So, essentially, you can get latency in that case, right? If there's network problem, then you have to wait and it comes back with it always would come back with the most accurate data, then it has to wait for the network to come back. Now, if you kind of more important to have availability, then uh databases

like DynamoDB, Cassandra, Apache Cassandra, Voldemort, or Riak, or CouchDB, these are more they pay more attention to availability. Even if there's no network connection, I'll still give you back data, but the data could be older, too. And then over there, too, there are also some database like Google Bigtable, MongoDB, these are really kind of heavy like consistency-based. They really will, if there's no network connection, basically it

will just say, "Okay, I won't give you back data because it's more, you know, it's more important to give you back most accurate data." So, the thing is I I'm sharing with you because at some point when you're let's say some of your architects who are in in the in the audience, you need to decide on which database to use for your apps, and that's important to

know about these kind of differences in your particular distributed data apps, too. Okay. So, one more thing just want to point out is about PAC cell. This one is stands for uh partitions, uh availability, consistency, and then else, latency and consistency. So, essentially, too, even without partitions concern is basically saying that you still need to choose between what is more important, consistency or availability. To, you know,

because it also affects on what kind of compromise you need to do. So, let's say, you know, we already look at, you know, the the availability consistency part with partitions, but if there's no worries of partitions, then you have to see, do I care about, you know, low latency type of apps, you know, that type of stuff. Let's say a trading apps, you know, you get that

absolutely don't cannot afford any latency drop in latency, too. But in some cases, if you're dealing with transactions that are dealing with financial data, then probably consistency should come first because you don't want any, you know, kind of transactions to have people's account, you know, being like miscalculated all their transactions, too. That type of stuff. So. Okay. So, then kind of go going back to like GenAI,

how do we reframe GenAI? So, if you kind of look to it's basically for GenAI, the else part will dominate. So, essentially we're saying that user will always expect, you know, we're streaming tokens, all of these things, and we want the tokens to come back much quicker, less than 1 to 2 second. Then you want actually really want like low latency read, too. And essentially, too, if

you kind of have also heard of the term eventual consistency. So, the theory is that even if, you know, you you have connection problem and you're not getting back the most recent or kind of like consistency is suffering, but we believe in eventual consistency because network will always come back. And if you kind of do more reads again, eventually, when the network comes back, you should be

able to get the most recent data, too. So, that type of stuff. So, just want to kind of point that out. Okay. And then here, too, and just again, just quickly kind of talk about CQRS and agentic AI, too. So, CQRS again, is a software architecture pattern. It separates out the data model being used for writing and reading too. That's That's what it is. And it's important

because it impact your data storage requirements as well if you kind of get to that level stuff too. Um so and but the thing is with CQRS should be suiting very well with the dynamic needs of agentic AI systems too. So because AI agentic AI systems too, we should kind of worry about especially when we get to enterprise level you have to worry about scalability, concurrency, performance,

and resiliency that level of concern too. Okay, so now I'll just really quickly then talk about the RAG too. So um I actually already kind of talk about it but just kind of put it into another section and kind of make sure everybody understand too for those of you if you're new. Um it's it's essentially RAG is an architectural pattern too. It's not you know, it doesn't

dictate how you implement things but it's essentially have two models. One is retrieval, is generative. So if I use a diagram to kind of explain it then it probably will worth a thousand words and basically as we all know, right? And again, this I'm a bit simplifying because I'm not kind bring out some important aspect when you're doing RAG too. But let's kind of first look at

it, right? A user can be submitting a prompt to do something to the RAG application. So the the prompt will get sent to the RAG application and then the RAG will then go and do some semantic search for example to a vector database, some external sources and get more context and build up that context. Let's say in this example is a vector store. Again, it can be

replaced by a Kafka topic or whatever external sources. And then from there, then essentially the RAG application will then need to enhance the prompt with this additional context and send it over to the LLM. So then then it will kind of should come back the LLM should then come back with an answer. So, that's kind of like kind of quite, you know, simple too. And then the

the generative model will come in is basically come in at the point when before you send back the response back because it's essentially generative side will generate, you know, kind of beautify, enhance your response, send back to the sender too. That That's what it is. Now, what I didn't kind of talk about here is that it simplifies things because, you know, in some maybe smaller application is

not an issue. You can be reading data from some external sources. But let's say if you are dealing with let's say you work with a hospital. From what I know, I talked with some folks who are doing systems in in which rag application. And let's say you have to go through a lot of you know, kind of data. Let's say from a hospital, you have years of

archive of information. So, they are all, you know, in different, you know, huge amount of of data. But as we all know, LLMs also have limitations. It can only take so many tokens, right? When when it's process processing data. So, you will you will need techniques such as I call chunking. Yeah, that's for example is one way. You chunks meaning that you divide up your input, you

know, into certain kind of chunks. But even with chunking too, it has its own techniques you need to be concerned about because how do you divide up your data, right? Let's say a whole whole bunch of data and you need to divide them up. But you know, how do you overlap each section? Because if you don't do it right, then you may not be getting back the

correct answers in what you're trying to ask for. So, the chunking piece actually become quite important to how you kind of divide up your input to be sent over to a rag application. So, so I said you know, here this talk isn't dealing with that level, but I just want to also point out to there's this concern of dealing with you know, huge amounts of data that

you need to worry about with with chunking. Or you can use techniques such as like filtering to filter out what you don't need to that type of stuff. So, again, just wanted to point out that to you. Okay. And then here too, just a repeat the role of the retrieval model. If you kind of think of it, it's just like a librarian that pulling rows and information

from a database or corpus of documents of these things. And then they will get sent over to the generative model to eventually, right? Because then you then enhance your prompt, send it to the LLM, and LLM will then come back with the results. The result will then pass through the generative model. It will take the retrieved information and then it will kind of that's why too, let's

say if we're trying to ask a chat GPT or ask any kind of model to help us write an essay, it always comes back looks like it writes very well because it's, you know, with with with the way the LLM works, the generative side will actually give you back very very readable, very nice kind of syntactically correct and enhanced type of output, you know, let's say if

you're asking for a text kind of thing. So, okay, so that's that. And then I want to then get into the event approach for agents and workflows too, if we apply to that. So, agents too, it's again, I spoke a little bit about agents earlier, but essentially too, for agents, you know, even in a purest kind of definition, it's basically saying that the one who's authorized to

act for or in place or on behalf of another. So, in this case, you know, AI agents is a computer application, AI application that's designed to automate certain tasks too. And then agentic AI, when we get into that, is basically we we use these the systems that use AI to pursue their goals and then complete the all the task on behalf of the users or and or

another system. And then it should have reasoning and planning as well as memory too. And also has a level of autonomy as well. The autonomy part is what gives like a the edge, right? Of of able to do things um different than traditional way. And because it's supposed to know and know how to make better decisions of what to do, break down the all the task into

different steps, and also learn and adapt too. Um so, it's supposed to be no human involvement, but as such we know that right now I think all the agentic AI is not truly like no no completely no human involvement. I say, you know, we still need some human in the loop there, too. So, again getting into like gen AI agents is basically you know, these are software

entities that can orchestrate complex workflows and also too, it should be able to coordinate activities. Let's say in a in a vertical architecture, you can be coordinate activities of multiple agents too, like that. And it in a horizontal architecture, maybe you can have agents that, you know, it's like by maybe using different protocol of selecting any any leader at any time, much like Kubernetes do also has

this leader protocol too, for example. But essentially too, just having maybe one agent being the leader to coordinate and process the logic and evaluate answers too. So, that's the And then there are also like multi-agentic systems, and that's when we put multiple autonomous agents to work together, and they all each specialize in different tasks too. And some of the, you know, essentially the agent-to-agent interactions too. Um

basically, you want the agents being able to divide up some complex task into manageable subtasks too. And then um also too, in this case you can also leverage the agents to work with different AI models too. So, the the ultimate thing is to be able to increase robustness and and fault tolerance of your system. Then that's that's the And also too, wanting to bring out, right? There

also standardization MCP, for example. I think there are a couple talks in here that talk about model context protocol. This comes from Anthropic. It's basically a open uh open protocol that standardizes how applications will provide context to the LLM. And as such, you know, this kind of satisfy like the agent tool calling side especially. Because then in this case too, we are we're kind of talking about,

you know, agents itself may not be able to do, you know, everything by itself. So you need to leverage on some other tools to help you do it too. So here. So another another protocol is A2A and that's one agent to agent is came from Google originally and now it's kind of actually got Linux Foundation Linux Foundation has adopted the governing of this protocol A2A too. These

are kind of relatively new is roughly, you know, MCP came out first and then A2A came out maybe last June, I believe. So not quite a year too. But it's interesting too that two of them actually they complement each other. So if we want to describe them, MCP is a bit low on the lower level side meaning that it usually it manages resources on the lower level,

the tools, external tools, anytime you need to have agents like interact let's say with a database, with some kind of resource and external sources, whatever, APIs, enterprise apps and with other things and you kind of use MCP. But the thing is for A2A is kind of design is more for agent to agent communication too. So it's a bit higher level between the two. So let's say if

I have been agent doing something and I don't have the answers, I can also use A2A and connect with another agent to kind of get what I want as well. So that's kind of how they kind of they each have their place, you know, in this way of doing things. So here too like I talked a bit more than about event driven and multi-agent for GenAI too.

So just if you have worked with event driven kind of systems, I just want to bring up for example some key components in event driven. It can be event bus, right? So, event bus will basically manage the event streams and there can be API gateway that handles some external events, for example. And agents, too, can be autonomous components. They each have like different capabilities as well. And

then there are also underlying models, too, that that kind of they're generative models that you can be using, too. Like these are so sort of like a couple key components, too. Then you can use events to help coordinate agent activities. Because as such, you know, event-driven system is you act on you act on the things based on some events that happens. And so, some events that happen

will generate certain event type that you can react to, for example, like that, too. So, let's see what other thing. Okay. So, I think to me I feel that there's actually a lot of synergy, I guess, between event-driven and agentic type of you know, if you're using agents type of kind of approach to design your So, you can have different like agent types, for example, on the

right side I kind of give a couple examples like coordinator. Coordinator can be decomposers of the task and then I'm doing agent assignment. And there can be also generator, too. So, it's basically generator agent is those is the things that produce content using AI model. And then there also evaluator, right? They're agents that's called actually eval agent, if you have heard of it. I think folks are

calling it like that, too. It's everything that new things coming out from the AI world. So, evaluator essentially will assess quality and reviews, you know, think of it more like a a reviewer like if it is like a software engineer developing code and QA engineer help to test it. It's sort of like evaluators like QA QA doing that type of work. Not QA quality assurance. So, and

then there also like refiner that helps to improve and iterate through the initial outputs all these things, too. And there are different event types like user requests type, subtask assigned, and result regenerated for example final response. These These are just a few example event types. You can have literally many different types of event types for your particular application accordingly too. Okay. So here is just an example

too. I just thought I'd share and I also have the the the link too. And by the way, I have the link to this slide slide deck if you're interested. So basically too, this it kind of explains and is using autogen which is a Microsoft open source SDK. And this autogen is particularly good for designing and using it to design like chatbot type of application too. But

as you can see, you can kind of look and into different kind of agents it can be built and multi-agent con- conversations and different patterns that you can do. And you can also like have human in the loop kind of kind of patterns that are human in the loop type of interaction in in your design pattern Okay. And then here too, I quickly kind of also talk

a bit too about agentic design patterns. This actually these four things are being highlighted and I took it from Dr. Andrew Ng. He's a well-known kind of researcher and also he has this company called deeplearning.ai. It is for educational thing. If you go there too, you can find a lot of educational materials which I find to be very useful. And he's very knowledgeable. He also has a

lab. Anyway, so I I kind of was following him and followed the newsletter he wrote about these design patterns. These These are new, you know, in terms of design patterns that are catered specifically for AI agents too. So one is reflection, the first one. And then two use and planning and multi-agent collaboration. So we can quickly kind of take a look into this. So reflection, just now

I talked about the type of agents can be eval or critic agent and then a coder agent. So think of coder agent as like soft normal software development engineer. Critic agent is like a QA engineer kind of doing work. So, in terms of, you know, this kind of pattern you can be is basically you can be prompting and ask your coder agent to kind of write some

task for it. So, initially to your coder agent is going to ask the LLM to kind of produce some code based on some instruction. And then the critic agent will then every step it will look into the results that it gets and then will eval do some evaluation assessing is it correct. So, based on that then then it will basically go back to the coder agent and

said, "Well, there's I find some problem. The generated code is incorrect." And so now you want your agent to be autonomous too. The agent needs then be able to know that, "Okay, there's some problem." So, have the coder agent ask the LLM again for another try. So, then it will kind of keep going and same kind of pattern and tell the LLM to improve the code and

then critic agent will evaluate and go back and tell the coder agent something is wrong. Fix it. So, so on and so forth. So, here too eventually too what will happen is that you're supposed to hear from the critic agent that, "Okay, everything is fine now." You know, perfect code and and then you can say it's done. So, this is like a reflection agentic design pattern. And

then there's also like tool use too. So, as such, you know, tool use is basically is you are kind of configuring your agent so then it knows what to fall back on. You know, it's doing something it knows, you know, it can get the problem ask the LLM to do stuff. But let's say it comes back and the results are not satisfactory and something is missing. So,

you you want to configure it so then the agents can get then, let's say, a Wikipedia page or some other, you know, kind of help tooling. And basically too you can use this pattern essentially help the agent to be able to find answers, you know, in times of LLM not able to give you back answers. So, it's relying on external tooling in the back And then this

one, too, is just an example I was using from this SDK called Crew AI. It's Python, too, but Crew AI takes a different approach of agents, too. It's It's more like a role-based type of agent system. So, this I won't get into all the details, but I have links for you. And this one is also actually quite interesting. It's basically a planning type of design pattern. It's

basically we want the agents to be able to figure out the steps of how to solve a problem. So, what you do is you present what you want. So, in this particular example, it's saying that I give I basically give to the LLM and said that okay, this is an image of a boy riding a scooter, but what I actually want the result, the goal is to

try to generate a pic another picture of a little girl in the same kind of position and reading a book. So, in that case, how do you do it? Right? So, you ask the agent to do it. The agent should then be able to go through kind of figuring out the steps that it needs to to do. And then eventually, it should help you generate that picture

that you want. Now, the thing is it's interesting. I also I'm sharing the link in there, but essentially, too, his lab Dr. Ing's lab, they they did this, but they were saying that it's the accuracy isn't very good or at least, right? That was like maybe like 9 months ago when I read the paper. But I think from I believe that at this point, I haven't unfortunately,

I haven't got time to go in and see if they have updated anything, but it's harder to achieve, as you can see, because the agents will then need to figure out what steps are actually needed to It takes a bit of reasoning in there. But however, this is just a goal like how we want the agents to work towards to. Okay, so this one is basically the

pattern is most complicated. It's called multi-agentic collaboration. And so here, it's actually there is already an example in GitHub too that they publish from their labs too. Again, I share this link to it. It's essentially mimicking like an IT the kind of model project development. So, you have different role base and they you can use a raw waterfall models. Have designer doing things and coder and tester

and also documentation of these things too. So, you can also kind of take a look in that link and then they will give you a bit more example. I think there's again there's GitHub example if you are interested in that Okay. So, these are examples of a few multi-agentic libraries like AutoGen I talk about from Microsoft or CrewAI. LangGraph is from LangChain that the that group is

a bit LangGraph is based on more nodes using nodes and edges and like that to a graph theory base. And then there's also AWS as trends. I think James Ward from AWS is here to also doing some talk on that MCP. And agent is SDK from Google. And then on the Java side, there's also one that's Quarkus LangChain 4j that's based on CDI too. So, these are

like different example. And then there are some additional example like CrewAI in GitHub. I have these in here. And also that there's also someone too from the Java community developed some AI workflow. Although I look at it and that was a while back he did that but he hasn't had any update. But it's still it might be kind of interesting for you to look at too. It's

more experimental. >> Okay. So, I think I don't think I have time to actually get into actually you know some kind of demo thing but I'm sharing with you my live stream too that you can follow me if you like to. But first of all, let me kind of point out to any kind of agents and AI agents all of these kind of challenging and it looks

like fun to do, right? We're doing new things emerging technology but I wanted to point out that there are lots of these are complex system design and uh too. Then agent memory, context management, these are not uh simple stuff. And also you have to ensure consistency across like different uh network boundaries too among all the distributed agents. And that you have to handle failures and partial results,

all of these things. And you need to also optimize for latency and resource usage too. And not only that you we have to remember there's a big sustainability issues too of using LLM. Using agents maybe at at any point in time it agents may not be needed too. So you need to also make that kind of uh judgment too when you're deciding how you design your app,

for example. And of course all of the LLMs are still having hallucinations, it's not all clear yet. And not only that there's AI safety and responsibility. I think ethical concerns is something that maybe as developers sometimes we may not think of it right away. But I do want to point out something because here I am doing AI Collective and we have activities with different groups of people.

Again, they're non-programmers too. And I think we should also listen from their point of view. And believe it or not, it opens my eyes cuz I used to be more, you know, with engineers we tend to think more rationally, right? And we do things and we will say that this isn't make a rational uh kind of, you know, that sense or something. So we might not be

paying attention to to that. But I thought this might be good for to share with you is that on International Women's Day we had some um like gathering with with uh different, not just with women, but it's just celebrating Women's Day in March. And then one mom was then citing example of her son, a teenage son, 15-year-old having autism too. So the the son has to take

some medicine medication for his autism. But then the son actually is quite smart kid like 15-year-old and he goes to school and little did they know he was kind of looking at working with ChatGPT as such, right? These days with all of the ChatGPT generate AI being so easy. He was asking the question about his medication. But I think what happened was eventually, you know, whatever he

was interacting with it eventually is the chat GPT came back was telling him, "Do not trust your parents. They don't know what they're doing. They don't know anything." Like that. So, now the funny thing was because he was using that chat GPT at school. So, everything's got tracked. So, thank God there was the teacher somebody staff monitoring of these children and using and that the teacher this

particular staff noticed that, "Oh, wow, this kid is asking this question is not good." Now, so that kind of points out the the problem here is that while logically we can do all these things, but we have to also think about the fact that is it even correct? You know, we know that these models are not all correct. It's hallucinating. Even us too, we may be looking

at things and working with Claude co-working of all of these. Oh, we're going to trust it. I say my my thing is that do do not trust it. Don't always think it is right. It may not be right. So, it's even like with us coding and we have that problem, but even with non-programmers I think they may actually be even more vulnerable. The non-technical people, the children

who are using it, the seniors people who don't know. They say, "Oh my gosh, they are stealing my money or something. Let me give you my credit card number or something, right?" So, just always educate people around us and tell them don't trust it right away. Be just be critical, right? Whatever it tells you gives you back may not be right, you know, look for the best

answer. Maybe ask for a few models for the answers and compare answers before you trust it and ask around it like those things. So, I just kind of want to point out that's even more important to to become aware of to using this technology. okay, I guess it's not going okay. And resources. So, that that's all I have, but this is the slide deck. If you are

interested in this slide deck and then it also has my contact information too and and all of these things in here. So, please keep in touch. If you want to follow me on my live stream, that's where I want to do some demos too on there. So, if you're interested, yeah, I actually do have some example is actually building a Kafka kind of example. Actually, I'm doing

another talk tomorrow too. So, if I time I can actually do that. I'll go a bit more into agent the the design pattern part too. So, and I I'll get that running. I can show you a Kafka example in the afternoon if you're still here. So, thank you. Thank you very much. I hope it is useful for you. And let me know. Thank you. >> [music]