About this talk
In this talk, Hari Rangarajan, a developer advocate at Solace, explores the challenges and opportunities surrounding AI agentic systems and event-driven architecture (EDA). He discusses the hype surrounding AI and the limitations of current large-scale production deployments, emphasizing the importance of successful integration and real-time data. The speaker explains how AI agents function like autonomous microservices, highlighting their need for memory, tool usage, and collaboration among multiple agents. He outlines various architectural patterns for implementing multi-agent systems and demonstrates how EDA can facilitate communication and scalability. Finally, he introduces the Solace Agent Mesh as a robust platform that enables seamless integration of agents, allowing for real-time data processing and enhanced enterprise capabilities.
Full transcript
Good morning. Can everybody hear me? Wonderful. Uh this is the first time for me doing it in a silent disco style. Uh I have been to silent discos when I was younger, but I this is really new and this is something quite fun. So, uh thanks for being in this session and my name is Hari Rangarajan. I'm a developer advocate at Solace. Uh that's my LinkedIn QR
code, so please feel free to uh reach out to me and connect me uh in case you want to talk anything about EDA, about Solace, about AI, anything, the weather, anything that you want, right? So, uh in the last 2 years, we have had a lot of uh hype, lot of news, lot of concerns about AI, uh AI and LLMs and agent AI agentic systems. Uh they
are having an impact on our lives, uh how they are going to take away our jobs, how they are going to uh uh impact the world, right? Uh Yes, definitely that's true. They have a They are going to have a big impact on our lives, on our world, but I think it's it's also a certain level of over hype within this whole um messaging, right? Uh and
the reason why I think that is uh with all the hype, with all the news that AI is doing or AI is generating, um there are not a lot of successful large scale or even medium scale enterprise grade production deployments or production use cases, right? There are not a lot of these success stories. Yes, some people are some lot of companies are using for coding, uh but
beyond that, on a productionized level, uh there are not that many use cases. And the reason is most companies, when they do these POCs, they all go well, but when they start deploying to production with all that enterprise grade standards and enterprise grade non-functional requirements, uh they start failing, right? And they start failing or they don't come up to expectations, uh and then you sort of lose
interest in that whole conversation of AI. Uh today, what I'm going to be doing is I'm going to be talking about uh some of these challenges and how event-driven architecture can help you address these challenges. Okay? what are agents? Agents are essentially small pieces of software which can uh reason, which can plan, which can uh use external tools, and uh rather than just generating haikus or poems
or songs on ChatGPT or like normal chatbots, right? Um and they operate with minimal human intervention. They can talk to tools, they can talk to other applications, they can query, they can do deep researches, etc., right? Um what I want to do is um I'm old, reasonably old, yeah. Um and I want to draw a line or at least kind of a parallel between uh AI agents
and what we had in like 5 years ago, the microservices. So, for me, um AI agents are intelligence are to intelligence what microservices are to business code or business logic. Yeah? Um both microservices and AI agents, they rely on contracts, they rely on autonomy, and they rely on these events to be completely functional. Thirdly, agents are essentially microservices, but with reasoning, reflection, and adapt adaptability built into
their capabilities, right? And lastly, where microservices enhanced business capabilities, AI agents, they enhance your cognitive capability for your business processes and your systems. So, what makes a system actually agentic? There are four factors which they should cover for an system to be AI platform to be called as agentic platforms. So, the first one is autonomy. You kind of say that, "Okay, this is where I am. This
is who I am. This is what I want to do. These are the guardrails that I'm setting for you. Go do it." Right? And it does it for you. It does it autonomously to a certain extent. Secondly, uh tool uses. They are tools are kind of hands for these agents, right? So, they can do API calls, they can do MCP calls, uh they can do web research,
web searches, etc. right? So, these are tools that are really essential for a system to be agentic. Thirdly, uh memory. AI agents or agentic systems need to have a memory around there, right? So, in terms of agents, they need to know who you are, your profile, uh your tone, your way of working, uh what are the expectations from your previous sessions, uh what are the previous things
or the previous uh activities that you have done with that agent. So, they need to know all of this. They maintain all of this memory in their context and they use it as a starting point instead of going from square zero, right? >> [snorts] >> And lastly, multi-agent. So, 1 + 1 is Somebody has two. Exactly. But, um when you talk about multi-agent platforms, there's this that
logic doesn't apply anymore. So, when you build an agentic platform, you want to have multiple agents who are capable of doing multiple things. And what we have found out is or what we see is the sum of these or the total output of these multiple agents is more than what they can deliver on their own, right? And because we have multiple agents coming in, it also means
that we need to have some integration layers or how do you say, at least architecture patterns that need to be used for these agents, right? So, the first one would be Does it work? No. So, the first one would be a single agent loop, which is They are simple. They have They They walk through a simple path, right? So, they plan. Sorry, they they understand the context
of the of the of the prompt or the use case. They plan. They act. And they reflect based upon your feedback on how did they go, right? And it's really simple, easy to And they really work well for simple, easy to you do use cases, but they fail or they don't scale out to enterprise use cases or large-scale workflows. Secondly, an hierarchical multi-agent multi-agent setup where um
you can have one or one's top-layer agent, which delegates agents to the sub-agents or a sub-orchestrator below it. So, the top-level agent is always called as an orchestrator. Uh it can the the the sub-agents can even delegate them to down under downstream agents if they have something like that. Uh and they all work together to form or they complete complex business use cases, complex workflows, etc. But,
there is a complexity in terms of error handling, governance that really needs to be handled in these situations, right? Or in this pattern. Thirdly, a peer-to-peer mesh where they act as nodes in kind of a network, right? And uh they communicate through shared infrastructure or events-based infrastructure. they they have like good robustness and scalability. this is where EDA or even-driven architecture really shines where they sort of
drive the communication uh between these agents in a completely transparent manner, right? And lastly, you have the workflow constrained uh patterns where you have agents, but that autonomy is quite limited or restricted to specific things that they can do. And where do they work really well is in regulated industries like payments, uh like defense, right? Uh you do not want agents going off off the rails in
these use cases. Uh you really want them to do one thing and really cut be within that limit, and that's where they come in, right? And these are really helpful, but they are predictable but less flexible, which is fine for these regulated industries. why traditional agent designs fail at scale, right? And this is what we want to answer or at least I want to try and answer
for you. So, what do most enterprises want from their agentic platforms or agentic solutions? They want Sorry? I'll come to that in a second. So, what they want is something like They want drag enabled. They want a multimodal multimodal. Uh they want something that is continuously self-optimizing that can uh that is that has all the enterprise data, but also knows what's outside in the world. It's uh
GDPR compliant, HIPAA compliant, it's enterprise guardrail compliant. Uh it's globally distributed. It's simple but uh complex but simple to explain. It should be free. It should uh obviously bring its own return on investment. It should also bring me my own lunch and dinner, of course. And it should be quick. It should be deployable by Friday and completely running on a Monday. And basically it's something on the
lines of it's a chat GPT, but just made for us. It should understand us completely. It should be proactive, but it should not be annoying. Something like this, right? This is a nice expectation. We are I think what, 30 years from something like this. Yeah, most of us will be retired by then. But this is what a lot of enterprises they try to expect from their AI
systems. We know that this is not going to happen for the next 30 years, but this is their expectations, right? But coming back to the serious topic. A lot of analyst companies like IDC, Gartner, a lot of consulting companies like Deloitte, they have done quite some research and they identified that implementing a proper enterprise grade agentic solution is two parts. 20% is actually the AI part or
the LLMs, the large language models, the agentic platforms, the frameworks, etc. 80% of it, the the big iceberg beneath the ocean that actually sinks the Titanic, is integration. Making sure that your applications have the right data at the right time. They have enough information to give you meaningful and contextual information or contextual responses. And your agents, if you think about it, they're only as clever as your
data that you pass on to them, right? So if you throw at it, you are going to get out of it, right? And that's where event-driven architecture really helps in building these platforms. And it's not really helping, but it's actually essential for these the platforms, right? So, what are some challenges to actually adopting agentic solutions? First one, no real-time data. So, without real-time data, your agents are
going to be stuck using stale or outdated data. That means that they can hallucinate. And they don't just hallucinate, they hallucinate with confidence, right? Because they don't have enough context or they don't have the right data in their context, right? Secondly, oh, before I do that, right? So, 42% of enterprises, um, when they start implementing AI solutions or agentic solutions, they find that they need around eight
plus data sources or streams into their solutions or into their platform to actually make meaningful, uh, output of your investment, right? Secondly, uh, your enterprise data that has been gathered over the last 30 years, it exists in ERPs like Salesforce and, uh, SAP. They exist in your CRMs, they exist in your databases, in your APIs, and even older companies in your confluence pages and Jira tickets and
in your Excel sheets. None of them are ready for actually, uh, helping you or actually AI ready right now or agentic ready right now. And most of these systems, they respond to each other by using just simple rest-based API calls or request-response patterns, right? Vendor lock-ins. So, enterprises, large enterprises or distributed enterprises, now what they do is, I have heard, I have seen use cases where, uh,
companies are using LangChain, are using, uh, Azure, AWS, and Microsoft, uh, sorry, and Google, uh, LLMs all at the same time, right? And why? Because each of these models they are for different subtype or a subtask. They have their own capabilities and specializations. And none of them talk to each other. None of them want to talk to each other. And why would they want to talk to
each other, right? Similarly, all your agentic platform development development platforms like LangChain, Open Crew, Crew AI, N8N, they all build these agents, but none of them talk natively to each other. But you want them to talk to each other because that's where the multi-agent platform comes in, right? And how do you do that? You have to build custom glue code within these between these agents. And that
is something that you'll be stuck maintaining, handling, and securing for the next 5 years, Your reliability. So, when an agent sort of is overloaded by requests or it sort of downstream it calls your downstream APIs and MCP servers too many times, one of those is going to be failing because you can't simply handle that kind of a load. And that means one agent that is failing is
kind of going to be blocking your whole chain. And that means a bad user experience. It means a bad customer experience. And that means that your loss of confidence in your solution, right? And more importantly, once thing goes down, you have to maintain it. You have to scale it up to come up to the next level, right? And this is going to happen. Why? Because a runaway
agent you can't control it without actually monitoring it 24/7 or actually understanding it what's happening. And you have to control that, right? And then it's difficult to do. Lastly, with agent with your whole agentic solutions, you have more and more attack surfaces that are coming up like memory poisoning, context your your your delegation of your access and responsibility authorizations, right? Things like those. And your non-deterministic behavior
is proving difficult for your traditional security standards and security governance, right? And this is where you need to build up security and governance as first-class citizens of your agentic solutions. You can't think of it as something, yeah, we will get to it. We'll just do this POC and see how it goes. No, you have to build it up from the bottom, right? How does EDA solve your
challenges? Let's first take a step back. What are events and what is event-driven architecture, right? So, events are all around us. So, anything that is a change in the state of a system, a business process, an application, your lives, they are all events, right? And they kind of Any change can be categorized in the real world as an event. Now, you can build applications which can talk
to each other by publishing these events and consuming these events. And that's what event-driven architecture is all about. So, one of the most common ways of implementing is a publish-subscribe where an agent or an application produces one event. It just forgets about who is consuming it. It doesn't know who who is consuming it. More importantly, it doesn't want to know who is consuming those events, right? That's
the true decoupling that event-driven architecture brings for you. what if So, how do agents communicate with each other? Google came out with a nice protocol called as the agent-to-agent protocol or the A2A protocol. It kind of defines what information an agent should give out so that it can interact with each other and how can these engines interact It specifies that HTTP is kind of the or SSCs
they are the most common transport mode, but it doesn't actually talk about a specific transport mode for these conversations, right? So, what if we could build a solution where these agents are talking to each other using events, right? So, an applica- So, just think about it in your real life. You ask ChatGPT a prompt. It goes to a central orchestrator. That orchestrator sends out an event to
a sub-agent or an sub-orchestrator, which takes that event, consumes it, understands what to do, processes it, and pushes back a response as a new agent back Sorry, new event back to the broker, right? And the orchestrator gets that, it collates cleans it up, and sends it back to the user, right? As a response. This is how you can build out completely decoupled agents using events, right? Publish
and subscribe patterns. If you had, let's say, 10 agents, right? Because we said that we want to have a multi-agent infrastructure. Um let's say that you had 10 agents. And in a multi-agent infrastructure, you want all those agents to be talking to each other or at least be capable of talking to each other, right? So, when that happens, you need to run You need to understand how
many connections you would need between those agents. There's a nice formula which says n into n minus 1 by 2. So, which is for 10 agents, you would run into 45 connections. If you flip that out with events, each of those agents would be connecting to a central broker, which can publish and subscribe events to it. And that means you are just connecting 10 agents with 10
connections. Bi-directional duplex connections. If you want to add a new agents, you can do that without any impact on the existing ones, right? Agents, they consume real-time event streams. They can understand what's in your enterprise data. They can understand changes in your system, changes in your APIs, changes in your databases, and they can even react to those changes. So, for example, if your let's say shipping price
changes for a product, right? It can send out notifications to your customer saying, "Okay, we have a low shipping price for this product. Uh would you like to order something?" Uh or we can let's say there is a new war somewhere in the in in the world, right? And that's impacting your oil prices. You can actually have an event trigger for that, and you can say, "Okay,
analyze the impact of this war and this oil price change, and see how it's going to impact my business and my products, right?" all these events, you can actually um kind of instrument them with OpenTelemetry. So, that means that um all your changes about the events that are triggering these agents, uh the the decisions, why are they being taken and how are they being taken, the the
decision rules, um the agent to agent conversations underlying them, all of this can be understood as or actually is visualized as uh OpenTelemetry events, uh OpenTelemetry traces, and you can actually build visualization tool to understand what exactly is happening in your agentic platforms. Fault tolerance, of course, uh let's say that you have a runaway agent that's hallucinating or it's making too many API calls. All or it's
an downstream MCP server is failing or an API is failing or a database is getting overloaded or it's responding slow, whatever. You can identify these. Um you can have error handling where you can push these events to dead message queues or dead letter queues or whatever for retries, uh exception handling. You can even actually understand that an agent is hallucinating or it's running wild and you can
stop it or terminate it based upon some rate limiting or throttling of these events, right? what is the Solace Agent Mesh? Um so, the Solace Agent Mesh is a platform that the that comes from Solace. Solace is our I'm from Solace, of course. And Solace is an event broker event-driven integration platform. We have been around for 20 years and um I think we are there in any
of the domains that you can think of. We work in lot and lot of places. So, if you drink beer or if you have checked in and checked out at NS or if you fly by plane to Asia, South Africa, North America, South America, anywhere in the world, it's a good chance that Solace is involved in these touch points, right? So, think about that. Solace Agent Mesh?
So, all those capabilities that I described about agents being talking to each other using the events, we took all those theory and we implemented it as a product called as a Solace Agent Mesh. It's an open-source platform where these agents can communicate to each other using events, but more importantly, it's a platform where you can build your agents in wherever you want, right? You can build in
Azure Foundry, you can build it in AWS Bedrock, you can build it in open open crew lang lang chains, N8N, doesn't matter. As long as they are A2A compliant, you can integrate with the Solace Agent Mesh and you can build them together and you can make them talk to each other, I have this one slide that's literally going to be the almost end of my presentation. Um
so, I just want to talk about what's the capabilities of the the the center is the event broker, of course, where you see the That's the That's the messaging infrastructure, but as a AI developer, as an AI platform, you will never see it, right? You don't have to know about it. Comes in the entry points. So, the the thing that you have an AI solution, you need
to be able to access it. So, you can have a web UI, you can have a Teams integration, a Slack integration, an event-based integration, anything that can trigger the Solace Agent or the the That's an entry point, right? Uh the entry point is kind of the the dynamic and flexible brain. It's backed by an LLM, and it understands or it has a visualization on all the agents
that are deployed in your ecosystem, in your whole enterprise. Doesn't matter where, just as long as it's introduced into the Solace Agent Mesh, the orchestrator knows about it, right? So, it knows about the capabilities, the skills, the tools, what it can do, and what it cannot do, etc. Then, native agents. So, these are agents that you can actually build uh as long as uh using Python, Go,
whatever technology you want, as long as it's an A2A compliant agent, we are good with it, right? And these are like first-class citizens of your solution or your platform. Any AI solution needs to have LLM backing it. We can do that as We do that as well. And uh this is where our kind of vendor neutrality comes in, where we say you can use any any LLM,
any model of your choice. You can even have multiple agents spread across multiple models for a planning, for a general purpose, low-cost models. You can define what your agent wants to do uh uh and we'll just work with it, right? Then, connectors. So, I mentioned that your enterprises have data in a lot of ecosystems, right? So, ERPs, your CRMs, your files, etc. Those are the connectors that
bring that information back into the event mesh or into the agent mesh or the platform itself. Your agents that you are building on anything other than the Solace Agent you just include them or you have a you have a nice proxy that you can bring in and introduce them plug in into the Solace Agent Mesh project, right? And it doesn't matter what protocol, what technology you're building
or what framework you're building them on, as long as it is A 2 A compliant, it works for us. We wrap a nice small proxy around it so that those agents can talk as events within the Solace Agent Mesh. Outside, they can do whatever they want, right? Then, tools. Of course, you do data analysis, artifact management, and things like those. You can enhance them, you can add
in new tools as you want. Your data management, of course, in terms of context monitoring, context management, compression, etc. We have a native functionality that takes care of all of that. You can even visualize the number of your percentage of context and clean it if you feel that it's approaching thresholds. You don't have to do that, but we do that for you automatically. But you if you're
an advanced user, you can even do that. You can run evals on your agents to understand how they are working. Are they working with the same confidence that you have them or you want them to? So, you can do those. You can uh What was that? Yeah. You can have a visualization of what are the different to task No. what are the different steps that an uh
orchestrator or the sub-agent is taking? You can identify why it's taking and why not something if something else is going to supposed to happen. And you can use this for auditing and actually tracing and explaining it to auditors on why a decision was made or not made and what is the impact of it. You can build agents in low-code and no-code manners, of course. And lastly, uh
you can deploy this uh the solution as an Kubernetes solution uh in Kubernetes on cloud, on prem, whatever matters, whatever works for you guys, right? So, this is a complete Solace Agent Mesh platform. Uh it's an end-to-end production-grade, production-ready platform which helps you build your agents uh from POCs and take them to production, right? Uh we have quite some active customers. For example, Douwe Egberts, the Jacob
Douwe Egberts, the coffee become one of the largest coffee companies uh in the world. Uh they are using the Solace Agent Mesh for running their internal uh AI solutions and AI platforms. And they are like publicly referenceable customers of us as well. a minute. Any questions? No? Okay. But before you go off, right? Uh I have a small request for you. This is the Solace Agent Mesh
uh GitHub repository. Can I ask you to uh scan the QR code and when you have some time, just give it us give us a star. This will help us inform you about the new features that are coming in, about the the the product updates, and uh more importantly, it helps us spread the word about the about this product. Thank you very much. hopefully, you have you
found something useful. And I think you should have a uh great conference ahead. Thank you.
