About this talk
This talk introduces the Eclipse Elmos project and its focus on Agent Definition Language (ADL) as a means to enhance the reliability of AI agents. The speakers, Arun Joseph and Pat, discuss the challenges faced by enterprises when scaling AI agent systems, particularly issues of reliability and compounding entropy. They explain that ADL serves as a critical framework for defining and executing agent behaviors, making it easier for users without programming experience to create specifications that directly represent their needs. The session highlights the importance of a structured approach to agent behavior that can adapt to various technological environments while addressing reliability concerns. Key features of ADL include its agnostic framework, state management capabilities, and the ability to define skills in a way that ensures compliance with business requirements.
Full transcript
[music] >> Hi. Uh hello everyone. Loud and clear? Awesome. So, we'll talk about uh an Eclipse project, which is referred to as Elmos. And uh we will specifically talk about something referred to as agent definition language. the topic of the talk is around making AI agents reliable. That's a That's the problem statement we are addressing with Elmos ADL. About us, Pat's is [snorts] also going to come
here, but myself Arun Joseph uh we conceived Eclipse Elmos when I was engineering lead for Deutsche Telekom. Pat was a colleague and uh we built the first version of Elmos while at uh Right now, uh I'm the co-founder and CEO of Exozones Systems. What we do, we uh automate operations uh for regulated industries. Uh autonomous auto- autonomous operational infrastructure is what we built. We call it decision
computing. Uh there is a lot to talk about it, but essentially the problem statement that we address is reliability. And Patrick is going to come on stage later, but I would uh have him introduce himself later, but he's a lead for the Elmos ADL project, which is agent definition language project. He's currently working at uh Deutsche Telekom engineer builder and uh and a good friend. >> [laughter]
>> So, we're going to talk about Eclipse Elmos. There is a little bit of history. I would skip all that part to the ADL part. We built Eclipse Elmos as the first agent D platform, one of the first agent D platforms that went live in 2023. Uh in a multi-country multi-country pass as a multi-country pass platform. So, what is the problem that we try to address with
uh uh Elmos? Thank you, sir. Two things, reliability, which is what we refer to as is the agent doing what you have specified it to do. The second is a scaling problem. For example, if you have a if you're a large enterprise like insurance or a telecommunication company, if you have hundreds of agents, how do you know how do you not have the compounding entropy loop from
having the agentic systems collapse? Um we are one of the original contributors or writers of the reliable AI manifesto along with uh Eric Meyer, Jonas Bohner, and and a few other things. I'll show you the our reliable AI manifesto later. And we will come back to ADL. the core topic around ADL, you can think about it as the kernel around which uh you can think of it
like Docker and Elmo's is kind of the Kubernetes version. So, ADL is a core which allows you to have the reliability problem solved for business agents. Um I just put up the slide up. So, there are two problems to scale agents reliably. I would call it as uh just like the Turing test, we came up with uh Carl Jung test. It's uh there's this famous quote from
Carl Jung. you are what you do and not what you say you will do, which is the case with most enterprise or agentic system. So, the test is true if if what you have specified for your agent is what it actually does is a good definition for loosely reliability. We have mathematical uh there mathematical formulations for reliability in the company that we're actually building. But, this is
a good mental model to have. Or even this week, what you specify is what you get. This is item potency. How do you build systems which allows you to have this uh is the reliability definition. And the second problem is scalability. And uh I would use this principle of subsidiarity, which also comes from uh European Union. Uh the core topic is if you're the people who are
closer to the job should be able to specify what they want instead of, you know, moving higher up the ladder somewhere. For example, if you are an insurance agent, a human working in the insurance claims department, and you know better what are the logic conditions around which to approve an insurance claim. If you have to convert that into specification requirements, project managers, and then programmer sitting somewhere,
then already there is a lot which is lost in translation. So, how do you collapse the gap between people people who know the job, who might be experts in what they do, and run them reliably? If you solve these problems, you can scale agents I This graph is based on I cannot share the customer graph, but this is loosely based on what we observed with one of
the deployments that we had in the US, uh Accu Sort Systems. So, this graph talks Something interesting happens when you deploy AI in operations. So, you might appear like when you start chatbots or even assistive agents, let's take the example of insurance claims processing. You might deploy an assistive agent to the human operator. How do you measure the ROI for it? You will start to observe that
after a period of time if the accuracy is around 60%, 80%, 90%, the work actually increases is what we started to observe. Because the human is always checking, is unsure of what is correct and what is incorrect, which leads to this uh compounding loop of mistrust, which which results in processing time. Uh this graph talks about on the left side the processing time to process a single
claim. It was around 22 minutes. And even if the agent gets to a 90% you're actually increasing the time because sometimes uh you're double-checking uh what is correct, what is incorrect, etc. And this is one of the uh hidden a few things, but this is from an actual deployment that we started to observe while solving the reliability problem because we have built of our entire company around
around one metric, which is what we refer to as this reliability rate. This is an example of regulated operations of medical reports which are being extracted and using medical judgment you're coming up with whether it is ready for insurance We're observing a 98% reliability rate and its direct first order effect on the number of processing which a human operator can do and the direct impact on the
right side. What you see on the right side is the the number of claims a human can process with manual scaling or even assistive scaling versus autonomous semi-autonomous scaling. So, uh the the the point around this is reliability is extremely critical if you want to ever have any any ROI on scaling AI agents. So, the second problem is on scaling. Uh in enterprises one agent, two agent,
it works. So, there's an interesting interesting thing to note around uh the microservices era. Some of you would have heard of this thing called as Death Star uh architecture micro Amazon, Netflix championed microservices, but a lot of organizations who went down that chain resulted, let's just say, not so optimal results. But at least in the microservices world people used to get results before the complexity exploded. So,
we came up with term. What is the agentic era architecture? We call it the Frankenstein architecture because at least in this architecture, what happens is you start to build many agents. You don't even have any outcomes. You end up with a Frankenstein-like monster, which is not doing what it is supposed to do. So, in a large enterprise, this this actually compounds in an exponential curve of entropy,
which is extremely critical. If you don't stop you'll never even get the ROI unlike the So, these are the two problems we are as a company, and there is a lot of things which are contributing back to open source as well, some of these elements. And I would have Pat here to talk about ADL and do a deep dive. Thank you. >> Okay, hello everybody. Can I
get a thumbs up that my mic's on? Very good, very good. So, my name's Pat, and I'm happy to come here today to go through the nitty-gritty of what ADL is. So, ADL, simply put, is kind of like an extension to markdown, allowing us to define agentic behavior and agentic systems. And today we're going to be focusing on ADL skills. So, you may have heard of agent
skills. This is very similar to that, and that we are defining agentic behavior, but I like to say we have more features, maybe. Let's just go through a simple example. So, here in this skill, we have the customer wanting to know weather location. So, we start off by describing the situation in which this skill should be triggered, right? So, the customer wants to know the current weather
of their area. Now, we have a block defining the solution, where we provide some hints to the LLM on how it is to achieve this or solve this issue. And then we have a context block. Now, the idea behind the context block is that we define additional information that the LLM can use to just have a better quality. It's not vital to the execution of the skill,
but it just provides a bit of extra quality with the customer, for example, has any follow-up questions. And then finally here we have examples and these are now example utterances that tell us when this skill should be triggered. So this is particularly helpful when we're using something like semantic similarity. So let's just give you a little taste. So as you can see here, we have defined structure.
We are decoupled from any framework and ideally also from kind of like any LLM. So we want to be both framework and LLM agnostic with this kind of like approach. And the whole thing is also meant to be future-proof. I will get into some examples of why we consider this to be the case. But an important, I think, question we always ask ourselves about ADL is is
this kind of like a prompt? Cuz when people ask me, I say, "Well, it's similar to prompting, but we don't really want it to be prompting. Prompting is difficult. It's time-consuming. It takes us hours and hours and having to apply different patterns. Ideally within my skill, I am defining my requirements on how a particular situation should be resolved by the LLM." But okay. Before we dive deeper
into the features, let's just take a step back and look at like the how the whole thing is meant to work, right? So it's quite simple, really. On the left-hand side, we have our ADL files. These could be markdown files that are stored in your local file system. They could be stored in a database. They don't even have to be markdown. But this is where we define
all this kind of like logic. Now on the right-hand side, we have all the different clients that want to access the context that we have put within these ADLs. Now in the middle, this is the crucial part. This is our ADL engine. And it has kind of like two main goals. On the one goal, it provides endpoints allowing the clients to access that data. And this could
be HTTP, it could be REST, it could be GraphQL, it could be MCP. Um and in the middle, the agent facilitates these endpoints. But the more important part is that the ADL engine is meant to generate the prompts from the elements. Not only generate them, but also optimize them for the calling client and the LLM model running that client. So, that's why I don't really like saying
it's prompting, because I just want you to find your requirements on the left, have the engine generate the prompt in the middle, and facilitate that to the And this is important, because this is how we want to achieve not only our reliability, but also accelerate our agentic development. Okay. So, to highlight this, to underline this, I have this great example, the must command. So, this is a
syntax, it's part of the language, and the story is quite simple, really. Our business team were working on their ADL, and they got stuck, right? So, the nice thing is our business team, not our prompt engineers or engineers, our business team also have access to this, and can write their own ADLs. But, anyway, they came to me and they said it's broken. So, I took a look,
and yes, you know, that the bot was working pretty well, but there were certain commands that it just wouldn't fulfill that were vital for the business use So, after like an hour, I gave up, and over the weekend, I had this idea, well, let's simplify this a little bit. So, I came up with this construct that you can define your really important commands in a way that
our engine can pick them up, extract them, and then run secondary LLMs to enforce that they have been fulfilled. So, this case here, we have a password reset. The customer wants to reset their password, we give them some instructions, but it's super important that we inform the customer that when they reset their password, this can take several hours. And as you can see with this magic must
command, written in capital, the engine will now extract this and run the secondary LLM, and if this information was not a part of the original response, it will simply add this information to the response, and therefore ensure that the rule I have defined here is actually implemented. I gave this back to the business team. They came back in a week later and said, "This is amazing. This
is how we save hours a week." And I think this was one of the points where we realized we're actually doing something pretty cool But not only does this ensure the reliability that Arun was talking about, but it's also future-proof. Because even if we need these secondary LLM calls today, tomorrow we might not. And that's okay, because the ADL we've defined here will work either way. So,
that's the must command. So, I'm just going to go through some features now, just to highlight it before we get to an actual demo, just so you get a feel of like the scope and all the things you can do here. Tools is obviously a great topic, right? So, we love tools. We have a lot of tools, and we have a lot of ADLs. The problem is
at some point this gets really difficult to manage. And our recommendation thus is that if you have a skill that requires a tool, just explicitly have that reference defined, so that we can validate that the tools you need are available. We can validate also that the tools get called, but the right tools get called. And we can also optimize the context. Because the LLM will only be
given those tools referenced in the skill that has been I hope that makes sense. We'll have maybe an example on this later. But the syntax is very easy. We have an at symbol, we have the tool name, and two round brackets. We can extend this with an exclamation mark, which will enforce the tool Cuz this is the problem we had a lot of times, that the LLM
just simply chooses not to call the tool. And but sometimes we need it to call the tool, right? Ah, I can see some heads there, yes. Yes, it did. So, by doing this, we have again a secondary validation process that ensures that this tool gets called. And then finally, you can add static parameters. So, as you can see here, there's a lot of fun we can have
with this syntax. And also, this is a format that you can pass. Anyone of us could write a script to allow us to pass these tools so we can run our own validation on the output. So, that's tools. Ah. Templating. So, we've all learned over the last I mean I run on me I've been doing this for a few years now. We've learned the best kind of
templates or the best kind of prompts. You know, I said it may not be prompts. Um are those that are generated run time and tailored for the specific client and for specific user making the call. So, ADL needs templating. And one part we have this mustache. You can't see that. This mustache syntax here where we can inject like data usually provided as a request parameter. Um but
we also have like our simplified conditionals as we call it, which are these things here in the pointy brackets. And they act kind of like it's if statements. So, as you can see here, when the customer wants to buy a new car, if he is an elite customer, we are asking the LLM to call a different tool being the get elite car deals tool. Otherwise, they use
the standard normal customer get car deals tool. So, as you can see here, we're combining now two constructs within the ADL to really enforce that kind of um desired outcome that we want. Now, coming up to triggers. Triggers I thought was an important thing to mention because for the most part we may have a request response kind of pattern going on. But the ADL is not restricted
to this. So, we can have your a trigger section where we define either a time-based trigger, an event-based trigger, or even a goal-based execution. So, you can imagine if we have a chatbot and the customer gets really frustrated, the LLM can pick this up, generate an event, and then trigger this skill in order to manage that customer. Um yeah, and the same with the goal-based. So, we
can have the skill running until the goal has been achieved, and only then will we stop running this skill. So, we're not limited, I think, to the request-response pattern. Now, we come to state. So, for the most part, we can be stateless, right? You can just send a call and you get a response. But, the high adding state to the ADL engine, we'd get a lot of
features that we can build on top of So, for example, here we can track each skill that gets triggered throughout a conversation. So, your customer comes in and says hello, that triggers the skill greeting. They ask for the weather, the skill weather. And then he says something about other questions like the capital of England, which we don't support, which defaults to our off-topic skill, which simply says,
"I'm sorry, I can't handle that." But, as you can see here on the uh that's your left, my right. Um we have you kind of like a state machine, where we can track these skills. And if we see that the off-topic skill has been triggered over and over and over again, then we can see something's not working, and we can then fire an event that again can
trigger another skill, like I don't know, hand over to an agent or some other kind of logic. So, that's why I think skill or state adds this a new dimension to the whole Okay, so I think finally we get to see in code. So, I'm going to show you a demo. Uh but if you want to try this at home, um the best way to do it
is to go to our internet page, to go to the GitHub, to go through all the GitHub code, to create new PRs, really cool features, get them merged, and then run it locally. But, if you don't have time for all that, you can just start a Docker image. So, it's hosted on the GitHub Docker image, so you'd have to log in with your GitHub login credentials. But,
once you start the engine, you'll be able to do exactly what I'm showing you now. Also, the examples I'm showing you now are also available on GitHub. Okay, so let's take here for example, you think it's cool, nice engine, nice logic. So, now you want to simply put this into your application. So, as I said, there's different ways you can do this, depending on how much of
the features that you want. Yeah. Too small? Uh we can't do much about it, I guess. It's okay. Okay, we're going to have to bear with me. I'll describe it. Um so, let's say you want to put this in your LangGraph application, for example. What's Can I have highlight this? Is that better? So, basically, all you need to do is you can you reuse the Open AI
and simply attach yourself to the engine in that form. So, this way, you don't even have to add any ADL libraries, any anything. You just call it the same way you would call ChatGPT. So, as you can see up here, this is where just so you know I'm not tricking you, this is where we are defining the local host, cuz I've already started my engine. And let's
see here. So, what we're doing is we're saying hello. We're saying hello. We're saying hello again, because we're super friendly. And then we're asking what the weather like is in Berlin. So, let's run this. It's running. Okay, so let's take a look at what we got here. So, hello Pat. So, this is a variable I injected within the request. As you can see, this quite a strange
that we have both answers are pretty much identical, which is uncommon, I think, for an LLM. So, we'll get a look at that for a second. And then finally, we give the the weather in Berlin. Just so you know, this is not the actual weather. We're mocking this as an MCP Um okay, but before we take a look at ADL, let's just add a session. As I
said, you know, we were we were stateless. Now, let's be stateful and do the same thing with a session. So, the conversation has now changed a bit, right? So, initially it was repeating itself, but now in the second it's changed, right? It's it's it's kind of like it's kind of step further. Again, we have the weather in Berlin. Again, it's 22°. But there's something has changed and
this is, you know, kind of like why we like this this this statefulness. So, let's take a look at the the skill that was triggered. So, the skill that was triggered is the greeting skill, obviously. And we can see here on the one hand side the the username. So, this is the our templating I mentioned. A feature I didn't mention before was this quotation. So, this quotation
means that we are sending back as a response the exact string defined within these quotation marks. Now, this is not necessarily a feature you will always use, but if for example, within a greeting you have any kind of like I don't know, disclaimers or any kind of like like rules that you have to define or present to the customer before they continue, then this is a nice
way of doing it because you don't have to play around with your LLM ensuring that it doesn't say something silly. It will always return this exact response if this event or skill has been triggered. But that's going to get boring if you ask it two or three or four times. So, then we have here an alternative solution. So, the alternative solution is nice because it means now
that we're tracking the state, we know we tried the initial This somehow didn't work because we're coming back again to the skill. So, now in this case the alternative solution gets triggered. Now, what happens technically is the LLM will only ever get to see one solution So, in the first iteration, in the first turn of the conversation, we have here this solution here. But, in the second
turn, we present to the LLM only these set of instructions. And again, this comes back to reliability, right? We're not confusing the LLM with a whole bunch of instructions it's to follow. We're feeding it exactly what it needs to know for that conversational status. So, again, this is probably the easiest way to integrate your ADL within your your system. Um again, I said there's like MCP tools
that allow you to pick out certain features, but this is the only way you get to like that full spectrum of features because, as you can see, we can add not only the parameters, but also this kind of like stateful session. Okay, so that was the code part, but I would be amiss without having a look at our ADL studio. So, everything you build nowadays has to
have a nice UI, otherwise, what's the point? in the same Docker engine you created, we have kind of like this UI here, and you can see here, these are all the skills that I've defined. Uh cool thing is I can come now here in the search, and I can say something like hi, and then I get an indication of what skills were selected for this input. So,
this is obviously semantic similarity that we're doing here, and what we usually do is we give the LLM, I think, like the top five or six of these skills so that it can make the final decision which skill actually gets And again, with the scalability, this means we can have hundreds, 500, 600, 700, thousands of ADLs, and because we're using semantic similarity, we can find those skills
that are applicable to the conversation fairly quickly. Uh that's not the one I wanted to show. Where's the one with the weather? Otherwise, when you're forging your ADLs, right? We have here this kind of like section where we have three columns. One is where we define the skill. The middle one is where we can define tests that we can then use to test the skill. And on
the right-hand side we have kind of like the performance column, which allows you to get a gauge the performance of your And then we also have some AI AI suggestions, which will help you maybe improve kind of like your uh your skills. And one cool thing I like about this is cuz we have the syntax with the tool, I can just write at and then I get
a list of all the available tools that are available on the MCPs yeah, the MCP servers that I've integrated into the application. So, a nice bit of syntax highlighting, a nice way to to test it, and to really harden your skills. Then you can come here into the chat Oh, we obviously need a chat interface. One thing I like to highlight here is the fact that when
we do chat, we get here the exact skill that was given to the So, this is the processed one. And we also get kind of like a compliance score. So, because we have our skills, we're pretty confident we know how the LLM should behave. So, whatever answer gets generated by our LLM, we can compare that with our skill to verify that it's being compliant. Okay. So, again,
if you have any ideas on how to improve this or any features you all like, please it's all open source, it's all free to download and to mess around with. And I think with that, I am finished with my demo. Yeah, we can I rule will take us home. >> No, no, we can stay You can stay here. Yeah, because we will continue the So, how it
ties back to the two problem statements, right? This is the perfect segue into it. One was the reliability to some degree because uh autonomous operations require a larger engine than what we see here, which is what we're building. But but in terms of how ADL facilitates the reliability problem, right? So, as you can see, there are a few interesting design decisions which were made. In the code
that uh Pat showed, the import statements were from LangChain. Also, there are some LangChain imports in there. So, ADL itself is framework agnostic. It does not take into consideration whatever tooling that would come up today or in the future. It is a certain for you to specify the exact behavior you would need agnostic to programming language. That's the key. Do we have all the constructs today uh
to for all the uh universe of uh constructs which is required for reliability? Not yet. But it provides a fair engine as a language and a specification. Along execution engine which is decoupled from the programming languages, which allows it to be used in any enterprise or your applications and plug it directly. This is solving the reliability problem without without ripping off your existing implementations. How does it
actually solve the other problem, which is uh scaling problem? This is the most interesting aspect, Pat showed uh uh the the ADL file. Right? Where is the ADL file? Where was it you were showing? I can click here. Yeah. Right. So, the ADL file. Who is supposed to write the ADL file? Is it the Kotlin engineer? Is it the Python engineer? No. We want The original idea
is the person who's closer to the job should write this file. You can build your interfaces however you would want without having to know about programming languages, prompting, uh babysitting the prompts, writing test cases. None of this should be known or exposed to the actual poor person sitting in front of uh a call center or processing a claim. They should be able to specify. That's the way
you can scale it. So, Para also mentioned about 500, 600, how many ADL files you can write. So, because we have this interesting segregation into a unit of computation, which is ADL somebody processing uh health care related claim answering in an insurance company can write that particular ADL It should not interfere with a motor vehicle uh reimbursement uh claim file, which might be another within the same
insurance company. With this way, none of that is actually interfering with each other. The engineering teams in these insurance companies or whomsoever could build smaller units which directly maps to their microservices architecture and integrate this. So, it doesn't compound into this uh puddle of uncertain, unreliable Frankenstein monster, which is what is happening. So, these is These are the which we believe is why either ADL or somebody
has to build it. This is what we believe. Otherwise, you'll end up in in this puddle of uh uncertain behavior. So, with that, there was a So, where do we see it actually evolve? Some of the design decisions we made ever since we started this project Elmo's was there and within Elmo's there are certain core elements there was the Arc Kotlin framework which was there. Then we
evolved into a place where we said we need whatever we need to build should be agnostic to two things. One is a model curve either the model is going to get or the tooling curve there might be more frameworks, more tooling, what is going to come out. But should it change to the actual person or business the specification? No, right? So the business process is not going
to change and there should be a neutral way for you to specify this and execute it reliably because the underlying engine can change because one of the examples what Pat mentioned the must command or so or alternate solution it might involve today in the engine to call the LLM for verification but tomorrow we would actually change it. This is actually we are doing in Excel where we
are building neuro symbolic systems which has a different engine which compiles it into a symbolic system and executes it with not with zero LLM calls. Uh So there are ways in which we can evolve the space irrespective of whether the model curve is moving or tooling curve is moving what will not change is specification like this and what best place to put a neutral specification than Eclipse
Foundation which is also the reason why we are incubating it right here. So with that help contribute and help shape ADL. I just pasted a an email which I received yesterday from one of the students in Karlsruhe Institute of Technology KIT. Uh none of the enterprises have picked up ADL because we are not good in marketing it and uh potentially uh so this student was actually doing
an assignment in a hackathon in Munich and he wanted a structured way to do lead qualification. And this student in KIT was able to pick up ADL and wrote, "This is exactly the problem we are solving and uh we like the way ADL actually solves it." He wrote this email yesterday. Just wanted to mention that. It's not the big uh we're not LangGraph, we're not LangChain, we're
not the grassroots is actually potentially picking it up. So, there is a way in which we can evolve this as a as a true neutral specification layer with underlying compilation engine. For those of you who are interested in compilers, interpreters, traversal engines, ASTs, it is a fascinating project for you to come and contribute. That's all uh we'll have to say. All right. Thank you. >> [music]
