From Automation to Autonomy: AI-Native Workflow Generation with Multi-Agent Systems - Sharath Kumar

This talk explores the transition from traditional automation to AI-native automation within Target, a major retailer in the US. The speaker discusses how Target's automation systems evolved in response to increasing complexities and operational demands, such as curbside pickup and same-day delivery. To enhance efficiency and minimize human error, the company developed a low-code automation tool called Siglang and later implemented a multi-agent system known as Cygnus. The focus of Cygnus is on creating intelligent systems that can adapt to dynamic changes in processes and data, using specialized agents that understand user intent and manage automation tasks effectively. The speaker demonstrates how this approach simplifies user interactions by allowing them to specify desired outcomes instead of detailed procedural steps, ultimately streamlining operations and increasing reliability.

Firstly, thank you everyone uh for your joining us here. As I spent some time in reflecting on our journey and how we moved from traditional automation into, you know, an AI native automation, right? And a step we are thinking it is towards autonomy for us. And before we get into all of this, right? I'm just going to say I'm going to I'm here representing my team uh

which includes a few humans and few AI, right? So, with that, we'll start with a little bit of context about Target. Uh it's one of the leading leading retailers, sorry, in the US. $100 billion, 2,000 plus stores, which also talks about the scale, And it's Target is actually known for its uh stylish alternators, right? Compared to its other retailers. And it also differs in a way we

treat guests, right? Guests are at the core, at the center of what we build, typically. All right? And over time, all right? I think this uh systems have evolved, transformed, and also become complex in a way with all the new features and the demands of uh the current uh I would say operations of uh retail, right? Whether it is curbside pickup, same-day delivery, you know, all of

these are testing actually the stress on the system, right? And we want at this point to be really efficient in a way, all right? From the All right? So, that we can deliver on our promise, right? For instance, uh imagine Target's uh decides to, you know, carry a new product uh and it from its planning to all the way it hitting the right store on the right

aisle takes an army of people. Right? Operating behind the scene making sure everything works. All right? And a small impact or small error can actually cause a large impact. It could could be things like, you know, you have your winter wear which should be delivered to Minnesota going to Florida. This is exactly what we don't need, right? So, our aim has been to reduce the human error

and increase efficiency. And our goal has been to always simplify automation. So, that it's accessible to all. And then we can deliver, I mean accessible to all and they can, you know, automate some of these complex processes, right? So, let's just see just spend a minute going through our automation journey here, right? When we began initially, right? All the automations were built by the developers. Like this

is some time back. But when we started showing the company, right? The efficiency that we can drive there was a lot of interest and people also reached out saying that, "Hey, can we build automations by ourselves?" And these were SMEs. And it would make sense, right? And let them build their automations, right? So, then we started our DIY journey where we actually built a simple DSL called

Siglang which is a signaling language loosely based on um, you know, Groovy. And then we launched, you know, the low-code automation tool. We trained a bunch of interested people and they went back and, you know, built their automation, right? And they were able to again drive a lot of efficiencies. Which was very encouraging, right? But then as we were training some of these folks, we also realized

only about 8 10% of our intended audience knew scripting. Right? So, we thought we have to solve that. So, we went ahead removed syntax from our automation platforms and said you don't need to worry about the syntax. Just drag drop. Right? And you can build your automation. But this is when I think uh you know, our adoption grew manifold. This is also where we started cracks appearing

in our platform. Right? Not in a way of scale, but in a way of how automation traditionally works. Right? So, the automation these people built, right? Worked beautifully for a few few days, few weeks. And right and suddenly we saw we see a flurry of incidents and you know, we have to run into these calls, explain what's happening, triaging. And all the team would get involved in

you know, recovering, rerunning these jobs. Right? um the team that was supposed to build automations and features is now doing triaging and all that, right? It's counterproductive. So, to solve this we actually built another system called sickness AI. We'll talk about it a little bit more. But let's look at what were the issues, right? Uh what actually started breaking for us? So, as I said, we all

the systems are always changing. Our process is always changing. It's a dynamic space, right? And we are never constant more than like a few weeks. Um right now it's even lesser, right? And with in this era, we are building automations which are static. Right? Where we assume the process is a static, the formats are static. Right? And as soon as some of this changes, you know, our

automations begin to show uh cracks. And this is the thing we wanted to avoid. Right? And we just took a step back and wanted to see what are the real problems here. Right? So, let's just say you want walk into a coffee shop and there's a machine you're talking to and say, "Hey, I want coffee." Sorry. I want coffee and let's say that machine does a response

saying, "Hey, do you want sugar? Do you want milk? What size What size of coffee you want?" All this stuff, right? And let's say you were talking to another system. Like you just walk in and say, "I'm I'm tired talking to another system." You just gave you coffee. And now the system understands that it understands and it identifies you and say, "Hey, is it your regular, you

know, latte? I just added a double shot. Is that okay?" You say, "Fine." Right? So, this is the way we are getting used to interacting. Not the previous one, right? Where you have to answer a bunch of questions. And tomorrow if I say cream instead of milk, it shouldn't break. So, what we were building were rigid systems with static rules. Right? Which wouldn't apply as soon as

your process changes. And all your data and your uh formats that are changing consistently, that will break your validation rules and all that. We don't want to be, you know, um tightly coupled and we don't want to lock into any of the DSLs. So, to change this, what are the thing we need to do to not land up with these issues? We wanted some kind of an

intelligence built into the platform. A reasoning engine which can actually think for us. Right? And it can have tools at its disposal. So, that it can understand when a situation changes, it knows which tool to pick. A semantic understanding of the underlying data. Right? So, this was the change that we wanted to build. And this is what led us to the paradigm shift. And today I think

in the last six to eight months, like or maybe more, right? All of us are writing All of us are writing more prompts than code. Correct? Right? And this is the paradigm shift we wanted to bring to our automations also. We don't want users going in there and, you know, writing code for every condition of what can happen, right? And if there's a new condition, it breaks.

And then you patch it, the patch becomes the new norm, and the things go around, right? So, just as, you know, we moved from assembly level language to, you know, high-level languages, we're moving in from, you know, handcrafted codified workflows into more intent-driven flows. Right? And today our systems still currently understand only code. Right? How will now How will it understand prompts? Right? This is where we

had to change thinking in a way and rethink our to design AI-native systems. Right? When I say AI-native systems, we're not talking about just, you know, strapping a AI assistant on top of our pipelines, but it's going back and thinking ground up of how we have to build these systems, right? Think of today when you want to move from, let's say, point A to B. All of

us will open up our, you know, favorite maps and just feed in the destination, right? And then it the maps does the rest of it. It'll understand where you are, the context, where you want to go, it'll figure out the route of where you want to go. that is where we want to be. We don't want to be building static route maps and all that. So, before

we had the maps, how did we travel? We probably had printed uh I would say maps or static instructions. I would just go at the end of highway and you know, have a human and then talk to him, right? Like, how do I move next? So, that is exactly what we which what we don't want to do. What we want to do is to be able to

tell the system what is the outcome. Right? And not how to get to the So, this was what led us to move away from execution-based engines more intelligent AI-native system. Right? And that is the core of what we built in Cygnus. So, like, if you take the maps thing, right? It understand the destination. Okay? That is the AI reasoning. It knows where you're starting from. The route

is basically your policies and guardrails. The tools are what will track your movement. Right? And what we built with Cygnus was a control plane, which will bring all of this together. Your business intent, your automation logic, your tools, infrastructure, and guidance. So, this is the core or the foundation of what we wanted to do. And to build this, right? Uh we had to we have tried different

options. So, we first tried just wrapping up an AI and LM call to some of our features already there. We tried to build a co-pilot, which will, you know, take an instruction and generate code or generate steps for our automation. But that was suboptimal and it was it would, you know, break after some days. It's not able to take the complexity of the automation that we want

to build. Right? So, then what we did was we tried an AI agent, which again would understand the intent and do all the good stuff. But after the set of instructions, it would hallucinate. So, that is where we thought we have to simplify this and we went for specialized agents. A team of agents, all right? We didn't want to build single agent. Each of this agent had

a specific purpose. The guardrails under which they have to operate and the clear instruction of what is expected out of them. Right? And so, we built a few agents. The planner, which is basically understanding your intent and breaking down what you want to do. The executor agent is then going and looking at all the tools that are available to your system. Right? And then aligning this tool

to what you want to achieve. And the validator then looks at what output output came from the executor, And then validates it. And see it checks if all the tools are correct, if all the intent of the user you know, taken care of. And all this happens within the boundaries of our policies and guardrails. So, just take a simple example, right? If you think of a restaurant,

right? When you place an order, the head chef is basically your planner agent. He's trying to break down what user has ordered. And break it down into the steps and then send it to the line cooks, right? And that is where your actual automation is being built. And once it happens, there's a validator agent which is actually looking at everything that's come out. Right? Whether it is

exactly what the user wanted. It's up to the uh you know, it has all the ingredients and all that correctly then. And then we have the policy layer, which is basically general policy. The entire operation of this lies within that policy and guidance. Right? So, this is typically how you can think of the multi-agent system that is working behind the scene. So, we did this and we

built a system. I'm going to show you a small demo where our digital team generally what they do is they go on to target.com and verify all the information on the website is correct. Could be the price, could be the label information, could be the allergens and all of that, right? So, let's just go into this demo. Hello. Let's look at how digital ops team will use

Signus AI to do one of their automation, which includes auditing certain items from target.com. User provides the details in the prompt and the planner agent starts its work to reason and break down the task. The validation agent will go around checking everything the planner agent does. Once the user clicks execute, a browser opens up. It streams all the things happening on the browser and the user can

actually verify the outcome by looking at the final output. Now that the user is happy with the outcome, he would want to do it for multiple items. So, he puts the Excel file into a SharePoint folder and asks the planner to pick up from the same. But the planner could not find it. So, it asked the user to provide the file. Once the file is provided, it

reads through the file, goes through the planning and reasoning cycle again, and then generates the outcome. On execution completion, all the data that he needs has been updated to the Excel. He can save this automation using the save button. He can then go to the automation hub where he can manage all his interactions with the system. This page can have multiple versions, which we can go ahead

and edit if we wish. We can see that it has opened up the previous versions of this workflow. Let's say the user wants to mail this outcome to his email ID. So, he just puts it in a prompt. Once the workflows is generated and the user is now sent an email. Let's open the email and verify if all the information is provided. It looks like the system

was able to extract all the information needed. Right, so what we just saw was the user interacting with the system and telling it exactly what the outcome was needed. Right? It didn't tell it how to do it. It didn't say what tools it has to use. It just communicated in a way with a simple natural language. Right? And the rest was done by a multi-agent system. Right?

This is just one such example. But, let's just get into the details of what exactly happened, right? So, when the user first communicated with the system, he just mentioned what he wants to do. There was no context, so nothing came from here. It went to the planner agent. The planner agent has access to all the tools within our registry. And it's governed by all policies, so whether

you have access to certain rules or not. And whatever tools you have access to, the planner has access to that. It puts that in a way that it goes into a sequence of steps. All right? Then it goes to a action action filter. Right? This is where the catalog of uh you know, a subset of those um steps are matched and we see if all the steps

have tools associated to it. Right? And if there is something that there something some information is missing, let's say. It goes back to the user for the input. Right? And once it has all of that, it it has attached all the tools, it goes to the synthesizer agent. This is where basically your JSON is created, which is the list of sequence of steps. Right? And once the

JSON is generated, right? It goes through a bunch of checks. One is the hallucination. We make sure that it's not creating its own tools. Right? The second one is a validator agent. Here there are a few things which is it just looks at the schema, all the JSON that is coming out, all that is correct, the correctness correctness of the schema, correctness of the sequence of steps.

Right? And there's also an MEL rules, we'll talk about that in a bit. And then a final JSON was generated. Which was sent to the sandbox for it to execute, people verify. And then once he was happy he said, "Okay, I want to do it for multiple number of items." Right? He put it in a file, put it in SharePoint. And now he said, "Hey, do it

for all the all the, you know, contents in the file." So, here now it came as another instruction. We had the context which we, you know, saved from the previous time. It brings that context in, sends that to the planner. Now the planner knows it has to do the exact same thing. But just for the contents in the file. So, what it's adding is now it'll just

pick the tools relevant to the file processing. The rest of it is the same. Right? It goes through the whole cycle and even for the email part, right? It's exactly the same thing. You extracted the context, you already knew what he was doing. And the email he just looked at the email and then got the email tool that was there and shoot up. Right? So, it is

this is explaining all of that. But to do all of this we had to solve a lot of I would say problems that we had faced. So, I'll talk about like four problems that we had faced to build this. Right? Let's look at our first challenge. Right? This was when we started interacting with the system and there were more interactions, right? What happened is for every interaction

we were generating the entire workflow. Right? Suboptimal, takes time, tokens, and a waste of energy. And it also can cause hallucination because there's too much context. Right? So, to So, I mean to fix this, what it did is this what we did was we surgically replaced only the affected steps. We didn't change the entire workflow. Only the steps that were changed. Like, if you take it from

our example, right? The second time when I was generating the flow, it was only taking the file tools, and only that was being changed in the JSON. And we achieved with this feature called JSON patch. Right? So, now we figured out how we can generate the step effectively, right? And without causing Yeah. Okay, 10 minutes left. Right? The next one was actually Now, the steps are generated,

how will each of the step within interact with each other, right? Your tools are not always same, they don't produce the same Right? So, the previous tool may have produced parameters three parameters, but your next tool may need four, or maybe two. So, how will this interaction happen? Right? So, this is where we used an MVEL expression, the executor will understand the tools. So, between the two

tools, there is like a handshake, and that is the MVEL expression. So, when the when the executor is running, it will execute this expression, and then the next step gets what it wants. So, now we figured how we can, you know, build a JSON effectively, how we can interact the tools within the JSON. But, the next one interact with the system, and there are a lot of

interactions, there are also chances of failures. So, we didn't want the users to go back and debug, and look at, you know, what broke in the uh what broke in the automation, is the name of auto auto fix loop. So, the user would now not have to go at the time of the error, the exact context, the telemetry, and the reason of failure is sent back to

And the planner understands that there a mistake, and then it will recreate the And at this point, 80% of the time, this has been successful. And there have been issues where we didn't have the tools for it. Right? And that is the next problem. When we didn't have the tools for it, we built a fallback mechanism. And our fallback was to generate code to augment the tool

list. Right? But every time it generated the code, it had to pass four checks. One is we had a white-listed list of, you know, packages. Right? That it can only generate from. Anything out of that, it'll go back to the user interface again. But if it is within that, it'll go through that. It'll do a code scan. It'll also check if it is running in a sandbox.

Right? Once it runs in a sandbox successfully with the desired outcome, it becomes a part of my step. And it's not like we've solved all the problems. We still have a few. Right? And as you interact with this more, and and we are talking about 100 steps and above, right? Within your workflow, there is a context rot possibility. You have too much context that you're working with.

And then also the output generation. So, after a certain limit, um model will not be able to generate all the steps. Could be incomplete workflows. Right? And then of course the latency which is associated with all of this. For context rot, we actually have a deep agent mechanism that we are looking at, and how we can roll over the context, summarize the previous context, and add it

to the current context. Call it a rollover context. And then the output generation limit, we're also looking at compaction and how we can shrink some of the content that we are exchanging between the models. Right? And when these two gets taken care of, the latency will automatically come down. But to move all of this to production, one thing which was very important was we had to still

make sure we had all the checks and balances in place like we do for a traditional uh you know We needed to have our rate limiters, our circuit breakers, our execution error loops, right? All of those. And with LLM being a part of your architecture, right? You can't take anything at face value. You need to have end-to-end observability. Why a certain decision was made, right? And what

was the data and context at that point? All of that. And with all that, we needed a sandbox to make sure everything is running. And running in a way that the user wants to execute. Right? And that is what we're saying it whatever you learned in your um currently so far building your engineering systems, right? And all the best engineering practices is not to throw away when

you you know take these solutions to prod. It actually fortifies the solution in a way that makes it stronger. is what we think is the first step in the direction of autonomy, right? We are able to tell the system the outcome and the system A is able to associate and align agents and tools to perform the task for you within the trusted accountability in a reliable way.

And before I close, there are three things that I want to focus on. a system over scripts. We already spoke about how scripts can be brittle. And then multi-agents. Specialized agent gives you uh the ease of managing and maintaining uh applications. And we can also restrict how they act and react to certain inputs. And trust and governance cannot be an afterthought. It has to be the very

fabric of what we think and build. That's all I have. >> [music]