AI Is Not the Risk. Architectural Drift Is - Sunil Kalkunte

In this talk, Sunil Kulkarni discusses the impact of AI on system development and the inherent risks of rapid development cycles. He highlights the phenomenon of 'drift,' where the pace of feature delivery may lead to an erosion of system coherence due to unclear boundaries and undocumented dependencies. Kulkarni explains how AI-generated tools can increase development velocity but often overlook the necessary context of the systems being built. He introduces concepts like the 'drift zone' to identify where systems may be drifting away from their intended designs. The speaker emphasizes the need for clear contracts and boundaries within teams, as well as the importance of human oversight in automated processes to mitigate risks associated with fast-paced changes in technology.

I'm Sunil Kulkarni, by the way. I'm part of 7-Eleven Global Solution Center based out of Bangalore. Uh been with solution I mean GSC about a year and a half and I run products and platforms and run engineering for that with basically works across markets and also powers about 1,000 stores. And one of the things I've seen in the last year or so is AI and AI-led development

and led development has increased the pace at which we are developing and the systems have changed faster than ever, right? Some of these systems that we operate and work with um highly critical. They're on 24/7, no downtime, Um that's kind of where we're seeing the gap to come in and it's slightly invisible in terms of where uh we're operating as well. And if I can relate to

a typical use case, right? Uh some of you probably have uh automatic cars. If you drive a sports mode, right? And you generally see that if you increase speed, right? Um it's not necessarily guaranteeing a best commute or a better commute, right? What it gives you is uh greater drift, right? And I'm comfortable, right? So, that's kind of what I'm talking about today, right? Uh where the

gap is today and where it is getting introduced, right? How fast are we building? Uh where is the gap invisible today and how are we lagging in terms of where design as a review and design as a checkpoint is coming in, right? So, most of you are in this space today, right? And all of you have access to cloud or uh Wind Surf or GPT-based tools. And

all of you have seen your um in what I call the golden age of velocity today, right? Um velocity doubling, tripling in some cases. And what you've also seen is the rate of change in terms of delivery has significantly improved, right? And if you pause and kind of look at uh how are we operating from a design perspective, I'm sure all of you will probably concur that

all of these tools that we operate today, right? And the best of class tools, right? Are not necessarily helping us in the same way in keeping up with the pace of change, right? And what I generally now intend to call this is calling the drift zone, right? And that's kind of where the gap is today, right? How much do you understand of all the chain that's happening?

How fast are you keeping up with the change? How are teams able to understand if boundaries are being broken? And how are they able to keep the system coherent, right? In a lot of cases you'll probably also realize that the AI-led tools don't probably have the full context of the system you intend to build, right? In a lot of cases you do, right? The system doesn't. That's

kind of where some of these concepts needs to kind of uh need to be paid a little bit more attention because you at the end of it you'll probably see that the drift is too far and it's too hard to catch up, And if you look through in terms of where most of you probably will be, right? And keep me honest, right? You'll start seeing that most

of your PRs were correct, right? I don't think anybody would argue with that, but if you will eventually see that you probably the system was wrong, right? Um what does it probably look like today, right? It's not a code quality problem. It's not a developer problem. At the end of it you'll realize it's probably a system design issue, right? Give you some examples in terms of where

um in terms of it looks like today, right? Let's say you have a system or a service that you built or AI-generated that's working across, let's say, pricing, loyalty, inventory. And I'll talk it from my world and if you want to know about my our world, please visit our booth and you'll know more about retail, but I'm reference referencing that, right? And if you look at some

of those getting built, probably not stopping to see if some of those services that are getting mangled are a great idea, right? Um you'll also see when uh let's say for example if you look at a debt that you'll inherit inherit in terms of semantics, right? Like customer ID, user ID, or a member ID. They're pretty much the same thing and the same concepts, same models, but

if they could mean in different systems, right? And they will end up in a different way in different systems, right? Um but if you look at it and when a issue happens and if somebody posts on Slack or Teams, depending on what you use, and you have about a couple of days which were before you get a response, you have a gap and there's a problem, right?

Uh if you look at what probably happened here, right? Most local decisions look great, right? Uh it made sense and when the service was created, every time a decision was made, right? Every time a PR was reviewed, great merge, every service was built correctly, everything worked, right? But you start seeing that the system drifted, right? Uh into a shape that nobody designed or desired, right? But it's

also the paradox like I talked about in terms of way I see system development, right? Um every local decision looks and is faster, looks easier, but it doesn't model the system that you really intended to, right? It's kind of where you come in and most of you have probably seen this and probably living it right now, right? I want to run through a quick poll, right? Um

just to see where are you landing today, that poll one, right? Um so basically from getting a sense from all of you, right? Like where do you see today, right? As a problem in your space, right? And you're seeing this but it still happens and if you don't mind taking a poll and and giving us a response, we'll probably see what a pulse check looks like today.

>> [snorts] >> Um awesome. We have another 3 seconds left. Cool. okay, this boundary erosion hidden coupling. So lot of ownership fog that is saying you don't know who's owning it today. Okay, it's nice. It's a good good mix. What I want to do today is kind of give you a way to think about it, right? And given kind of what you saw on the poll, right?

You can kind of look at the your systems to be in in a spectrum of these. I call it the diff spectrum, right? Three zones basically, they're all heading in the same direction, right? And if your systems are in the coherent direction, right? Or a coherent you your teams understand the system, right? Your boundaries match your intent, right? Your ownership is clear, and if you ask your

teams what it what it's exactly building, everybody can give you a quick answer, right? Contacts are explicit, you have a service history, you have a contract registry, everything is clean, right? And if you and teams can actually describe what they're building very accurately without looking at the code, without looking at Slack, any history that that they want, right? But then if you look at the fragile spectrum,

nobody understands it, everybody's scared to touch it, right? Boundaries are theoretical, it's been built over a period of time, you're still building on top of it, context of the system is low, still struggling, right? Coupling is structural, right? Change requires a lot of work, right? And I would probably think of it more as a lagging indicator because you will start seeing that velocity has deteriorated. I used

a decent word, but velocity would have been really low and would probably atrocious. And everybody knows about it, right? Where you really want to kind of pay attention is the drifting side of it, We think we understand it, but boundaries are fuzzy, right? You still don't have clear domain boundaries that you are able to articulate, document, and talk about it. Couplings are hidden because you have teams

building probably over domains and contacts. Um contracts are not cast in stone. They're informal. So, you don't still don't understand who's building what, I would treat this more as a leading indicator today. If you are able to catch it, uh it you want to catch it before it moves into the next zone, right? You want to be able to make sure that you don't want to end

in the fragile zone, which is not the danger zone from what I call it. Um you want to prevent yourself going into the testing zone and because you don't know that you're there. >> Right? Um just giving us insight into what potentially could look like, right? And not to just talk about philosophy, right? A little old pattern, right? Um like let's say some of my teams uh

convince retail at scale, right? Uh let's about assume a payload, right? From three market teams that I operate. Um loyalty payload, all of them shipping differently for different markets, right? It's fast, yeah, assisted. Every local market local team is running at high velocity. Well reviewed, all PRs merged, everything looks great, system looks great, right? Um where the drift could come in, right? Let's say one of the

market teams, right? Change in a field name, right? Uh no contract change, field name change, right? Midway, AI generated, everything looks good, locally correct, right? Uh all approved, right? Um two consumers still reading the old name, right? Uh no alert comes through, right? Um and think about it. No alerts, uh no exceptions, and some of you might be wondering, "Hey, this should throw up an exception, right?"

But if you look at a JSON deserialized payload, um field name disappeared, new field appeared, consumers probably think it's a null field. So, my loyalty con or a loyalty uh redemption number could start becoming zero. Where does it end up, right? It probably ends up in a business report. Somebody from your FinOps team, somebody from your uh business team is up in a business report two days

down the line. It's more of postmortem question, right? And how long has it been happening, right? You are not able to figure out up front. You are not seeing that as a problem because it was a failure that never showed up as a failure until it showed up in the report, right? That's kind of where you would want to pay attention to because nothing failed. Everything worked

and that's kind of where the problem lay. >> Right? I want to give you maybe some ways of looking at it, a framework that maybe could help, right? How would you kind of start looking at attempting this, right? You don't need a dashboard, right? If you think if you see that you're probably drifting. You probably want to ask these questions for yourself or your teams. All right?

You have clear boundaries, right? Can each team describe the boundary clearly? They don't need to look back on Confluence, Jira, or any tool that you use. They don't have to go look back on threads, right? Or ask somebody else, right? If they're able to do that, right? You're good, right? And are all contracts explicitly documented, tested across boundaries, across domain boundaries, right? Regardless how the code was

generated, whether AI, human, doesn't matter, right? And you have any system, any file in the system in your repos, can you identify a team that is responsible for it in under 60 seconds? Can you name the team, If you know that, then you're kind of on the right track, right? I mean a new feature is added, right? Does everyone consciously ask, right? Where does this lives and

why, right? And if you are breaking domain boundaries, you want to be able to ask these questions much earlier and to the right teams as well, right? The critical part is asking the questions about when AI generated or human, at what point is the human in the loop coming in? Cuz as we accelerate autonomous AI generated development is already there or or just about getting I do

want to kind of also get into a second part of it, right? Um some of you we kind of talked to the framework before I kind of get you to think through on what to do, right? I want to see one more poll. This is poll two. Can you tell me from that? Yeah. What? Okay. Let's see if you can do it faster this time. Let me

do this one. >> Okay. You good? Oh, yeah. I got it. Thank you. You did that. That's good. I still want to keep that in this the Doing good. >> What happened? Let's fix that. Let's do it. You don't know it's another 30 seconds. Let's take him. What's going on? I got to find that thing that What happened? Please find. All right. Now that you kind of

saw what I was asking to write, for all the system that you care about What do you feel is true today? Right? Are you shipping fast, but you can't explain boundaries? your features land without asking where they live? You have undocumented dependencies that today nobody owns? I honestly all three, right? Interesting. I claim I come back to my deck. All right. Most of you big deal, right?

That's That's not not three separate problems, right? In that sense. It's all one, right? Um and if you go in this probably a path is for it and and you basically going to look at how do you solve it? So, I can give you some uh ideas in terms of um how to kind of approach it, right? Um let's say your problem is in boundaries, right? Think

of it and some of you might probably be familiar with circuit breakers, right? Whether the actual circuit breaker or in patterns that you're using, think of it in architecture circuit breaker, right? When a change crosses domains, right? And the boundaries are crossed unexpectedly, you want to be able to trip the circuit breaker, right? Full stop. Human in the loop. Human makes a decision, irrespective of who wrote

the code. AI, human, doesn't matter, right? Don't think of it as an approval queue. Think of it and not as an approval checkbox. It is the same way the circuit breaker works. It has to stop. You have to move after a decision is made, right? Your problem is seen in contracts, right? Two services sh- sharing similar data structure, um you want to start thinking about how do

you solidify schema entries, schema journals. And if you've You're probably attending the talk tomorrow. The Dr. Venkat is doing talk on ADR on architecture design receipts. Probably give you a perspective of how to kind of solidify some of these things, right? Um and we come back to contracts, right? Uh you want to be able to write it down, test boundaries, right? So, it doesn't matter who wrote

the code. You still have a way in a structured way to kind of make make sure that you're not um breaking those, right? So, and you also want to make sure that um your contracts are also defining what an architectural call can take, right? And what it doesn't break. So, uh feel free to kind of take that session tomorrow. It's going to help you kind of set

the course on some of that. So, if you were all also pointing to D, right? You need to kind of slow down. We're going to start looking at where the change lies and where does it live today, right? Some of you have to ask the question in terms of making that decision. Um not sometimes, on every single decision. Not just for big features, for every PR merge

in the normal slow of work. That's kind of where the change will happen, right? If that question goes missing, you'll start drifting, >> I want to kind of come back to this in a bit uh in terms of what where we landed, right? And I talked about AI not being the risk there, right? Um drift is the risk. A structured approach to it is the response to

it, right? Um if you look at how we've started to build, AI is not breaking our systems, right? AI is didn't break break those. We're shipping faster, shipping higher quality code. But AI also exposed how weak they were already, right? And some of you probably seen that more than anybody else. It's only going to get faster, right? Tools are not going to slow down, The only thing

left is how do you manage the change, right? And the only variable how well you can articulate the change, how well can you understand the context, how well can you build the context for the system, right? And how deliberately can you manage the change in design, right? Uh that's kind of what I wanted to share today, right? And how do you kind of structure that and um

happy to kind of take any questions. I'm also around on the 7-Eleven booth. If you want to come by and we can talk more, but happy to take any questions. >> [music] >> Oh.