The Ferrari in the Jungle: Orchestrating AI-Native Engineering in the Enterprise - Sunit Parekh

In this talk, Sunit discusses the challenges and strategies for integrating AI into large-scale enterprise modernization projects. He compares AI to a high-performance vehicle, like a Ferrari, emphasizing the need to navigate the complexity of corporate environments—referred to as a 'corporate jungle.' The speaker outlines the unique obstacles enterprises face, such as slow approval processes, legacy system integrations, and the necessity for quick decision-making to properly leverage AI. He proposes a framework for embracing new ways of working, focusing on orchestrating AI-native engineering rather than traditional coding methods. Sunit highlights the importance of defining methodologies, setting clear specifications, and understanding organizational contexts to successfully implement AI in enterprise settings.

Uh, hi, I'm Sunit. I'm from ThoughtWorks uh office. Uh, I work mostly on the large-scale enterprise modernization projects, and that's where this talk comes from. Where AI is what everybody wants to leverage. And uh this is where I have a learnings from my own uh ways of working in the large enterprises. And how we have leveraged AI at scale in the enterprises, what has worked, what did

not work, what are the challenges that we have faced. And I think many of you must be following similar challenges in the large enterprises. Because in case of uh fintechs or startups, it's like very natural that okay, you will go and start using it. But in when it comes to enterprises, you need to be very careful what you do and how you do it, right? And this

is all our talk is about uh using the AI, which is like a Ferrari, and enterprises or the large organizations, which is like a jungle, right? And I think we say it's a corporate jungle, right? That's why this whole title came in my mind that how you can drive Ferrari in the jungle, and what are the things that are required to uh make that happen. Like so,

it's about orchestrating your AI native engineering in the enterprises. It's not about white coding or something, but it's more about orchestrating the whole AI native engineering in the organizations. So, it's like I think Ferrari is a high-performing car, right? And you give it it helps you to get go faster, right? Same way, I think when it comes to AI, AI can help you to build software faster.

But, when we are in the corporate world, there are a lot of challenges that we face. Not only from the ways of working, but also from the architecture side or the legacy system integrations, right? In in the enterprise world, most of the time, one application integrates with 20 other legacy systems. And that's where the challenges starts. Uh you are working in the organization where your uh whenever

you have to go and start or change a process, we have to go and talk to so many people. Like I think I I say career many times that Jao Jake us to sign off like you all just have to go operations with Infosys on the go. Jao Jao us to sign off like you all. Compliance with the on the go, right? And that's what we run

about in the organizations, right? Pillar to post. To make things work or any process change, you have to go and talk to so many people. Just to make a simple process change, right? And now with the AI world, this has become more bottleneck for us. And that's where I say that it's a bureaucratic environment. Lot of approvals you have to chase after. And in that, getting AI

work is a challenging part. And that's where it's okay. You are not only in that world. Lot of us are in the similar world. But how we have to find a way how we can make that work in such a complex, large organizations also is the key to it. So, the first thing that I talk about is embrace new ways of working. Now, in case of typical

corporate world, we have slow approvals. The The culture is more liability focused where I say that, "Okay, take everyone's consensus and then only we can move." Right? And getting that is a very big task. Whereas, AI thrives when you have quick decision-making. Like, for example, in case of process change, I have to first convince that if it's a sales-related process, go talk to say sales people. There'll

be impact of that process change to the operations team. Now, I have to go and talk to the operations team, make sure that everybody is aligned, right? And these are the reasons why in the large enterprises, people or the people are afraid of change. Because aligning so many people of those change is a typical problem. Similar stuff happens in the tech world also, right? If I have

to make any new language use any new language or use any new framework, first go talk to the architecture board whether this is allowed or not. Go talk to the infosec if this is this library is allowed to use or not, right? And that's where it becomes more and more challenging of using. Forget about we are struggling to use any open source libraries. Now, think and imagine

about using the AI agents and tools and all of that, right? It's It's a kind of challenging part and first thing that we have to start thinking about what are the new ways of working. Making sure that we align everyone in the organization that don't worry, it's a it's a I I say this as a follow the Agile principle, working software over documentation, right? Now, the same

stuff we have to start applying here. You have to make them that we'll build the software, you see it. If you think there is a change required, we can change it very fast also. And that's the beauty of AI. In the past, they were afraid of software changing or any changes required in the software will take a lot of time. That's why they don't give a sign

off very fast. So, we have to start talking to the people, making them understand that now software development can become faster. If you give approval faster and if we'll do the embrace the change also, right? Again, going back to the Agile Manifesto, right? Embrace change. Which is what we have to make them aware. And this is how this ways of working, establishing this whole new way of

working is important. It can take time. Generally, it took me two three to six months to make everybody align, get them ready to understand new ways of working. And it's okay, I think in the startup world, this might be totally different. People will be asking, "Oh, can you go more faster?" Here, it's important that take your go step by step, align everybody on the new ways of

working, then you'll have a better success there. Same thing happens with the architecture In case of simple applications like monolithic or three-tier applications, you can do very fast and better in the AI. But when you're you're in the world of corporate where you have microservices, you have micro front ends, you have front end application which is written in web, iOS, Android. So, the whole codebase is also

fragmented in a way, right? Not as a monolithic or at one place. So, that the AI can have a context of all the code that is already written. And many times, it is more of a brownfield development, I would say. Because when you build, you have to keep enhancing the feature on top of that. It's not like you are scrapping and building it again and again, right?

And another problem is the integration with the legacy systems. That's where you don't have even contracts. Integrating with Google Maps, integrating with the open standard APIs are very straightforward. AI can do that straight away. But you don't have even contracts in the world of enterprises when you are working in the legacy system. Your back end system, first you have to find out who is the right person

to get the contract, right? So, in that world, it's very difficult to follow what we see on the YouTube videos and all that that okay, you give the prompt and things are ready. It will not work like that. And don't worry, it's okay that we are not doing white coding, but we are still leverage AI. I'll give you some ways in which we have tried and worked

for us. So, that's the second. So, again, yeah, here, the architecture sweet spots, monolithic, your context is fully localized, cohesive, available. The AI suitability is very high. it just works for you. And that's what most of the If you go to Replit or if you go anywhere, they just build the monolith, right? For you. Everything working like this. But that's not the most of the enterprise applications

are right either they follow three tier architecture or and that's where we have to start learning how to leverage that so here in the three tier also it's good boundaries are very clear front end back end and the database right AI suitability is also good because you can keep everything into one repo possibly three folders or like that and then AI can have all the context in

one Standard code reviews and all that can help you to get there and use AI. Now in case of microservices or distributed systems the whole context of the is fragmented. The AI suitability is low. How you can increase that suitability by leveraging spec driven development we used to talk about contract driven development all of that is the key to make AI work in the world of microservices

and distributed systems and here if we can write machine readable contracts which is spec driven development contracts APIs in the form of standard formats then it works very well. enterprise development it is more of orchestration rather than wipe coding. Wipe coding means you write one prompt one paragraph and it runs for some time and it gives you the application it doesn't work that way so you have

to think about how you can orchestrate the development and make sure that the people in the organization are ready for and that's how you can make it work for So I talk about five building blocks. You have to pick your agent. Choose your model. Think about what methodology you want to follow. Spec defining the goal. That is what you want, and the context is how you want

to build it. So, I'll talk about that in detail. So, the first is agent. I call that as hands. This is the software that does work for you. These has tools, access. It'll make orchestration for you to do the software development. Second part is the model, which is the brain. So, you know, all the models are there. These both of these work in hand in hand, right?

So, these are the hands. This is the brain. The third is the methodology now. So, you have to have some method to the madness, right? It can't be like, "Okay, I give this prompt here. I give this prompt there, and then it will work." No. There are many methods available in the uh open-source world. Uh one of the famous one is the BDD, if you have heard

of it. It starts from even you can start from project management. How you do the design. Then you convert your requirement specification into master story list. That's what we do in the Agile, right? And then write and go epic by epic or story by story for the development. So, for example, if you want to build login, then build login as a one epic or a story. Do

with that. Then go with the second. Then go with the third. Like that, you can go epic by epic or story by story. Now, that's the methodology. And here at the organization level, we need to start thinking about how we can make sure that everybody understands this. Everybody knows how to follow this process. Like we have done in the Agile world or in the past which we

don't follow anymore is the waterfall world, right? That's the methodology. So we have to craft our own methodology that works for us and make sure in the organization everybody is aware of it. It's like a playbook for the organization. So we have to either follow something available or based on the whatever available make your own Then comes the spec part of it which is I call this

what. What we want to build. A goal or the main application that we want to build needs to be defined as a requirement articulation. So whatever application we want to build part we define in the what. And there is a method also there. And then the last part is the context or the how. How means we talked about context engineering, hardness engineering, guidelines, guardrails, all of that

are defined defined in the context. And the context is where it's a institutional knowledge which means that it is more of a organization knowledge. Okay, in our organization we follow Java as a default language. We use Spring Boot for API development. We use React JS as a front-end development. All of that needs to be defined. Now this can be done by the architecture group or the layer

who can define those standards, guidelines, blueprints for the And this becomes the knowledge of the organization also. And that can be leveraged across multiple projects. So, this is how we need to start thinking about and defining in the All these layers. And this is more around doing the software development. Now, this is choose your agent, which is a hand. Now, the options available here are cloud code,

open code, client, copilot. But, there are so many options available, right? Now, at the organization level, we need to make sure that which one is allowed to use. What are the uh guardrails to be built around that? All of that if defined and kept very well, it's easy to adopt. Second is the Where there are so many models available again here, right? So, which are the default

model that we want to go with? Now, one of the example is in case of Indian clients, we are using models hosted in India. And that's why we are choosing AWS Bedrock. Hosted in India. So, that there is no problem there. Very clear guideline that the data or the code should not be used to train the model. When you are using free or uh all the models

available which are on the subscription model, they can use your data. However, when you are using Bedrock, it is more costlier, but they will not use the data. So, define all this. If required, put a LLM gateway for the organization. So, that anybody wants to use the LLM, they can always go via gateway. So, you restrict you have the control, and you have the given them the

ability also to use AI. And always keep upgrading the LLM gateway to have the newer models. So, if I can say in the organization, I want to allowed only Claude Sonnet 4.7. 4. Whatever is the latest, I'll keep giving them. And that's how you make it available to everyone. And you control it so that it's not going on the internet directly. It's always going via your uh

gateway, so that you can access, you can restrict, you can rate limit, you can do whatever you want to do in your way playbook. We talked about this. These are B mad. Even in ThoughtWorks, we say our own methodology that we call we start from reverse engineering. So, we say that we first legacy modernization is the most critical problem of the enterprises, right? We say that you're

not always doing forward First, you have to understand what is the code written 10 years back in either mainframe or in PL/SQL. And from that, you create your knowledge base, and then you build the new system. So, we have to think about what path we want to follow here. We can have our own also. There is no uh restriction there. But, I'm giving one example. This can

be helpful for you to learn from or adapt And then comes the spec. where spec it is there, open spec is there. If you are following B mad, in B mad, there is a three-step process for step-by-step epic or a story development. You follow that for every individual stories that you put. The here, the objective is one prompt or one paragraph will not help you to build

the whole application. It'll be always step-by-step guided to build the application. I was talking to Venkat uh sometime back. He gave one very good example. He has many talks uh in this uh event, right? Conference. So, he wanted all the talks in his calendar organized. Direct prompt of getting all the data of all the talks into the calendar did not What he did is that he said,

"Okay, first get me the list in the normal text file." That's the step one. Then, from the step file uh from the calendar invites. So, if you do it, break it down, it And in the enterprise world, we have to write coding one prompt will not work. You have to break it down the whole application into multiple epics, like we do exactly, like we do in our

normal application development. And then create spec for that epic. And go step-by-step, follow either spec it open spec. There are many such uh available spec-driven development. So, if you search for spec-driven development, you will get many uh tools available there. And then comes our context engineering. we talk about agent skills. I think in the talk of Vanya, where we talked about seven uh take uh seven uh

things trends of uh AI there is one trend which is agent skills. And these are the agent skills that can be built for the organization. So, you build these agent skills for the organization and then at the organization level, you have the repositories of agent skill. This is your context. And these are reusable across the projects. Similar to I I give example, whenever we have a new

person joining the or new person joining the team, what we do first thing is we onboard them by giving all the context of the organization or the application. We say that in the organization we don't follow XYZ. We ABC. We say that we follow Java, we don't do we are not a dot net shop. Or we say we are a dot net shop. Or C sharp is

the thing default not Java, right? All that is the context. Once defined, you don't have to repeat it. So, this is very important that we build it at the organization level. Then it can be leveraged for many, many projects or across the projects. And you can have a better standards now. Because humans can make, but uh human can bypass even after reading. But one thing good about

AI is that once you give the instruction that don't follow this, they'll not do most of the time. So, again, now you have the new engineering stack for your organization. remember that this is all about orchestrating the whole thing. Take an agent, choose the model, decide on the methodology, do the spec to define the goal and context to set the guardrails and guidelines. Again, the hand, the

brain, the playbook, the what and the how. So, there are many, many tools. I think we learned and talked about many of these things in the overall conference many times, right? So, harness engineering is what uh we we learned about, right? And all of that comes across many places. You have to have the harness engineering in the hand as well, in the agent, in the context part

also, in the spec part as well. You put the guardrails, all of that. So, now when it comes to the organization, it's also important to define what kind of guardrails, what kind of [snorts] guidelines to be set because then there is always a risk in adoption, right? So, I say that go with use case driven approach adoption in the organization. So, first use case is frontline AI.

When I say frontline AI, it's more about revenue generating direct to customer AI usage. Like for example, if you are using generating code for the insurance or doing the underwriting for the underwriting workbench or customer service chart, these are direct to customer. In this case, risk level is high, critical, right? So, organization cannot afford to cut corners or do innovative stuff or anything. We have to focus

on strong risk controls. Cross model verifications, explainability of how the or the result has come, right? So, auditability is required because if you have to show that how I derived this quote for a And then continuous monitoring or observability is required. So, when it comes to frontline AI, you have to be very careful. Second is your productivity AI. Which means that in my day-to-day operation as a

employee of the organization, I want to use AI. Here, it's more of a assistant for business and operations activities such as I have some data, I want to generate some analytics or report out of it or I can have a internal employee chatbot, all of that. Now, these are productivity engines, right? The risk level is moderate here. Think of a tool and think that always the result

of what AI is going to give, the human is going to verify. So, if I generate some graph, I'll verify that the graph is correct or not. And these are the tools that we have to choose which are embedded. For example, in case of Well, many of us are using Jira. Right? In Jira, what is the AI tool called? Row, right? So, Row is a embedded AI.

So, we have to enable that. So, that the people you are writing stories and all of that can start leveraging that. So, most of the time these productivity tools or AI will come embedded inside your system. So, if you are using your Microsoft Office, then Copilot is embedded, right? Think. And as an enterprise level, we have to enable those. So, that the second level is there because

it is working on the data. Anyway, the data is available to that tool, whatever we are Jira. They know all the data. And their AI can leverage that to make our productivity better. So, that's the kind of thing where the risk is moderate. And choose for the embedded tools there. And the third is the supporting AI. So, software development of AI I call this as a non-customer,

non-business facing, right? I am using AI to develop software. Software is what going to use at the end. In that, there is no AI. But generating code is where I'm using. These are supporting AI. Now, here you can go for a innovation. Don't limit yourself here. Just go for it. Risk level is very low because you have multi-layer checkpoints. So, whatever code I generated first reviewed by

human, it's not possible that human can review all the code. But we have static code analyzer, security scanners, vulnerability scanner. All of that can be there in the pipeline to make sure that all the checks are built in a automated guardrail way. So, before the code goes to production, it'll go through all the testing cycles anyway. So, you are having all the checks and balancing last supporting

one, right? Go here as a Organization should adopt and do more in terms of the last one very easily. So, define a framework which helps and define these guidelines of using AI in Right now, if I go and talk to anyone in yeah, AI, look down, look down. I mean, I'm going to put something in here. And then, it stops for everything. See, they are worried about

this, the first one, which is true. But, why they are stopping the last one or this one? Should not be. So, that's where we need to have a strategy of leveraging AI within the organization, also. to build a software, the last part I'm talking more about is more about orchestrating, pick the pick the model, choose the methodology, write specs, and provide the context to get the whole

orchestration working. Yep. That's all I have. Thank you. >> [music] >> Mhm.