Martin Reynolds & John Crespo - AI for Everything After Code

This talk focuses on streamlining the software delivery process in the context of AI integration. Martin Reynolds, from Harness, discusses the challenges faced by organizations after code is committed, highlighting issues like fragmented workflows and rising cloud costs. The speaker emphasizes the importance of treating internal developer platforms as products, ensuring they are user-friendly and effectively meet developers' needs. He shares successful patterns observed in leading organizations, which incorporate principles such as embedding security and compliance checks into the development pipeline. Additionally, Reynolds introduces how Harness employs AI to reduce cognitive load on engineers by automating and enhancing the software delivery process, ensuring reliable deployments while maintaining a focus on security and cost management.

So play big round of applause for for the team from Harness. Thank you very much. >> [applause] >> Outside. After this talk. Mhm? Yep. Oh, okay, we're on. We're on. We're almost on. Hey, okay, there we go. I knew we'd get there in the end. It's It's Friday, it's the afternoon. You probably just eaten lunch. So, I will try and make this as interesting as possible. My

name is Martin Reynolds. I'm field CCO at Harness and this is my colleague John. John Crespo, solution engineer at Um, yeah, nice in Amsterdam. See, I was going to ask some questions, but I actually can't see any of you. So, the there's just blackness in front of me. So, maybe you can give me a shout, right? So, how many of you use AI in coding today? Okay,

so almost all of you by the sound of it. Okay, and how many of you use AI after code? So, AI in testing, AI in security. Yeah, that's a little less enthusiasm. Okay, completely different question and I'll come back to this at the end. How many of you like Star Wars? Yeah. I less enthusiasm than I thought. Okay. I'll come back to that question later. So, today

we're going to kind of focus everything that happens after where engineer hits that final commit and says, I want to send this code and get it in front of our customers whether they're internal or external. Um, you know, right now what we generally see is a lot of fragmentation and, you know, it's spiraling cloud costs, all sorts of different things. As it's not a great experience. So,

you know, generally the question we're trying to answer is not, you know, how can we generate more code? We've already kind of solved that. We're doing that a lot already. It's actually how do we ship that out scale and we keep it reliable and secure and safe. thank you. My beautiful assistant there is going to move the slides on for me. So, a lot of the platform

teams that we talked to today, you know, they'll tell us, "Hey, you know, we've already got some kind of platform or we've got CICD implement implemented. You know, we're deploying out to Kubernetes maybe or we have an internal platform that handles this." And so then I change the question really from, "Hey, how are you shipping software?" to, "You know, from that stuff you've built, how many of

your teams are actually using it?" And the answer is quite telling because often that answer is, "Well, it's not consistent, right?" Some teams use it, but some teams go around it, which is, you know, and that's often because it's quicker to go around it than it is to use that that blessed path to production for one of a better phrase. So, in fact, I had one customer

that described it, he said, "Right now, it's like we're building one piece of furniture, but it's been delivered in three separate packages and we don't know which package is is right for the furniture." He was making it He's just saying it was super difficult and AI is generally making that worse, right? If delivery was, you know, you were struggling with delivery now and now you're generating more

code, more change, then that's a problem. So, um, yeah, we can go to the next one. Thank So, the you know the AI conversation honestly that people avoid is is what this slide represents, okay? Which is that there is a genuine explosion in change. So, often we talk about it as a general explosion in code, but really all the all that code is just change. Change in

the systems, right? And you know, the DevOps principle, you know, going way back is, you know, you want small uh amounts of change that you deploy regularly, you know, because then you're reducing risk. But what we're actually doing now is we're generating, you know uh we're kind of going the other way a little bit. We're generating bigger changes. Uh we're generating them much more quickly, you know,

and actually often, you know, we're we're seeing, you know, if your systems for delivering that, you know, whether that's the getting the testing done, which might be manual, you know, whether that's getting the security scanning done, which is often kind of done after the fact. Uh you know, or, you know, validating that infrastructure as code, or validating, you know, the security and everything else that goes around

it. You know, if you were struggling when you were doing maybe 100 uh you know, commits a day you're going to struggle when you're doing 400 commits a day. It's going to break those systems. So I've got to stop saying you know. Uh go on, carry on. the way we're going to break this down is um going to talk a little bit about just some of the

success patterns that we're seeing, you know, with people that we're working with. Uh and then show you a little bit of how we at Harness facilitate that. So, we're going to start with, you know one of the first things you really need to do is is treat it your platform, your internal developer platform, or whatever you want to call it, your delivery system. You actually need to

treat that as a product, okay? So uh times, you know, what engineers are experiencing is they know they have to go to one place to do security and then they have to go to another place to maybe get some compliance done and maybe they have to go and ask somebody else and raise a ticket to get some infrastructure. And a lot of people are treating their platform

as an integration product for all those things, but it doesn't change all those gates that are in you know, in place. So, what you actually need is genuinely to treat it like a product. You know, if going back to what I was saying before, if your engineers aren't using your platform or your golden paths, that's not a criticism. That's feedback and you should feed that back into

the platform. Um, you should be uh, you know, gathering those requirements about how it's used. You should be able to publish SLAs about the way your platform will deliver. Um, it should have everything embedded in the process, whether that's security or or um, resiliency testing or anything else in the process. So, if we go on to the next slide, thank you, John. I see. He's a beautiful

assistant, so helpful. Uh, so a lot of the best uh, engineering organizations, you know, software delivery actually becomes incredibly boring and really predictable. That's not a bad thing, right? Developers can go into one place, whether that's a portal or maybe directly from their I you know, their IDE, Windsurf or Cursor or, you know, Cloud Code or whatever it is that they're using these days, but Visual Studio

uh, and they can say they can go in and uh, just get everything they need with a simple request. Right? So, it's really about kind of getting everything that they need. So, security is embedded, resiliency is embedded. They get cost information up front before it's deployed, not later afterwards when it's a surprise bill and Amazon and Google tell you, "Hey, you just spent an extra $20,000 on

that new feature you just launched, which isn't necessarily always a good thing. And it's really about kind of, you know, not deploying and you kind of change the focus. It's not about the mechanics of all those stages, you know, all those things that you're seeing up there. It's actually about what's the outcome that you're looking for. So, what you actually want and most organizations want is they

want to get more product features out the door so that they can realize the value of those features. Thank you. So good. Uh you're almost predictive now. I I'm I'm I'm happy. So, at Harness we essentially focus on everything that happens after that commit, everything that happens after code. We apply AI in kind of three core ways. So, first of all, you know, it's really about reducing

that cognitive load. We want to reduce the number of tools that uh engineers have to uh go to. Um and we do that in a in a whole series of ways which we're going to show you, but you know, it can be with the pipeline governance, it could be with approvals, it could be with testing, it could be with the actual rollouts, right? And that's all embedded.

Uh so, second is surfacing risk and cost before release, right? You want to know what the risk of that change is. You absolutely want to know what the cost of that change is. And you want to be able to surface that as signal, right? Right up front before the developer deploys that and and it's meaningful information to them rather than later when it's already been deployed and

a a month has gone by and now you're seeing the impact. And then uh the third one uh excuse me is uh you know, automating safe change or automating safe action. So, for example, if you're doing a deployment, uh maybe a canary deployment and you'll see a lot of canaries on our shirts and everything else, but uh um if you're doing a canary deployment and it goes

what you actually want is automatically have your SLOs embedded into those releases and automatically understand that this release isn't good. You know what? This We always see these same errors, but this time we're seeing some different errors or we're seeing the frequency of those errors increase or actually it's running about 15% slower and have the system automatically decide to roll that back because you don't want the

end user to have a poor experience from the change that you generated. Okay, so now I want to do is kind of zoom into onto kind of three core pillars that make up a lot of the work we do and and this is um where we're seeing the most successful organizations apply these principles. when we talk about golden path, we're really talking about those safe paths to

production to those safe landing zones in known architecture that's already secure and what we do at Harness is make the adoption of those super easy. And in fact, in today's world, it's becoming even easier, but you know, you could go to an internal developer portal and you could say, "Hey, actually I want to deploy this, you know, regulated service to our Kubernetes production Kubernetes cluster." What they

get is not just a templated repository, they get what they actually need, which is the repository and the pipelines attached with the security scanning embedded, the resiliency testing already in place, the required uh rollout strategy in place and all the compliance and governance and integrations that need to happen so that you can get as quickly as possible from code to production. And the way that a developer

asks for that can also be, you know, right in in their IDE. "Hey, I need to deploy this to our healthcare platform. Please, can I get a you know, a pipe a pipeline to do that?" and it should be able to understand that and attach it, understand the context of your organization, understand the rules that your organization has for being able to deploy, and guarantee that all

those things happen before it gets out Okay. So, uh yeah, let's do the next one. >> [laughter] >> Thank you. Uh this is one of my favorites and actually one of my colleagues, uh he always asks uh how how often security teams and development teams actually talk to each other and it's often never in a nice way. It's always when they're saying, "Hey, there's something wrong in

your in your change that you've done." Actually, security should actually be embedded and in fact, in an AI world, it should be more than that. It should be, "Hey, I can see that there's a hundred uh security vulnerabilities in this code, but actually, only five of those came in with the change that you made. The commit that you did introduce five changes uh five security vulnerabilities. Of

those five security vulnerabilities, we've checked and actually, only two of them are reachable. So, that they're the only two that we care about. And you know what? What we've done is we've gone ahead and generated the the change that you need to make to actually fix that problem. So, you just need to review it, merge it into your branch, and you can move forward. That's kind of

the change that's happening with AI. No longer is it, "Here's a list of security things that you need to fix," but actually, "Here's the security things that need to be fixed that actually matter, that can actually be reached." Oh. Oh, I'm back. And here's how you fix them. Okay. >> That's just like that. It's magical. so, yeah. And then, if we go on to the next one.

Thank you very much. So, this is super important. Uh and actually, where I originally started writing this talk from was this kind of this idea that uh, you should have all the indicators in place for your platform to know whether a release is safe. That you've mitigated all the right risks, that all the right steps have happened along the way, so that actually you can detect those

signals easily and make the right decisions based on that. Whether that's, you know, your uh, physical kind of um, observability metrics around performance and all those kind of things, but also around the business and governance and compliance stuff that you might need to do inside your organization. And, you know, here in Europe there's a lot of regulation that goes on in terms of, you know, resiliency testing

and and compliance that just needs to be met. I I used to talk to organizations and they'd say, "Hey, I'm a bank. We're a regulated organization." Now, it's rare I talk to a software company that doesn't have some kind of regulation and compliance rules that they need to adhere to for the market that they're working in. Uh, so, you know, we need to be able to automate

that. And actually, we've seen huge success with this. I I put uh, Ancestry uh, up here on the slide. By just by integrating resiliency testing into their golden paths and pipelines, they reduced their downtime by 50%. And bearing in mind that for Ancestry you know, the website is their front door. That is their, you know, the gateway right to their business. That's where they collect their revenue.

Every minute of downtime costs them money. A 50% reduction by embedding their resiliency testing is a huge improvement. So, last one before we get into showing some stuff. And I think I've overgone as I always do cuz I love to talk. Uh, but we started late, so I'm going to keep that on in mind. Uh, so, the last one is, you know, we do not want to

limit uh, the amounts of innovation that engineers do. We don't want to constrain that, but we do want it to be informed. You And this is a real change that we we're seeing happening is that, you know, those best teams that have that innovation and doing it in an informed way, they have that cost as a signal. Actually bringing FinOps and DevOps together, stop it being something

that happens after it's deployed, and start it making it part of the pipeline. So that actually, you know, the engineer is getting the prompt saying, "Hey, do you know this is going to add 15% to your bill if you deploy this." And that might be absolutely fine. It might be that, "Hey, this is a big feature and we know it's going to add a lot of infrastructure

and that's absolutely fine." Or it might be, "This is going to add 15% to your infrastructure and I've just done a three-line fix. What the heck is going on?" Right? So but having that up front and knowing it and being able to treat it as a signal is like a huge change and a huge evolution in the way software gets shipped in a in a cloud world.

So what we're going to do is we're going to try and make this a little bit real. I'm going to let John do some talking because you're probably sick of my voice at this point. So I'm going to let John do a little bit of talking. He's going to show us some of the system. He's going to start off showing us some of those golden paths and

actually some of that kind of AI-driven onboarding that we that we've talked about. Thank you, Marty. Although I'm not sure if they're going to like my voice better, but let's see let's see how this goes, okay? All right. So what we're seeing here in the screen, guys, is an end-to-end software delivery pipeline. We're taking a piece of code, uh we're creating a commit, we're creating a PR,

and then we're triggering this whole deployment flow, right? As you can see here, we have different stages in the upper side of the screen. We have built and test and push. Then what the pipeline is doing is encapsulating all the Terraform scripts that I have into a single works- workspace that is relevant to that this particular service or application I'm deploying to a particular environment and he's

doing that a family and the reason being is because of the cost savings, right? And I'm going to show you in a moment how the pipeline also shows the cost estimation for this particular ephemeral environment. And after we have a deployment environment available for us, then what we can do is to actually create some sort of deployment strategy. And one of the things that Harness allows you

to do is to actually be very aware of the supply chain. So as you can see we are injecting in here quality gates or security gates just to make sure that the artifact has been signed is the same artifact that we built in the first place. No one has been tampered with it. We have self verification for compliance reasons and regular regulations reasons. And then all the

change management service now creation of Jira tickets etc. etc. All that is embedded into our pipeline. Automatically populated with what's relevant for that particular ticket, right? So gone are the days in which a developer has to have all the cognitive load on all the different tools disparate tools that they need to context switch to so they can actually focus on what's most important which is writing cool

features that will ultimately drive revenue to the company, right? And same thing happens with database update database updates etc. One of the things that in my developer days I hate is a strong word but I had very strong feelings towards it was the QA part, right? Because I usually got three QA engineers with a with a spreadsheet a manual spreadsheet back in the day showing me all

the reasons why my code didn't work or all my APIs I wrote didn't work and all the test cases but they they told me what failed but they didn't tell me how to fix it, right? So all the test use cases should be embedded in a pipeline. So as an engineer I to focus on writing my code and then all the tests, particular tests that they're writing

and executing for me, if they embed it in our pipeline, as you can see, if I have the ability to create Jira tickets, all the remediation steps and all the feedback should be living in my backlog that then I can go and pick up instead of losing my zone of what I'm doing at the moment to go ahead and fix something that might not be as relevant,

right? And with that in mind, before going to production, we all are familiar, I guess, with change advisory boards, change release management. So, these are the people literally like that goes to service now, check that everything is in order, and then go ahead and click approve, right? But because everything has been automated in the pipeline in the in the delivery process, so all they have to do

is to go to service now and and approve once they review, and this pipeline will pause and until that hasn't be approved, the pipeline will not continue, right? Once the approval is done, as we can see here, we have all the complex on-premise strategies, etc. So, this is to to Martin's point, this is what we call a golden path, okay? There's no reason whatsoever we need to

create a pipeline per service or per team or per environment. If we have front-end developers, we should have a front-end pipeline, for example. It doesn't matter what technology you're using, we can create dynamically variables, right, that will then dynamically determine what path in that pipeline we're going to be taking, right? Instead of us making a pipeline from the start. once we have this golden path, we can

create naturally, we need a platform that is powerful enough to be able to integrate with different cloud providers, it's cloud agnostic, different artifact registries, different code repos, ticketing system, monitoring, logging systems, right? So, Harness has a very comprehensive connector list that we use to connect to all these, you know, usually disparate and disjointed tools that exist in the in our organizations as I will see. Uh with

that in mind as well, um Harness has the ability to actually templatize, right? So, let's say in this golden pipeline, I want to go ahead and create a template out of this pipeline so I can go ahead and share this pipeline across different projects, across different organizations in my account, right? From a maintainability perspective, that that that is very relevant. So, now, the most important part in

here I would say is the AI generation. Now, we are living an AI native world, and I'm going to revert refer back to what Alexi was saying in his last presentation. You cannot have a Ferrari which represents an agent nowadays if the road is full of, you know, traffic and holes in the road, etc., etc. So, you need to first fix the road and create a baseline.

If that baseline is created, we can go ahead and ask an AI to build a new pipeline, and the AI, by the way, knows what service I'm working on, what security scans need to be executed, what artifact registry, what kind of environment am I am I going to be deploying to. And the pipeline goes ahead and ask a few questions, and voila, I have end-to-end pipeline to

which, by the way, I can go ahead and keep adding things as I ask the AI to generate those for me, okay? I'm just conscious of time. We just have 5 minutes, I think. >> Yeah, yeah. Um so, I'm going to move on to the next section, which is the security section. Shift left security. Someone has heard that phrase before, shift left security? Yeah? Yes, no? All

right. Well, pretty much that that I have strong feelings toward towards that phrase because someone had the brilliant idea of let's put the security responsibility in the hands of developers, and no one wants to be a security expert, right? Um some people might be, but not everyone wants to do to to be a security expert. So, what Harness does is to actually get all the security scans

um that usually organizations have and then gets them into a single place, right? To a single place in which I can I can see all my critical CVEs, all the CVEs that have been introduced only by my code change, not the the ones that have introduced by my team. And then you can go ahead and see the remediation steps. Because developers, as an engineer, I don't I

don't mind you telling me that I have a problem, but tell me why and the impact of that problem and also give me the remediation steps, right? And because we live in an AI-native world, why don't you create the PR for me already as well? So for example, if if my problem is in the code, I can go ahead and click on Semgrep Git Leaks and uh

usually what I see is um that the AI here gives me the remediation and has created a fix for me. So then it creates a pull request. So all I have to do is to go ahead to that pull request, review it, approve it, and then the pipeline is going to continue. And guess what? Everything I can do with my IDE. Give me the status of the

current execution. This is the status of the current execution. Give me uh all the CVEs that are relevant for this particular execution. And more importantly, give me the remediation. So as an engineer, I don't really have to context switch to Harness or to any other tool. I can just focus on uh what I'm doing in my IDE and that's it. And I'll get the feedback feedback immediately,

right? Just to finalize um every time I spin up a resource using that pipeline, we need to be aware what's the impact in the budget of my team. Because every CTO, every engineering manager has a very limited budget. And by the time they need to ask for more money, the answer is usually no. Or help me understand why you need uh more budget for your team. So

having that visibility as to how much are we spending by introducing this change that is going to have trigger this pipeline that is going to uh spin up as an ephemeral environment is super important. As we can see in here, this represents 184.3% uh increase in in cost. So, what Harness does is you can simply create a policy that will block the execution or the creation of

this particular environment, Now, more importantly, to finalize, gone are the days in which as as an engineer or even as a DevOps engineer, I have to, you know, be constantly monitoring or babysitting my deployments by looking at my observability tools, right? Dashboards are pretty cool, etc., but you know, no one has to be on call anymore and at 3:00 a.m. something is on fire and then I

need to wake up and everyone gets in a room. This In an AI-native world, that shouldn't exist and that's why a pipeline should be able to integrate with these observability tools, gather the metrics that are really relevant for that particular deployment that is happening, and track these different metrics in real time, and compare them to a baseline that you had before. If we find a divergence, well,

guess what? The the pipeline intelligently knows the path to revert that change that uh has been introduced. Right? In this case, I'm rolling back my canary, I'm rolling back my database changes that I've made with the pipeline, I'm updating the ServiceNow ticket, I'm closing it. It lives in my backlog on Monday morning when I wake up. I I I check my IDP, I check my Jira, and

I have the remediation, the impact, and the and the what happened as well. So, hopefully that gives you more or less a gist of what Harness can do and I'm done now. >> That was whistle-stop. I'm impressed. So, just just to finish really, really quickly, cuz I know we're right on the limit now. So, you know, we want to help organizations ship fast and stay safe, but

we also hope that we've just given you some good practices to follow. Uh I'll just remind you some of those. So, you know, treat your product your platform as a product. Take that feedback, use it, refine it, improve it constantly to make sure that things like security, SLOs, costs, governance, compliance, make sure they're embedded from day one, right? And part of the process, not something that's bolted

on afterwards. And then, you know, use AI where it actually makes sense and reduces cognitive load. You know, especially in that space after code where there's so much context switching between tools. Making that easily accessible and embedded is key. I think the one two last things. This is one we have a developer experience summit coming up. We have people like Gene Kim who's if you don't know

who Gene Kim was is very influential in DevOps, wrote The Phoenix Project among other books. We also have Nathan Harvey from Google who will be joining us who looks after the Dora metrics project as well as a bunch of others and it will be very much focused around developer experience in the world of AI. So, finally, I would say, "Hey, thank you for coming and seeing us."

For those of you that like Lego or like Star Wars or both, come visit our stand, get a ticket and and and get raffled in to win a win a Lego. We're going to raffle that off at 3:00. So, you know, please please stop by. If you want to see how to ship fast, stay safe, and get more than the 10 minutes of of time from John

here to actually see what's going on behind the scenes with Harness, again, come see us. We really appreciate your time and we really appreciate you listening to us. Thank you so much. >> Thanks, everyone.