Sponsored Keynote: Scaling Platform Ops with AI Agents: Troubleshootin... J. Palma & N. Yellin (ASL)

This talk explores the integration of AI tools in daily operations, specifically focusing on enhancing productivity for Kubernetes users. The speaker discusses how tools like Headlamp, Holmes GPT, and Flux facilitate efficient troubleshooting and management within cloud environments. A real-world scenario is illustrated where an issue arises during a checkout process at a pet store, leading an on-call engineer to use Holmes GPT to diagnose and resolve the problem quickly. The speaker highlights the role of operator mode in Holmes GPT, which monitors cloud environments and catches issues proactively. Additionally, advancements such as self-mutating agents are mentioned, showcasing the potential for AI systems to autonomously create integrations and streamline operations further.

Good morning, KubeCon. So, you've all seen how AI is becoming pervasive and we've all been told that we probably should be using it to become more productive. But, that can become a bit cumbersome when we are find ourselves between our day-to-day tools, different chat windows, asking them questions, copy-pasting from our tools to different context windows, then going back and doing something with those tools again. Wouldn't it

be great if that were actually well, fluid and integrated? George, you work with a ton of Acast customers. How do they do this on a day-to-day basis? Well, we have a lot of different customers with different tool sets, but there's a few tools that stand out that are very, very common. One of them is Headlamp, the Kubernetes UI that is part of the SIG UI as a

subproject, and it's an extensible and simple UI that allows you to manage clusters and applications. We also see them use things like Holmes GPT, which is an agent for operations as well as SREs. And we see them as we've seen today on stage that they're using GitOps flows with things like Flux. So, let's see how we can put all together and troubleshoot an issue that one of

our users So, we're Contoso, we're a pet store, and one of our users is making a purchase for things like catnip and some cat toys. And so, they're going to add to their cart and they're going to try and check out, typical flow, but there's an issue. So, they're going to open a ticket and it's going to go to one of our on-call engineers who luckily has

no idea what's going on and is going to go into Headlamp and ask the AI assistant, which is powered by Holmes GPT, to what what's going on. And it's hoping it's not DNS once again, cuz it's been apparently typical. So, here Holmes GPT is actually going to leverage a myriad of tools such as kubectl, of course, but also things like the Inspector Gadget tool set. So, it's

going to run a eBPF gadget from Inspector Gadget to do a TCP dump, figure out that it's actually calling a order.dash-service that doesn't exist. What exists is orders, plural, -service, and it actually found that as well when listing the services. So, it's telling me, "Hey, there's a typo here. You're pointing to the wrong service that doesn't exist." And it analyzed that looking at the TCP dump and

where the application was calling. So, now I'm telling it, "All right, go right ahead. Please submit the PR. Give me a good description cuz I need to hotfix this, so I need to give this to my incident management." And we're going to go straight up and merge it cuz we're in hotfix mode. But, this is our production cluster, so we can't just apply it now. We're going

to go ahead and go to Flux. And we see this from the Headlamp Flux plugin. And as we access our application, we're going to sync it from source and we're going to get this new PR that resolved the typo issue with our service. So, typically, what you found here is in 2 minutes, we were able to diagnose, mitigate, and root cause what the issue actually was. So,

now we're probably going to have to do some explaining to our team, so we're going to ask the AI to write the root cause analysis for us so we can explain to our users and explain to our stakeholders what happened here. And what happened here was, of course, a developer submitted a PR that wasn't had a typo because their environment might have order.dash-service, but in production, that

didn't match. So, this was pretty good. In 2 minutes, we did all of this, but could we do any better, Natan? Well, AI is fantastic at troubleshooting and fixing problems. We all know that. We all do that. But, it still needs a human to notice that something went wrong and to trigger an investigation. Not anymore. As you all know, Holmes GPT is the CNCF SRE agent, and

we recently launched operator mode. In operator mode, Holmes GPT monitors your entire cloud environment and any other data sources you've connected, on-premise virtual machines, network devices. We have customers that use Holmes with an entire stack running in their environment. And in operator mode, Holmes monitors the deploys from those on your teams using coding agents shipping multiple PRs to production every day. It catches the problems in production

as soon as they're rolled out and something goes wrong and before your customers know this. With operator mode, you can ship at the speed of AI and also catch problems and fix them at the speed of AI. That's a good tagline. That looks pretty cool. What else you got on your road map? We have a busy road map for Holmes GPT, but I want to focus on

just one item, self-mutating agents. Holmes today can connect to any MCP server, any raw HTTP API, even if it doesn't have an MCP, any database with our generic database connectors, and still people come to me all the time and they say, "Natan, can you connect Holmes GPT to Uber?" And I say, "Why would you want to connect an SRE agent to Uber?" And they say, "Well, so

we can send a taxi to pick up our on-call engineers." Now, I don't know that that's a good idea, but we know this that when that happens, I say to my dev team, "Create an Uber integration." They do what all of you would do. They go into their cloud code or Copilot. They spin up their coding agent. They develop it, and we ship it to customers. And

we started thinking, well, we're really not adding much value in that chain. We humans are actually just making the whole process slower. So, how can we take inspiration from things like connect yourself to Uber." So, it's giving an API key. Giving an API key. It writes its whole integration in safe environment and sandbox, and then it goes live without ever having a developer in the loop. We're

going to make a big announcement around this when it becomes available very soon, but from initial testing, it works phenomenal. Very, very good. So, to learn more about the road map, to see more about these demos, please check out the Microsoft booth at the Expo showcase. And please join the community. You can define what the road map is and what we build and how to shape the

future of Agnetic operations. We can't wait to see what you build with this. Have a nice