Sponsored Keynote: From Cloud Native to Accelerator Native: Kubernetes as the Distri... Jago Macleod

This talk covers the current developments in Kubernetes, led by JGO Mloud from Google. The speaker discusses how Kubernetes serves as a critical platform for developers, emphasizing its role in enhancing velocity, flexibility, and scalability of application deployment. He highlights the importance of efficiency for platform teams and how the ecosystem of Kubernetes extends capabilities through integration with hardware providers. The talk outlines the phases of Kubernetes evolution, from container orchestration to becoming a distributed operating system for AI. Key topics include workload awareness, agentic applications, and innovations like GKE Labs and Open RL, which facilitate experimental development. The speaker conveys a vision of Kubernetes as a foundational tool for future innovations in both AI and traditional applications, fostering an environment of experimentation and scalability.

Hi, I'm JGO Mloud and I have the greatest job in the world. I lead Kubernetes at Google and it's never been more fun than it is right now. It's also never moved faster. As the creators and the most active maintainers of Kubernetes, I wanted to share some of what the community is working on and why and how it'll help you make some stability in the chaos. At

Google, we often write down a vision statement to help focus the team. Right now, our focus is on velocity and utilization. And I'll share why those two ideas are helping us shape the future. Kubernetes has this amazing flywheel effect that starts with developers, gets amplified by the platform team, and then gets accelerated by the ecosystem. And we'll talk a little bit about each of those. For developers,

it's all about velocity. And Kubernetes gives developers a platform that provides velocity, flexibility, and scalability without a lot of work for them. The platform team, it's about efficiency. And so you have to have operational speed, but also reliability and of course scaling. But the ecosystem is where the magic really happens. I talk about Kubernetes as the narrow waste of the hourglass of infrastructure, but it's not a

bottleneck. It's actually a birectional megaphone. And it's really interesting. If you work on a framework like Ray or Slurm at the top and you write to the Kubernetes APIs, you get access to all of the infrastructure providers and emerging hardware. And if you're a hardware provider and you integrate with Kubernetes, you are available to the entire ecosystem essentially for free. It's super powerful and this becomes this

distribution engine in both directions. It's really pretty awesome. Kubernetes emerged in a few phases. Phase one of course was about container orchestration and this was the big breakthrough. We started with stateless applications and then moved on to stateful and batch and more. And phase two is the ecosystem explosion that you see the 200 plus projects in the CNCF today. And that's that magic we just talked about.

And phase three we think of as the distributed operating system for AI. What does that mean? Operating systems really shift the focus to resource management and the extensible frameworks that let you run different kinds of workloads on the same uh underlying infrastructure. And then heterogeneous hardware is a place where hardware is really fun and moving fast right now. So we've been working harder than ever. Step one

was about uh re-imagining the Kubernetes relationship with the underlying hardware. And you hear about DRRA, that's what that's about. And we've pretty much accomplished that. There's a lot of work still to do. The people working on it will be upset with me for saying it's done. Uh but it's on great track. The next phase is to move into the scheduler space and let the scheduler understand the

topology of the underlying hardware better. So right now in Kubernetes 136, the workload API is emerging as this place where we're pulling in uh support for gang scheduling to schedule an entire job together, workload aware preeemption and the topology that we talked about. It's really exciting and I encourage you to check it out. So some of our core investments, we talked about topology and workload awareness, we

talked about DRRA, and then the agent sandbox we announced last in November has really taken off to provide the safety you need to run agentic workloads at scale and safely. But why do you care? What do you care if Google thinks about uh Kubernetes as an operating system for AI? Well, we talked about the changing landscape and it's moving really fast. So, decoupling the applications that run

on top from the hardware that run on the bottom helps you move forward and have some stability. So, let's talk about agentic applications. Uh, a v1 of an agentic application probably calls out to uh Gemini or claude or some model as a service provider. But maybe you want to fine-tune on some specific data you have or sensitive data that you don't want to share broadly or outside

your own premises. So the Gemma family of models from Google are have been highly customized across different domains from medical situations to translation to even talking to dolphins which I think is pretty funny and cool. And so maybe you want to try the Gemma models to fine-tune. Thinking machine labs came up with a product called Tinker and they implemented a really simple and flexible API for fine-tuning.

There's another project called uh Sky RL that runs primarily on Ray that also pivoted and adopted that API. So it's becoming somewhat of a standard. So like all of you, we at Google want to experiment. We created GKE Labs and one of the experiments is here. It's called Open RL and it's a way that you can develop your training loop locally on your laptop and then deploy

and scale it on Kubernetes in minutes. It's super easy and fun. You should try it out. Now, is this the thing that's going to win in the industry? Is this the thing that's going to take off? I honestly have no idea. But that's not the point. The point is that there will be dozens, hundreds, thousands of these things coming up every week. And you need the platform

that lets you experiment and innovate. And the key is that you can futureproof your own innovation. You don't have to make long-term bets right now. You can try things out. You can let your teams try things out. And so you can have this innovation but maintain some control over that innovation. And if you're one of the people that's approached me this week and said you're so tired

of hearing about AI, you just have more uh mundane problems you call them to solve. All of the innovation that we're building into Kubernetes today helps you too. faster startups, lower latency, the stability, uh it's all improving for use cases that aren't even about AI. So, I'm super excited. We see the future of Kubernetes shifting from being a platform for building platforms to being this nervous system

of for autonomous infrastructure. And I can't wait to see what we all build together. Thank you so much and have a great show.