Advancing Kubernetes AI Conformance: Current State and Road... Yuan Tang, Mario Fahlandt & Janet Kuo

This talk discusses the Kubernetes AI conformance program, exploring its inception due to the rising adoption of AI workloads that present unique challenges for Kubernetes. The speakers, representing organizations such as Google and Red Hat, explain how existing Kubernetes conformance does not sufficiently address the requirements for AI workloads, prompting the formation of a dedicated working group. They outline the program's goals, which include establishing a standardized set of capabilities and APIs essential for running AI and ML workloads effectively. The transition from a temporary working group to a permanent sub-project under SIG Architecture signifies the commitment to long-term development and maintenance of AI conformance standards. Attendees learn about the tools and processes implemented to track conformances, including automated testing frameworks and a structured requirement lifecycle management system. The session concludes with insights into future directions for the conformance program and an invitation for community participation in refining these standards.

Hi everyone, welcome to the uh Kubernetes AI conformance talk. We are going to talk about the current state and the road map. So these are the leads of AI conformance program and I'm Janet Quo. I work at Google. >> My name is Yan Tang. I work at Red Hat. >> I'm Mario Farland and I work for Curomedic. And we are missing one person who unfortunately couldn't make

it here. uh which is uh Rita Sang for Microsoft. >> Cool. Uh I'll talk about the history of Kubernetes AI conformance. So how it started? Uh as AI introduced as AI getting more and more uh adoption, we see that AI workloads are introducing new requirements and new challenges in Kubernetes. There are GPUs. They are uh GPU intensive. They have synchronously scaled workloads and there they require specialized

hardware and networking and security and without a standard conformance and it's hard for us to um have a common standard for everyone to have the same experience. So we figured that Kubernetes conformance is not enough. There are parts outside of Kubernetes core and we want to iterate faster than the Kubernetes because Kubernetes conformance care a lot about stability and being able to um have a stable release

and a standard that everyone adopts. But with AI we want to move faster. So we started the Kubernetes AI conformance working group in 20ou uh 2025 to solve this problem and this is sponsored by SIG architecture. We had a original proposal for what AI conformance could look like and we proposed the different requirements. We separate them by should and must. So should means that and this is

a recommendation or a direction that the community want to go and must means that this is mandatory that all the platforms should follow this uh requirements. So what is included in the kubernetes a conformance? This is additional requirements on top of the existing kubernetes conformance. So basically uh a platform needs to be Kubernetes conformant first before they can be Kubernetes AI conformant and and also I got

questions about oh does it does it tell me the best practice of how I run AI workloads on Kubernetes it actually doesn't we uh we only put uh the limitation or enforcement on the platforms so they should follow the standard to provide the capabilities or APIs in the platform so that it can cover most of the uh AI workflow use cases and users are free to choose

however they want to use those clusters. For example, they can just use the clusters for training or just for inferencing, but the platform should be able to cover all those use cases. And our goal is to define a standardized set of capabilities and APIs for running AI and ML workloads. We want to make it uh interoperable so that you can get the same guarantee that you used

to get with uh Kubernetes conformance that you can run you can write once and run it anywhere and with AI workloads too. So I said that we were a working group but working group is temporary and it cannot own any code. So we are now transitioning we have transitioned into a a seek architecture official sub project. This is a permanent home for the program for the long-term

maintenance and for its uh evolution. This is following the same path of Kubernetes conformance and you can uh take a look at the repo that we have. We have all the design and the requirements and discussion and all the code for and the automated test for the uh for the requirements there. >> So next we all talk about like what we have accomplished uh so far. So

the program was initially launched at CubeCon North America last year and we were glad to stand uh on the keynote stage and announce the program and it was uh amazing to see so many adopters and contributors uh for the initial launch of the program. If there's a link to the announcement if you're curious about the details and we also added a separate category to show like all

the platforms that are certified for Kubernetes AI conformance. There's a link to the CNCF landscape. Uh the first link on the slide and the second links uh the second link goes to the GitHub repo. uh if you see uh any organizations missing from this list feel free to submit a PR and we've also added uh the checklist for conformance um for multiple Kubernetes versions from 133 to

135. Uh on the right hand side you'll see an example for the conformance checklist for 135. Here uh basically you add uh you fill in the information for your platform and vendors and also requ uh the evidence for each of the requirement. We've also transitioned all the requirement tracking from Google doc to a structured GitHub project board. So it's to easy to visualize the statuses of each

uh uh requirement we are discussing in the community meetings. And we also defined a process uh called uh Kubernetes AI conformance requirement or K process uh which is very similar to the Kubernetes enhancement proposal process uh so that we can use this process to manage the life cycle of all the requirements uh we have on the right hand side you'll see an example template uh where you

can specify the background for the uh and in context text for the requirement and the graduation criteria and testing plans and implementation and so on. So I'm going to talk about what's next and so for 136 which is the upcoming release we're going to introduce a few new requirements around inferencing. If you if you came to the um keynotes on Tuesday, you'll probably see me live demoing

those requirements on stage. And for 130 uh it's actually 137. I think I made a typo. Um I think maybe we can add some more requirements and run agents. >> We are going back in time now. We re rolling back to 1034 and stick there forever. No, just kidding. >> Yes. And but if you have more ideas, feel free to come to you know our meetings or

slack channel to discuss what's next with us. And then for the automated tests, we have a test there in a repo. Given that we can now own code, we have some test framework. It's really simple. No additional um wrappers, just simple go code for running test. And as you know that on each platforms they have different uh hardware or maybe software for example they might be using

different accelerators. So we have a uh a flag there at the end for specifying different accelerator types. So when platforms they certify uh for AI conformance they can um just provide their own specific accelerator type or different you know maybe in the future for the networking type and then and we they can still pass the test. And then one thing that I also want to be able

to leverage is that we can have more conformance tests for a those specific areas around AI conformance for example and for the DRA driver. Maybe we can have conformance tests for those DRA drivers so that we can uh in AI conformance we can just say the platform must support DRA driver that's conformant and we can delegate those task to the sub areas. So the biggest question that

has always been asked is what is a conformance program? Why do we have a conformance program? So who of you I assume every one of you is running Kubernetes in production. Yes. Are you using a conform Kubernetes conformant implementation of Kubernetes? Do you know where to find the uh the vendors or the applications that are Kubernetes conformant? Anyone? Okay. Good that we talk about this. So um

the CNCF or the Kubernetes project back in 2021 2022 said like hey we need a we need a program to say like wherever we run we we want that it does the same that it works the same that I can go to the hyperscalers and say like the Kubernetes cluster which is running in uh Google cloud is the same as it runs in open shift it's the

same has the same capability ities as it runs in Microsoft. Uh and that's was the idea of creating like a conformance program. The good thing for the Kubernetes program is that Kubernetes conformance program is that we just use APIs, right? We can just check if you can use the API and it's working. Um Juan showed the uh the nice slide where you saw like the landscape of

certified AI Kubernetes offerings. Next to this is certified Kubernetes offerings. So basically you can go to the landscape and figure out what distributions of Kubernetes are actually certified and then you can be sure that all of the conformance tests that are listed on uh API snoop.cates skates do something uh are fulfilled which is basically all API endpoints who move to a uh stable state and that's basically

the conformance program that was created the idea is with those conformance programs and that's now coming back to this is we want to create for end users and for companies a way how we can actually ensure that you can move between all of those vendors or even open source projects and you know that it will fulfill all of my requirements so that the industry can say like

this is a standard that we can roll out globally and you're not locked in into a specific vendor and the community works together and as you have seen like technically all of us are competitors and we basically work together to get all of the input to um have like a standard that defines the industry so that everyone can have like a a baseline to agree upon and

for example we also have Puja who helped a lot with uh with the whole AI conformance program. So basically we have like meetings of 50 to 60 people to give input from different sit and it's not only vendors we also have folks from Nvidia in there they have no Kubernetes platform but they still give the input to define those standards and there are like a lot of

little tiny bits included in this whole process that we get as much input as possible and um a conformance program can be created for any graduated project in the CNCF. So technically there might be the option that there are more um conformance programs coming up. Um and this is usually uh there are different parties involved in this. The project is usually the main party because the project

defines the technical details and the technical requirements that you need to fulfill this. So there's no influence from any outside world except of the project to from the technical side. Then we have the governing board of the cloud native foundation that is basically saying like okay we established a trademark of a conformance program so that is that is protected that it cannot be uh attacked or from

the outside like uh misused u so they vote on this and then when it's passed um we also have the project team of the CNCF where I want to say huge thank you to them because they are the poor people who need to review all of the uh all of the distributions that are handed in and basically need to go through all of the stuff and check

it if it actually works and if there's a conformance program which will overarch then the TOC is also involved because the TOC has a technical oversight over all of the CNCF projects. So you all now want to get in and help maybe. So basically um as you heard we are not existing anymore. Uh so the working group how who submitted the talk is not here anymore. Uh

because as mentioned we are now a sub project. Uh we still have the same meeting time and I I think we have new calendar. We have a new calendar invite because now we're on the calendar of the um uh sik arch. Uh so basically we meet uh bi-weekly. Um for European time it's usually 700 p.m. There are like two weeks in the year where we have the

weird thing with daylight saving time where everything moves and everything is crazy because the calendar explodes. Um this is done. I think next week. So we are back next week to normal to normal timings. Um we have a mailing list. Uh you basically follow the cigar arch mailing list where we also announce stuff. Uh there we have in the kubernetes slack uh slack channel that you can

join. Um we are happy to get input from everyone. Um we are especially looking for input from enduser companies because we are still I mean we can define stuff how AI looks like but we don't we also want to know who's actually and how you're using it and what are requirements on this. So we would be looking forward to get any input from uh any end user

companies what are your actual requirements so that we can also implement this in the standard conformance program and uh as also uh Jennet mentioned we have those cars um so you can basically go to the repository and uh look into the cars and help us to yeah just talk about if this is achievable because uh the major topic where we discussed the most was it might work

in a hyperscala environment but what is on prem what is on the edge what is like those specific environments that are not the norm and we also want to cover those and that's the most important bit and there we are always looking for for input and as mentioned uh we also would like to hear if for example a standardized DR8 a driver would be something that is

interesting for all of you so that we can also like push in in the different other directions to say like hey we would really really need this because this is interesting for us and with this thank you and we would have like a little bit more time for Q&A because I think there were questions. Thanks. Questions, please come up to the mic or I or if you

don't want to be recorded, we can also I repeat the question. >> I guess my question >> um I'm very new to pretty much all of this. I think our company is likely to be looking towards adoption in the next year or two. Is there any value in me joining this as anything other than just kind of a quiet observer? Do you need input from from people

that are that new to it? >> Uh we appreciate everyone coming to the meetings because as I said we started the meeting with 60 people in one room who had uh uh 320 different opinions. Uh but you saw that in the process of this um we uh because people shifted focus again uh we we we dropped also an attendee rate and um so basically I would like

to get as much input as possible because you can also basically give us an input to say like hey we are now looking into this and this is some use case that we might have not thought about. So we are always happy to have uh any people with any background in those meetings and um yeah that's you we we also have a lot of people who are

just attending the meeting to get in uh to get started to what what is this about? Any other question? If not, thanks for joining. Thanks for the interest and have a nice last day of CubeCon. You almost made it. >> Thank you.