Navigating the Gateway API Maze: 40+ Implementations, 55+ Features... Beka Modebadze & Christine Kim

This talk focuses on the Gateway API, which was developed to address the complexities and limitations of the existing Ingress API in Kubernetes. The speakers, Becca from Google Cloud and Christine from Cisco, explain that as Kubernetes grew, the need for a more versatile and comprehensive load balancing solution became evident. They introduce the Gateway API as a next-generation API that facilitates Layer 4 and Layer 7 traffic management, highlighting its collaborative design and support for multiple personas, such as infrastructure providers and application developers. During the session, they discuss the migration process from Ingress to Gateway API, tools available for configuration, and the importance of conformance tests in ensuring compatibility and performance. Furthermore, they delve into the new release methodologies adopted to enhance the project's agility in response to community demands.

Thank you so much for coming on the very last day. Not the last slot, but pretty close to it. Um, I think there might be a few more seats and a few more people coming in. But don't worry people outside. This was recorded on YouTube. It will be uploaded and the slides are online on the schedule uh website as well. Um, so yeah, I guess we'll get

started. This is navigating the gateway API maze. >> Yeah, thank you for coming once again. I'm Becca. I work for the Google cloud. I mainly work on the GK gateway infrastructure and I'm also the release manager uh for the gateway API project. >> And I'm Christine. I am a software engineer at ISON at Cisco. I do some Selium stuff and some gateway API stuff. So yeah, so

first off, probably for some of the beginners. All right. Um we're going to set some of the context a little bit of what why we're here at this point. So if you're not familiar, the ingress has been a stable core API out of Kubernetes and it was easy to use. You know, it was really great for HTTP traffic. It was really easy to just declare something and

then you got traffic coming into your cluster from outside. But there was some difficulties with that too. You also had no portability because everyone with the features it kind of got complex um because h the ingress API was so simple but then was as Kubernetes got more uh integrated in everyday life. Uh then it got more complex as we saw it. Um different features also led to

different annotations and so declaring everything inside the annotations was a headache and a half. And then lastly one ingress object and many people touching the same object. It kind of led to some confusion. You know sometimes someone wants traffic to route to one way and then someone else did something else and then it's all broken. So that led to the development of gateway API which was I

think around CubeCon San Diego. So a while ago and this was like the next generation load balancing API and furthermore the cherry on top ingress engineext was recently archived as of two days ago. Yeah, still fresh. Um so this is now making the move to gateway API even more it makes more sense now. So this is the classic picture of gateway API. Um so yes next generation

load balancing of L4 and L7. So not just HTTP route and now because it's been stable for a while there's now rich ecosystem supporting your migration to gateway API from ingress. It's also one of the most collaborative uh API designs in Kubernetes history. There's a lot of great thought leaders who are behind it. So you know they actually think through the API and make it really detailed

and it's really familiar to the ingress in the sense of a controller. you know it's trying to match your desired state to what you want and also lastly it's not developed in core Kubernetes it's its own project which is really good because if you you know getting a keen versus getting a gapin I think it's a different timeline so you can see a lot of the feedback

with with the community and how features are developed faster and lastly the idea of personas so everyone touching the one ingress object now we have three different personas we have your infrastructure providers your cluster operators and application divider uh uh developers. And so on the gaps that you see on the gateway API website, they're usually referred to as Ian, Chihiro, and Anna. And so that has that

clear delineation between who can touch what and what kind of use cases you want. So that's really powerful. So what can you do with it? You can do something like this where you have your cluster operators creating a gateway API. And then let's say you have two application teams. one wants to route traffic to 90% of their version one and then 10% to their version two and

then you have another team that just wants to simple HTTP routing. So it's really great for that. Also there is this thing called gamma. I like highlighted it like that but it's declaring um east west traffic with your gateway object. So we all love not learning another set of CRDs. So this is really great for that usability. So actually this is the talk you know we solved

the problem. This is Yeah. Good job everyone. I lied to you. >> Yeah, we are not done yet. But uh as Christine mentioned, it solved a lot of problems, right? It solved um a lot of problems that existed with the previous solutions and gateway API is definitely a huge improvement over what we had before. So Christine introduced to us what was the personas right among personas one

of them was Jan and Jan is the infrastructure provider. So his responsibility is setting up Kubernetes cluster for multiple tenants with various rules and accesses including the networking solution. Jan did a little bit of research. He discovered the gateway API is pretty much the facto standard for networking in Kubernetes. So now he embarked on journey of applying it to the um their clusters. So what he needs

to do is before before we jump into some of the solutions I need to bring it up that gateway API is not just another open source project where you can download it you can install the binary and start using it. Um gateway at the core of the things is the collection of specs and conformance tests. So specs is the expressive API definitions and then conformance tests are

something that verify that all the behavior is doing what it's supposed to do. So in that matter um you still require some sort of controller or operator that takes your configuration applies to your cluster or clusters and then enforces all the rules. So Jan in this case has two options. The first thing is um he can take pretty much the specifications implement his own controller and go

from there run against the conformance test make sure it works but anyone if you talk to anyone that has worked on the gateway API controller including myself going to tell you that it's not easy thing to implement so it requires some time and resources so because of them he decides to go with the option two which is to choose one of the uh 40 plus available controllers

that are conformant with our requirements. And when I think about the yan, yan can have different type of requirements. And to simplify it as much as possible, I would like to group them into three different project types. The first one is the easiest. Let me use this one. Boom. Okay. The green field project. The green field project is something that you're just getting started with. Most likely

your requirements are relatively small, straightforward and I would assume that majority of them can be satisfied by the core functionality or core resources that live under the gateway API. Second one is the ingress people. So the ingress people as name suggests coming from the ingress they have some experience with it. Maybe they've been using whatever the flavor it was maybe they were using ingress engine x. Now

they heard that it's getting deprecated. We already said that oh no the gateway API uh maintainers update uh we archive the project so now they got to migrate to the gateway API and then the third one is the other so the other can be everything else that is left maybe you're already using some sort of controller you want to try another controller you want to try multiple

controllers it doesn't matter whatever it is so in that matter um we got to find the one and that's what the talk is about right picking among the 55 5 plus features available in gateway API um and then they have 40 plus um controllers there. So we live in an age of AI. If I have to recommend where to get started for sure go you ask your

favorite LLM. Okay, ask Gemini, Chad, GPT, Cloudy, whatever you're using, just say like whatever your requirements are. Do you want to get the example config for your cluster? Ask it can generate. You have a ingress config, you want to translate to the gateway API yaml, ask it. It does pretty good, pretty okay job, I would say, right? It's not fully exhaustive, but it's a good way to

get started with it. We're getting automated anyways in six months, so I don't know. But the next thing uh if you are more curious about if you want to I mean Jan is engineer just like a lot of us so the Yan likes to know what he's doing likes to be informed about it so goes one step farther from the chat GPT or Gemini uh and for

this one gateway API has very rich ecosystem including the documentation and guides and the tools um if you come to me and ask where to get started I would tell you to go the recent edition which is on our documentation page. By the way, everything will be linked on this slide so um later you can access it. And then this is the new thing we added which

is the controller recommendation or controller matching wizard. It's a very simple tool and it it's a attempt to solve the choices overload because if you go and list to all the features uh among all the resources that live under the gateway API um it's you're going to have a hard time deciding what do you actually need but everything starts with first of all picking uh what kind

of resource do you want to utilize right and the first you come here you choose the version minimum version so we always display the last three versions couple of weeks ago we released 1.5 and And I mean the reason we do this one, we don't want people going all the way back when uh gateway pie was not even a GA and like picking some weird stuff out

of there. So all the recent stuff you pick your version after you you pick your version you go and you select the resources right and HTTP road uh route and then gateway for example these are the core resources that are most heavily utilized and then first you select the core features. So, and as you see there's a must have and nice to have thing, right? So, the

must have is something that uh controller picks the matching um wizard picks the matching control based on nice to have is additional tiebreaker or the way to rank those controllers. So, uh you select the core features and after that one it will open you the window that going to give you the additional extended features. So if you're not exactly sure what kind of functionality you require, I'm

going to tell you just select the core feature there and at least guarantees all the core functionality that comes with it. Right? If you look under the core feature itself, there's the six features mentioned. These are the standard features that come with the HTTP route. For example, it's the same thing with the gateway uh gateway resource itself. After that one, uh if you're not sure about the

extended features, but you're consider you're curious, right? uh you want to know more about what are these functionalities that come with this resource, you can always click on the question mark by bringing the mouse pointer to it. It's going to give you additional description about it. If you're not confident that is something that you need, you can always select as a nice to have. So, um another

thing to do and finally you click match. It's going to display all the controllers that support those features. Um, so it's going to give you the uh once uh how many musthaves are matched, how many nice to haves are matched and then how many of them are missing. Uh good addition to this one is on the project it is uh linking to the documentation of that specific

controller. So if you like any of those just click on it, go to their documentation page and get started Uh if you if you want to go even deeper than that, we have what's called the comparison table or features matrix. This one has the full list of all the features and controllers they support uh organized in the in the in the version. So you can explore one

by one which controller supports what feature and um yeah spend more time on it if you want. But at the core of the things first of all I think the question is to ask like why we are supposed to trust this right and where it is coming from. Both of the tools that I just displayed is backed by the what's called the conformance reports. So any project

that takes the gateway API specification and implements them. They're supposed to run the conformance test suit against it and then it generates the report. they will upload the report that the gateway API repo and the report also includes the steps how to reproduce it. So any any interested party can go fork the repo follow the steps and you can reproduce the results. If they are not reproducible

I mean bad for them. So um and this is the example of the report right if you see on it on the report it gives you the name of the project it gives you the version of a gateway API it also lists what kind of uh profiles and then tests have been run how many them of them passed how then how many of them have been skipped

how many of them failed good that this report doesn't have any passed or skipped and then uh we are automating a lot of things it serves as a source of truth for these um various controllers and projects objects you can pick from and we are automated thanks to the new addition of a new field into the gateway class status. It's a field for the supported features. Um

it it lists like if you see here it lists what kind of features are supported by the controller and then it it use cases currently two. First it's um simplifies and automates a lot of things in the conformance test suit. In an old version of how you would run performance tests was you uh pretty much take the comment line you indicate all the tests you want to

run all the tests you want to skip. So it was very tedious manual work. So uh thanks to this feature it automatically infers what kind of uh features are supported by the controller and then matches to the test it's supposed to run and generate the report. So it's automated process um and also used by if if it's if it's a project that is already using the gateway

API and you're not familiar with this one just check the gateway class status there what kind of features are supported by your controller before you move somewhere else and this is the guarantee of portability right configure your controller once and then move wherever you want to move it and uh as I mentioned like if this is a good example here if you see something like the HTTP

route this is this is the promise for the core functionality everything that is under the standard functions and the rest is the extended functions that you may want to have your uh next Yeah. So beside that there is also a bunch of tooling that Christine going to talk about. >> Yeah. So there's some two there's two tools that I really want to highlight here. Um they're both

supported by SIG network. So it's like not some random tool that you go download off the corner of the internet. It's like super sketchy. So the first one here is ingress to gateway. Um, this is something that will take your ingress resource and then you just have to provide your provider and then it will spit out the appropriate gateway API CRDs for you. Um, so it's really

easy to use. This is pretty much what you need to do and it you don't like it doesn't do it automatically into your cluster. So it'll just print out the YAML for you. And if you want to read more about it, there's a Kubernetes blog that Becca and Steven published hot off the press last week. So if you want to see more of how the parody between

ingress engine X and gateway API for ingress to gateway has how it works then you can check that out. And then another one that I really like is gateway cuddle. Norwin has done a lot of work to make this way more usable. And this one's a really good one to see and visualize how your resources interact with each other because now instead of one object you're dealing

with more. It's kind of complex and people kind of get intimidated by you know how many things are interacting with each other. So it's really good for like you know outputting the statuses what describing what is what. Um but something that I do want to highlight that I think is a cool part about this is that it will output a diagramraph as well. So you can see

how resources are interacting with each other. So you know like oh this policy is attached to this resource and it's also affecting this and it trickles down to somewhere or like these services are all referred to in this certain route. So I think that's really good. >> Yeah. Okay. Um so pretty much we highlighted what's happening currently in the gateway APIs ecosystem but this is obviously not

everything. There's a lot of development a lot of conversations going. It's a heavily communitydriven project and recently not only recently year after year we've seen a huge growth in interest and people trying to not only contribute but ask questions. Uh they're coming up to our weekly meetings uh creating issues. they want to see a lot of new functionality and features into the gateway API which we are

very happy to listen to. Um but before I before I move there I want to I want to talk about the improvements we made on our uh release system. So again we changed our release system to cater to this increased demand uh to make the project more agile to let people experiment with things uh and see what holds and what does not and it starts with the

first step is write the provisional gap okay so it's it's it's extension proposal if you want to see certain functionality in the gateway API um write the proposal for it right it's going to be reviewed by maintainers after that you implement this gap And when it is implemented, we're going to release an experimental channel. Um, couple of words about the channels before I proceed with this uh

diagram how things work is that um we changed how we released our standard versions. Before that, pretty much it was the featured driven releases. So we knew exactly not exactly we kind of knew what kind of features we want to see in a new version and then we had to wait for the engineers that implement merge the code and test it properly and it was taking time.

uh now we really we decided to go towards the release trains which means we set out the date we know the next version will be released on this date. It urges also the feature owners to push their code on time test it properly because if you miss out the deadline pretty much have to wait another four months for the next releases and it it allowed us to

la in the last release I believe it was five features that we moved into the standard. >> Yeah it was hefty. Yeah. >> Yeah. So it was the highest number of features that moved from the experimental into the standard. So um on the first attempt of doing this new release system already showed us the good results and what is the experimental now? So the experimental I mentioned

like we want to be more quick and agile. So experimental is called the monthly release. So every month we release the experimental channel. So you get the chance to get your feature into the experimental for people to try it out. And the following process is we wait automatically for 6 months. So if no there is no interest or action on this feature for six months it's automatically

gets dropped and the room is open for someone else to put their feature. If uh started people started widely using it, we look at it. Does it work well? Does it uh does it guarantee what it um promises to do? Right? If everything is well with the next release, it becomes a standard channel, right? If if there's some issues with it, okay, we ask the owners of

the feature go back like fix it uh whatever and then uh try again. So if if they don't fix it or we cannot fix it, obviously we're going to remove the API. We don't want the broken API in our system. um if it's fixed then back to the standard channel and we also got rid of the beta beta versioning so directly from alpha toga so not too

many hoops to jump through and finally yeah so this is regarding the releases um again I want to urge any one of you if you have any feature on your mind you want to see in the gateway API please please please come to any of the slacks or or or the you open the issue or reach out to the maintainers and we're going to go from there.

Another thing uh before we close on our talk today is so far we've been talking about what's happening in gateway API like vertically but there's also development horizontally. So we are expanding uh with some projects that are adjacent to the gateway API and I want to mention two of them because there's only two of them. So the first one is the inference extension. So the inference extension

is the all the or you would say uh G gateway inference extension right. So gateway inference extension adds additional objects or resource to the gateway API that that is specifically designed for optimizing your LLM workload or inference uh workload with the primary feature of bodybased routing. So what the bodybased routing does is it can extract the model name from the request body and based on that determine

how to u redirect, how to route, how to load balance or how to prioritize if it is a uh latency sensitive uh request. Uh and good thing about it so if I have to summarize with one sentence, it is a smart load balancing for your inference workload. And the second one is uh which is currently in development right now. It's called the cubagogenic networking. And then uh

it's about the agents right. So this one defines a lot of policies and rules. It implements the MCP for agent to agent agent to LLM model agent to tool interactions. It comes with uh telemetry out of box uh to monitor your agent utilization um for authentication policies and various stuffs. But the one thing I need to point out about the about this project is they are implementing

uh reference implementation for this one. So when I said gateway API conformance tests, this one also has pretty much a control that you can use. So if you go on the link which is the quick start link, you can try to you can send up the sandbox and try working on it. Um and let us know what you think close. >> Yeah. Uh so what's next? Uh

call to action always like help us help you. We know that you guys have to migrate to gateway API eventually. Like I know a lot of you are on ingress engine X and you just don't want to switch over to another ingress implementation. Um we are really yeah developing fast. We love the back and forth between the community seeing what you guys need. Um we have lots

of guides online. Whoa. And then we also have the yeah the controller wizard. Try that out. Um yeah and get involved. We have two uh meetings as well. There's one that's North American friendly and then one that's EU friendly and it happens by weekly for gateway API. So come out to those. Um and we are a friendly group of people I have to say. So yeah. >>

Yes. Slack channel. This is the link for the controller matching wizard if you want to get started with. We have a bunch of guides and these guides are handwritten by the maintainers. Okay. Handpicked, verified and written by maintainers. So, uh, they put a lot of work in it. Let's let's, um, try to take advantage of it until the LM takes off there, too. >> And then, um,

feedback. Yeah, give us a >> Unless you didn't like it, then don't give us feedback. >> Want to feel good about ourselves after this one, right? >> Give us a wind. Um, but yeah, I think we might have some time for questions around the front here. Um, >> yes, of course. >> Ha. Can we expect at any point in time the CRDs for the stable release of

G2 API to be bundled with Kubernetes releases or is that always going to be up to the operators to maintain install those check compatibility with versions both with those and the controllers like as low bouncer controller etc. So the question is if if the project remains as a standalone project like uh I have not heard any conversations where we are trying to move gateway API uh under

the core kubernetes but there are a lot of talks about uh migrating let's say gateway inference extension into the upstream with the gateway API so instead of being a separate project pretty much um uh all the inference extension resources has to leave under the gateway API. Uh but this still to be confirmed and happen. >> Do you expect operators or people that create these operators to bundle

the gateway API CRDs on those deployments? >> Yeah, that's a good question. Uh we don't because if you if I give you the example right now, gateway API has over 50 implementations. Last time I checked, inference extension has over 25, right? So um yeah, currently they exist pretty much separately but if it is merged um I don't know that's a good question. more concerned about controllers like

are those the maintainers of controllers that are going to use gateway API >> if they are going to be the ones distributing the CRDs with their own deployments for us operators of the clusters to have those bundled together know that the compatible rather than we have to maintaining not only the controllers for bouncers whatever it is plus the CRDs for gateway API and as we upgrade versions

of Kubernetes making sure that all of those are compatible. What do you mean by maintaining CRDs? >> So we if I pick up a random a brand new cluster uh of Kubernetes 133 or 64 35 they don't have gateway API CRDs on them. I need to deploy those into the cluster and then I need to pick up a controller to manage something that goes it sends the

traffic into the gateway >> Yep. >> Right now those are two things I need to install on a cluster. >> Yes. I wonder if at some point those CRDs of gateway API come bundled with the chronic institution or from the the controllers themselves >> or you can tell customer to install themselves. So that's the third solution >> second >> you can tell customer to install CRDs >>

the operator of the cluster I'm responsible for that. >> Uh yeah yeah so that's that's the that's a typical work stream there. Um, that's what we're doing at the GKE2 and so far we don't expect that to change. >> Yeah, there's no hard and fast rule about that, but I do see like I can see how that would be a pain point. So like come and raise

a GitHub issue. We'd love to see that because it has been talked about before, but no one's pushed it hard enough and we're like okay. So from my testing what I've seen is some controllers are tested against a very specific version of gateway API which means if I install one that's either newer or older might not actually work with those controllers. So it means it's a bigger

overhead for us operators to make sure that we are tracking those things. If those were either provided with cluster versions that means that controllers will be matching those versions or if the controllers themselves bring the CRDs as part of their own CRDs they are tested against that versions >> but I will raise up issue and we can discuss that. >> Yeah, thank you very much. Good >>

Thank you very much for your informative presentation. It solved a lot of questions for me at least. Uh is the old ingress API and the new gateway API mutually exclusive on the cluster or can they run in parallel? >> Uh they are they can run in parallel. Yeah. So but like ingress it doesn't have any more like it's frozen. There's not going to be any more features

added. So like if you are not using ingress engine X then you can keep it in there. >> start moving to gateway. >> Yeah. that like to gateway. >> Yes, we gave you enough research to get >> Yeah, that's hard because F5 still doesn't support it. >> What is not supported? >> F5. >> Really? Okay. >> Oh, apparently it does. >> thank you. >> Okay. >> Sure.

>> Uh, thank you for presentation. Uh, what's your stability guarantees for gateway API? Can we expect uh uh to have few versions of gateway API installed and uh CRDs I mean uh installed and be not conflicting with with each other on upgrades >> uh not conflicting among the versions of gateway API. Yes, because you release new versions and I don't know how well they are aligned with

Kubernetes releases but as a operator maintaining platform for other guys and kind of have obligations not to break stuff deployed and they have no means to fix it. So I need to deploy in a way which will won't break things for them. >> Uh there may be breaking changes introduced in the minor version updates. It it is a possibility. I'm not saying it's going to happen. Um

but uh especially when we're moving a lot of features into the stable >> something let's say we are on the stable track only sorry channel only stable >> uh there shouldn't be any major breaking >> are you committed to this >> no breaking changes in stable ever according to right there. >> Thank you. I got my answer. >> Thank you Nick. >> Okay. I guess that's it.

>> Yeah, we'll be around. Thank you so >> Thank you very much.