Gateway API: Bridging the Gap from Ingress to the Future - Panel

This talk covers the transition from Ingress NGINX to the Gateway API, focusing on the recent developments and features in version 1.5. The speakers, who are involved in the development and maintenance of these projects, discuss the deprecation of Ingress NGINX and the importance of migrating to Gateway API. They introduce a command line tool called Ingress to Gateway that aids in this migration process and highlight key new features like listener sets, TLS routes, and improvements for non-HTTP workloads. The session emphasizes the evolved capabilities of Gateway API, including enhanced TLS management and the ability to support various backend protocols, thereby addressing more complex use cases in cloud-native applications.

Okay, everyone. We're going to get started cuz we got a lot to get through today. We have very packed agenda. Um those of you still coming in, please come in, find spots, don't feel bad. There's heaps of spots down the front. Stage stage right, your left. Anyway. Uh okay. So, we're going to get started. Cool. Uh welcome to Bridging the Gap from Ingress to the Future. Uh

we are uh talking about Gateway API and Ingress Um thank you all for coming. This is pretty crazy. Biggest theater I've ever done. Most full I've ever seen. Um so, yeah, thank you all for coming again. So, uh we will get started with round of intros. Um my name is Nick Young. I'm the leftmost person on the slide. Uh I work for Isovalent at Cisco, and I'm

a maintainer on Gateway API. James. >> Hi everyone. My name is James Strong. I'm a maintainer for Ingress NGINX for about another hour, and I work for Isovalent. >> [applause] >> Hi everyone. I'm Rosty, technical manager at Google, primarily focused on cloud load balancing, and also Gateway API contributor. >> [clears throat] >> Hello, I'm Katarzyna. I'm engineer at Google, and I'm working with Gateway >> Hello everyone.

I'm Norwin. I'm working at Airlock implementing Gateway API, and also contributing to the project. >> Cool. Thanks, everyone. So, today we got a few things to talk about. Uh James is going to run us through the what's going on with uh Ingress NGINX's uh deprecation and archiving. we will talk about I'll talk a little bit about uh what we're what Gateway API has been doing to make

the migration a bit easier. Then we'll go through a bunch of the stuff that we've added to the recent uh 1.5 release. Uh and uh Norwin will run us through uh funky new uh controller matching wizards that uh that Baka built for us. It's uh pretty sweet. So, uh but we do only have 30 minutes, and we have an awful lot to get through. So, we're going

to go pretty quick. Uh we will try and save some time for Q&A at the end, but if not, uh we'll be available outside for questions afterwards. So, over to you, James. >> All right. Let's go ahead and get this started, and it doesn't look like it's switching. Okay. Uh I wanted to do a quick run-through of just like some of the numbers. Uh it was really

hard actually to find the first commit for this because Ingress NGINX originally wasn't in its own repository. It was in contrib, and it was just known as I think like Ingress for a while, and then they didn't like that name, so we changed it. Um but it's about 8 years old. We've had 275 releases. Uh I think I've got maybe half of those, and it doesn't really

count cuz half are Helm charts. Uh a lot of contributors there, about over a thousand, so thank you everyone who's helped contribute to that. Lots of closed issues. Sorry if I ever closed your issue and never fixed it. Uh lots of commits. Uh I'm sure that fork number went up pretty high after November. Lots of pull requests, and it was pretty popular for a while, so I've

had a lot of people come up and say thank you for what you've done. Thank you for the project and maintaining it. Um so, thank you for that, and it's been a lot of fun. >> [applause and cheering] >> It It's not just been me. There's been lots of other people, so you know, Ricardo down there, so lots of people to thank. I feel like I'm, you

know, winning an Oscar. I don't So, we've got some final releases out there, so I don't know if some of the astute folks in the audience have noticed. We I put out a notice, I think, on Wednesday or Thursday of last week about a final release, and then we released another one on Thursday. So, that kind of gives you an indication that we had a new CVE

that was in the queue that didn't get fixed. It got fixed the same day. Thank you to Tabitha from the SRC and from uh Steven Jen from uh Microsoft for helping out with those. Um so, without those people, we wouldn't have been able to get out, I think, the eight or 10 CVEs that we've had in the past 3 months. As far as the deprecation is concerned,

the SRC will remain the CVE numbering authority, so they will still issue CVE numbers, but they just won't be fixed. And I can tell you right now that there will continue to be issues found. Folks will still find them cuz folks are still running Ingress Nginx, and I can't say this enough. I think I've said it probably 50 or 60 times in the past month. You should

be migrating off Ingress Nginx. There will be CVEs found. They will not be fixed by the Kubernetes org. Um a fork might fix them, but again, they won't be fixed by the main contributors anymore. We'll continue to work with the rest of the Gateway API community on working on Ingress to Gateway project. I've got some more numbers that we'll talk about that. Um again, I like to

talk over my slides, so we've already talked about the security piece, and we will be archiving the project. So, it's kind of a rehash. I've just been saying the same thing for a while now, so I hope everyone hears that and sees it. When we talk about the feature set, there is a Folks use a lot of annotations. We've got about 137, and we're looking at the

numbers. There's a spreadsheet here that we've tracked, we've updated for 1.5, so you can scan that, you can watch it, you can look at what you're using, compare it to what's available in standard and experimental. And if there's something that's missing, let everyone know, come to the community meeting, work with maintainers, work with an implementation in getting those feature sets. So, we've got about, I think, 45%

of the 137 that's there. Uh there's a lot in there that's got no plans. There's no gap, there's no conversation, so again, look at what you're using, what you're trying to accomplish, and go with the implementation that supports it supports it the best. As far as we'll talk about Ingress to Gateway further on, but looking at the feature set there, right now of that 137, only about

27% of those features are in Ingress to Gateway to help you. So, it's a migration tool. We'll talk more about it. Um but yeah, there's a lot of things that aren't in Uh I do have one fun thing to do before we go. Let's see if I can do this. Uh it's really hard to do now. Is it on the left or the right? >> It's on

the right. >> Oh, there it is. I found it. >> Who's nervous? >> It's hard. >> Okay. Thank you very much, James. Uh I was really hoping that we would be able to do that live on stage. That is awesome. Okay. So, yeah, I can talk about >> Once if you have any issues with the Helm chart, please let us know in Kubernetes or Kubernetes Slack. Thank

>> No worries, man. Awesome. so, yeah, uh what are we doing to help? Well, the main thing James already mentioned a little bit the the main thing that we're doing to help uh which is uh Gateway API has been sponsoring the uh Ingress to Gateway project. Ingress to Gateway is a command line tool that uh will help you convert ingress objects from anywhere to go API objects.

It actually also will do some other stuff as well that I'll get into in a sec. But the other thing that we have been doing is for our recent releases, we have put a lot of effort into prioritizing features that we knew would help people with migrating away from ingress engine X. Listener set is one of them, but there have been a few others as well cause

and some other things. So we pushed those forward, you know, a very actively to try and make sure that people have a place to you know, a new home to go to and that you don't need to keep using ingress. Ingress is a fine API that is just not just does not capture a lot of the more advanced use cases that people expect in this day and

age. So, ingress to gateway. As I said, it's for migrating stuff. It does have specific ingress engine X support as James mentioned. So you can literally take an ingress object with ingress engine X annotations on it and the ingress to gateway will convert those things to the correct gateway API stuff for you. It supports a bunch of other sources as well. So if you're coming from a

specific implementation, a lot of implementations have actually added their custom stuff in as a source. And some implementations have also added support for export as well. That is one of the new features. So ingress to gateway has hit version 1.0. Congratulations to Steven Jin who has been maintaining this tirelessly in the last little while and made a lot of changes. Can I get a round of applause

for Steven and ingress gateway making 1.0? >> Thank you very much. Um so yeah, we've got So the biggest thing is one of the other big thing is that we have emitters now that can do So you can do gateway API vanilla or like various flavors of gateway API with implementation specific things. So if you have something that's covered by a specific implementation, you may be able

to just use this tool to move straight to them. So yeah, thanks to the implementations for helping do that and again, massive thanks to Steven for coordinating all of that. He has done a a of work in a very short amount of time to make it happen. let's talk Gateway API 1.5. We released Gateway API 1.5 uh a few weeks ago. Um it has been a long

uh this one's been a pretty crazy release. Um we actually moved to a new release process. Uh we have moved to something a lot more similar to the upstream Kubernetes And uh so that includes we're sort of moving more as a uh release train. We do have monthly releases. So those are you know we tag the the tip of head uh at the start of the month

and that is the monthly release. Uh so yeah, for implementers, that's what I encourage you to use. Um for I would not encourage users to use that. Um the a lot of the time the implementers won't have implemented the stuff that's in the monthly release yet. However, uh we do do our regular semver releases, but one of the other things that moving to this has been uh

has let us do is start picking like specific dates to release and whatever's ready at that date, that's what we ship. Um so it has meant that uh we've had a lot more people focused on it with the with the deprecation. Um but yeah, so this is the first release to use this process and yeah, five features moved to standard, that's stable {slash} GA. Um that is

the most we've ever had. Um so yeah, um thank you to everyone who has put a lot of time and effort in to get all of those things moved over. Now, thank you. Go. >> Um so one of the biggest ones that I'm going to talk about so here are all the features. Uh listener set, TLS route, uh Gateway client certificate validation, uh certificate selection for TLS

origination, and uh CORS. And so um like all of these are pretty big features. We've done a lot of TLS work, as you can see. Uh there was a bunch of stuff that we needed to tidy up for TLS. and [clears throat] actually listener set, which is what I'm going to talk about, was one of the things. So, yeah, so listener set is a new resource that

allows breaking the listener part out of a gateway API object and having it in a separate object. Now, the important thing about the listener struct that we have inside the gateway is that that's where the TLS settings live. That's where you specify your certificates. One of the biggest complaints we had from people coming from Ingress was, "Hey, why can't I put my TLS details inside a resource

that application developers have access to?" Because when we were building this originally 7 years ago, um you know, TLS uh a free and easy to get things like they are today. Thank you to Let's Encrypt for their fine work. Um yeah, uh so we designed it to sort of reflect the reality that, you know, cluster owners own certificates uh and they sort of, you know, you wanted

to hide that from your application developers and say, "Don't worry about that. That's not your concern." So, the sort of the new world that we've arrived in has times have changed, and so we needed to change things. And so that was what we built uh to listener set is a separate object that lets you do that. And so there's two the two features that it gives you

are, as you can see, moving you can have to you can delegate the ownership of TLS out to application developers um without giving them access to actually update the gateway, which, because it often creates load balancers, can have a cost implication. Um but the other thing that you can do is use it to have lots of listeners. Now, people have continuously asked me, "Hey, why can't we

have Why can we only have 64 listeners?" To which I usually reply, "You 64 listeners should be enough for everybody." Um >> but um the uh but so but practically there are reasons for that. It's about to do with the size of a single object in etcd. Uh and so we needed to find a way to split that stuff out, and listener set turned out to be

the the way to do it. So, we've already got six implementations supporting this. It has moved to stable. Um so it is 100% go for everybody to use. Please start using it. Please give us feedback. Um you know, if you find any problems, log an issue, tell us immediately, and we will do our best to fix it in as soon as we can. Okay. So, I think

with that, uh, I am going to hand over to Rusty to talk us through TLS route. >> Thanks, Nick. >> Nice one. >> In my turn, I am delighted to welcome non-HTTP workloads to Kubernetes Gateway. What are those? Well, same old Redis, MQTT, Kafka, database is over TLS, MongoDB, PostgreSQL, you name it, and actually anything else based on TLS, including custom and proprietary protocols. You see, traditional

Kubernetes Ingress was built around layer 7. And so far, Gateway API was mainly focused on HTTP routing. We introduced TLS route specifically to address this issue. TLS route is mainly focused on routing based on TLS metadata, and it's not interested in properties of a higher-level protocols like HTTP. It is mainly focused on SNI host name present in TLS handshake. And now, Gateway has two modes, pass-through and

terminate. Let's dive into and see how TLS route works and interacts with both of them. when to use terminate mode? When you want to take advantage of a centralized TLS certificate management on a gateway. When client establish a secure connection, it sends a client hello message. And in this message, it tells a host name it wants to connect. The last session is fully terminated at the gateway.

Gateway provides a certificate and decrypts the data. And then gateway sends the raw TCP payload unencrypted back to the backend. When to use passthrough mode? Well, well, you have strict security requirements. For example, your traffic must remain must remain encrypted end-to-end before it hits the end destination backend. Or for example, your external client and the backend need to authenticate between each other. Same. Client initiates a secure

connection and tells the host name it wants to connect to. But the truth is is that SNI host name is unencrypted. So, the network proxy can read this unencrypted host and decide which backend to route traffic to without looking into the encrypted payload. And this is establishes this end-to-end TLS between the client and the backend. And in this mode gateway has zero access to private keys and

has no info what is in this underlying Please remember TLS route was designed for non-HTTP workloads. And if you end up using it for HTTPS you might be vulnerable to some of your HTTP request being misdirected because of HTTP connection coalescing Katarzyna going to be talking later about. You need to be very careful the way your certificates are issued. And now, handing it to Katarzyna. Thanks. >>

Thank you, Rusty. I'll be talking about new feature mTLS that we introduced in gateway API for HTTP workload. At the beginning maybe I will say something about terminated TLS. So, gateway is acting as a man in the middle in this case and terminates the connection. Fronted mTLS is the case where client starts originate the connection to the gateway. In this case, gateway acts acts as a server

and presents server certificate. Then the connection is terminated and once the TLS of course is established successfully, then the connection is terminated and gateway here acts as a client and originates new connection to back end spots, which are servers. In previous release we added the authentication part here that gateway is able to authenticate the servers from the back end service. And in this release we added full

mutual configuration for gateway. For fronted mTLS gateway is able to verify the client certificate that is provided by users. For back end mTLS we added the configuration for certificate that gateway is presenting and this is a gateway's client certificate. Then it's worth to mention those connection coalescing issue that Rusty briefly mentioned because it really impacts how we shape the API here. Connection coalescing this is a feature

that was introduced in HTTP 2. To actually this is a performance feature that once the TLS connection is established, then this connection can be reused. And this connection can be reused when for the destination of the same IP and port, and uh domains in gateway which actually has the uh certificates that overlaps. And there is the example of the gateway. Uh I have two listeners here, uh

which are using the same port. And there is the TLS configuration. This TLS configuration, this is the uh server's certificate, but I wanted to show the host name. So, we have Here we have very specific, for example, that car host name, and here we've got a wildcard host name. At the beginning, we thought that we uh want to put certificate client certificate configuration in those TLS struct,

because it would be you know, consistent with the current API. But it turns out that uh once the connection is established for one of the listener, it can be reused to to all of them, which are listening on the same port, and has overlapping uh wildcards. And what is the issue here? The issue is here that when connection is first established to the wildcard uh certificate to

the listener with wildcard certificate, then uh we are validating a client certificate against wildcard known well-known CA certificates. Uh but it does This validation will pass, and TLS connection will be established. But once a client will reuse this connection to access the second listener, the problem is that we are not validating client certificate anymore, because connection established, and we don't can expect SNI in this case. Uh

so, we are not sure if this uh client who established this connection is, for example, that car bar example, that car, or something else. So, we decided to take a different approach, and to move TLS uh configuration for this feature to the gateway level. So, we have this default configuration where which applies to all HTTPS listeners in the gateway. Uh and the configuration for client certificate, this

is a list of config maps which contains PEM-encoded CA certificates. And to introduce more granularity for clients who wants to diverse their configuration, we allows for per port override. Uh that means that this per port configuration will be applied to all HTTPS uh listeners which are listening on this on this port, and the host name that they have set set doesn't matter. Uh this allows us to

provide, you know, secure end-to-end connection. Uh also, we introduce these uh modes for um TLS configuration which allow valid only or allow insecure fallback. This is in uh extended, I would say, channel. Uh this allows us to, you know, testing the configuration and not be so strictly with rejecting the certificates. Thank you, and I will you know, Norvin hand it Norvin for backend update. Thank >> And

Scott Alicia >> So, let's move on with uh backend mTLS. So, to enable TLS on uh connection from a gateway to a backend, you have to configure a backend TLS policy which is targeting the Kubernetes service of the However, to establish a mutual TLS connection, the gateway also needs to present a client certificate during the This client certificate can be configured on the gateway resource. Uh you

can use the spec TLS backend client certificate ref in order to reference a Kubernetes secret. This uh reference will then be used and presented during the TLS handshake. Uh as you this certificate is specified on the gateway, this means it will be used whenever the gateway is establishing a connection with any of its backends. in this if you view view it like that, you can see it

as representing the gateway's identity. This concludes all the new uh TLS features that we have in 1.5 and we have one final thing, which is the HTTP route course filter. Course is a browser security mechanism that allows a backend to specify which origin other than its own uh is uh accessing its resources. In addition, for requests that might modify data on the backend, for example, like post

or put request, uh a browser is sending a preflight request in order to check whether the actual request would be allowed or not. By adding a course filter to an HTTP route, you can configure the course policy. The gateway will then use this policy in order to set the appropriate HTTP response headers and also to respond uh to preflight requests. But, keep in mind you're specifying a

policy. This policy may contain wildcards which are dynamically translated rather than the exact header values that being set. As Nick mentioned, we have a lot of features in this release, uh but there is one more thing. You're probably wondering, "How can I now try these new features? Which implementations support them?" So, let me introduce to the controller matching wizard. The controller matching wizard was designed to identify

which implementations support the specific set of features. I go and run a quick demo where I show the features of this release that you have presented and which implementations are already implementing them. So, let's open the Gateway API website and the matching wizard. Then we can choose a Gateway API version. Of course, it's 1.5. Then we select the features. So, for Gateway we enable the core features

and select all the new feature as nice to have. So, the backend client certificate, frontend client certificate and listener sets. We move on to the HTTP route where we select the course filter as a nice to have. And last but not least, the TLS Click on match. Review the results, choose one of the implementation, and try our new features and provide us with feedback. And with that,

I'm handing back to Nick for the wrap-up. >> Thanks, Noam, and thanks everyone for all those awesome updates. >> Can I get a hand for my co-speakers, please? Thank you. the last thing is the usual slide here. Um yeah, if you are coming from Ingress Nginx and you have complaints or like thank you feedback, either either are very welcomed. Thank you slightly more than complaints, I must

admit. Um but uh please please come and tell us. You can uh find us on Kubernetes Slack in uh SIG Network Gateway API. Um come to the uh you come to the repo, log an issue or a discussion, or um if you go to a Gateway API, that's uh dot six dot kates dot io, that's our website. There's a bunch of details there about our community meetings

and all the other ways that you can get in contact with us. So, um thank you again everybody for coming here We actually do have a couple of minutes. So, I had one thought while we were all talking. Since and Tabby, you came in late so you missed when James actually pushed the archive button on the repo. So, I would like one was wondering if we got

a pretty full theater. I'm wondering if I could ask everybody to let us record a video with all of you saying bye-bye Ingress Engine X. Are you cool with that? Is everyone Awesome. Okay. Okay. So, let's uh I'm going to James, can you give them the countdown? Yeah. Okay. So, everybody >> 3 2 1 Bye Ingress Engine X. >> Thank you very much everybody. Um please uh

feel free to find us and ask us any questions and we really appreciate your attendance. Thanks again. >> Thank you.