Not Yet Another Envoy Implementation - Exploring Kgateway To Write Your Own GatewayA... Ricardo Katz

This talk explores the development and functionality of the Agent Gateway, a new approach to implementing Gateway APIs within Kubernetes. The speaker, a software engineer at Red Hat, discusses the separation of control plane and data plane in service networking, highlighting the advantages of developing a custom Gateway API implementation without the restrictions of established tools like Envoy. They explain the reconciliation process for Gateway API resources, the integration of various backend services, and the transition from existing projects to the new Agent Gateway. The session covers practical aspects, such as using Go clients for service interactions and designing terminal user interfaces to monitor configurations and messages between the control and data planes. The speaker emphasizes the importance of building flexible architectures while cautioning against using certain coding practices that can lead to vulnerabilities.

Welcome everyone. Uh my talk today is about exploring agent gateway. Probably uh you are seeing like this crossed K gateway stuff and a lot of people they are asking me like why you cross it. So apparently uh K gateway maintainers they decided to make my life harder and they decided to split the control plane uh two days before my talk. So now it's like it's part in

agent gateway to write your own gateway API back end. Uh I am Ricardo. I am a software engineer at Red Hat doing all gate API stuff. I was doing previously a long time ago in resin genex stuff before James decided to deprecate and archive that. Thank you James. Woohoo. I'm maintainer of gate API at Kubernetes. Uh I like Lego are playing traveling and making jokes at the

wrong moments. Usually that's what get me in trouble. And uh my soccer team Vorons. Thank you. Uh okay come on guys. So uh you want to write you want your own gateway API implementation right? So um I've been hearing that a lot like people saying yeah uh I don't like envoy I want to do Apache web server whatever. So I wanted to have I tried that like

I wanted to have an Apache HTTP as a back end uh with FreeBSD. It's very bad idea. Uh I did that with Injet because it was my comfort zone but uh you know you can do whatever you want. It's your time your Gemini license. uh I do not want my data plane accessing kubernetes API so that's a trauma that I get from ingress in ginex right so

uh one of the things it's like uh when you are developing a gateway implementation or something that deals with the traffic coming to your cluster you don't want the same pod that access your API server to be exposed to your users otherwise you're going to get cve uh and I want to make it easy and reusable for other backends uh that's something that we tried with in

gape uh I'm Not sure how many of you know about or have heard about ingate the other project that we kind of archive it but we tried to do something like that which was like some generic control plane for gateway API. Uh so creating your own gateway API implementation is actually easy you know like if you just seriously you just ask you know some AI after all

like we are just doing AI uh to implement that and you ask like okay I need a gate API controller and um I just need to reconcile gateway classes listeners HTTP route put on the data plane and done right so that's easy and actually uh let me put this thing here because I wanted to make it easier so So you can see uh I asked Gemini to

I'm not sure if you can see that but I asked Gemini to hey just draw a you know a workflow of how gate API reconciliation looks like and it's it's super easy you know like you just need to put the right code on the right place and it's done it's usually how it works uh but the result is always almost the same so um I'm not sure

how many of you are familiar with actually how you know in gx hroxy envoy all of them they work but usually it's this is what you get as a result right so you get some listener some port binding and you get some virtual host configuration and those virtual host configurations they have attached you know routes like I want to send this path to uh this back end

and this path to this back end I want to use TLS on this binding port or not and that's how it works right so in gx you have this kind of configuration please don't don't try to read that it's just like you know I'm just giving examples haroxy or envoy envoy is beautiful right so you have haroxy like this one and avoid with this massive API but

I it's in the end it's like the same thing you have listeners you have filters you have ports and done and then when we were discussing about uh making you know some evolution on GX or or getting some better approach for that and gateway was created and void gateway approach was kind of uh you know exciting because it was a gateway API implementation that reconciles gateway API

resources and you have a control plane that generates envoy configuration. I'm not sure uh if you ever dealt with envoy configuration but you can have you know this server generating configurations generating uh you know the envoy configuration and pushing or actually the data planes pulling the configuration. So you have the separation between the control plane and the data plane, right? And envoy gateway did that. So you

have the gateway API controller that does all of that workflow that you've seen the easy one uh and generates the data model for envoy. uh but consuming envoy API it's kind of complicated uh and it's worse if you are not an envoy implementation right because you need to actually understand how that API works what all of those fields and you know it's made for envoy it's not

made for a generic approach then K gateway and Asian gateway appeared and um why they are important so K gateway was like a new proposal for gateway API that still uses envoy proxy right was announced in cubecon Chicago I guess I don't know and then Asian gateway which was kind of a derivation of K gateway that is gateway API but it's oriented for a AI workload right

and Asian gateway was written in Rust so and it uses K gateway as a control plane so this thing got me like my attention was like okay so maybe I can have something else that can pretend to be agent gateway and I have my Apache web server being a gateway API implementation right so maybe there is an opportunity here and basically how it works so you have

K gateway generating uh gRPC XDS server with aggregated discovery service right so that's uh a type of uh endpoint on on u gRPC where you can actually have the differential configuration I'm I'm going to show in details uh the model was generated for agent gateway Okay. And it's based on the intent and much simpler than envoy layer. And I'm going to show you that. So instead of

having that huge API for envoy, you actually have a bind for a port, you have a listener, you have a route, you have the IP definition, and that's all what we care on the back end. Uh and the model can be reused by other implementations, right? So you have the address for the workloads and protocols and you have the uh resources for listeners, routes, certificates, eventually backend

TLS policy. how other things that will appear because you know gateway API folks they don't stop adding stuff that makes our life easier or not uh and each message can carry a different type of kai so this is basically how it works I'm not sure if it's good or not sorry but uh uh I I I already put that on if you want to download the PDF

as well uh so you apply a configuration in your Kubernetes API then K gateway will do the whole reconciliation loop for you so it will do all of those things that it's like okay the gateway was accepted or not the gateway was programmed or not the HTTP router has accepted or not the backls policies attached to the service that has this uh uh you know status condition

and and then it will generate the model and the data plane which is agent gateway will pull that model from XDS and will keep telling the control plane I have this model I have this configuration or I need a new configuration I need a new configuration so it's kind of easy in the end So a bit of ads before I just jumping into the other things. Uh

ads is the aggregated discovery service that's a variant of XDS from uh you know envoy RPC. It consolidates the information on a single order stream. So uh uh when when it keeps getting the configuration it will get that on an order base and the client will keep telling like I got this configuration or I need the configuration from from this time to this time right it works

on request response and the authentication is based on the node ID and uh it says like hey I'm node blah I am apache 123 and I need the resource or I need all of the resources or all of the versions and the and then the server will send and the client will say like okay you're not okay right and basically it's how it works so you have

this uh client connecting to the server you may have a lot of clients connecting to the same server the server will keep track of who has which information and the server and you can tell the server like hey I got the information just from here so you need to send me more of that right and it answers the configuration and it keeps going until the connection is

done until the stream is actually done right cool So all of this is kind of nice in theory. It's uh okay. But where I could actually start? So I I wanted actually to get started. I wanted to connect to my K gateway control plane. And I wanted to see the information flowing and I wanted to see like hey in my resource my port like I don't need

to care about gateway API anymore. Yay. So I wrote a client. The client behaves as an g gateway proxy. So it connects pertaining to be an agent gateway. it will do an authent an authentication against the control plane and get the messages. But then I found a CVE uh which is kind of ironic because I am actually the person callsing CVS and English in GX. But uh

I found the CVE which was key gateway having weak authentication right. So u I was trying to get the authentication and I saw that okay if I just send a pod name and the pod name space I can actually scrape the whole control plane. So I have reported that responsibly and they have fixed that and then I wrote a nice uh terminal UI uh using this client

to uh to see the messages flowing and this this uh terminal UI is actually important because still you need to understand how the control plane is sending the messages to the data plane right so when you are implementing your own back end you need to understand what it means when you change a backend TLS policy from the control plane to the data plane cool so I have

discovered those two types uh of messages. One is the workload address that contains just you know workload addresses like the endpoints um and that kind and service which is like an aggregation of workloads and the other one was the resource that contains everything that it's not the workload ips right so it's like the binding port or the listener the route and other type of messages that are

appearing like I've seen there is it's already ready for TCP route or UDP route it's not yet but it does have all of those messages already. So the nice thing is like I don't need to care about the whole reconciliation process because I know that the agent gateway folks are being they they are doing that for me right. So as soon as they are ready I can

just you know uh uh make it my my own back end to consume that. Cool. So this is kind of the client that I wrote which is kind of the same thing that they did. It's uh a go client. uh basically I say when I get a message of five workload address um it's going to be sent to this store new address store otherwise it's going to

be to the resource store right uh part of the authentication and basically those two uh stores they are just like okay remove or update so what happens when I remove a resource or what happens when I update a resource one nice thing to see here and I wanted to call out but I did that on the end it's like you can see that the type of address

it's from O and the type of resource is from agent gateway right so uh this convergence is kind of happening if you are watching closely development is going to be supporting agent gateway soon I think it's kind of merged or it's been merged uh so at some moment probably the APIs are going to be really close to each other cool uh this is the tui I'm going

to show you on a better screen because this sucks uh this screen yeah so let me Let me go to the T TUI. So basically what the T the terminal UI does it's like I have this you know address that contains my endpoint all of my endpoints right and I have my resource here that contains like HTTP route. So if I come here and edit my uh

uh HTTP route echo debug and I change something like uh let's see the path to echo one right a lot of things they're going to be happening behind the scenes to make that work to make my gateway work but in the end what I'm seeing here it's like cool I got a new message here that says my resource of HTTP with those keys saying the host name,

they point out to echo one, right? And they point to this back end. So that's what I care on my proxy. I don't care about the whole reconciliation loop adding status conditions to the gateway API, right? I can just come here and see that my gateway API actually accepted that. So if I get HTTP route echo debug- was accepted, I don't care, right? Makes my life much

easier. Nice. So I have started implementing an enginex proxy and let me try to be really clear with you. I vibe coded that. So if you try using that in production please don't right the GitHub repo is there you can take a look you can you know play with that but don't don't use that in production. So what it does it uses that client that I have

developed it triggers different actions for resources or address as you've seen here right so uh if I get an workload or if I get a HTTP route uh which is actually the resources or something like that it will trigger this update and say like okay you need to configure your inex proxy you need to remove this back end you need to add this back end you need

to add this and uh for every message. It parses the message and write a configuration file adding or removing configuration. Right? It uses a library call inex go crossplane instead of uh templates because again it's more trauma from ingress in gx. The majority of the cve they are because of uh go plating. So please don't use go plates on your proxies and they reload in gx. So

basically that's what it does right. Uh I'm planning to probably when I have some more spare time to do more things on that. Uh and a warning this is not a gate API conformance implementation as well. I tried so it kind of passes 19 tests over 30 but uh was a fun play. And to make that work on agent gateway what I do is actually I pretend

to agent gateway that this is my um that this is an agent gateway proxy but it isn't right. So I have this agent gateway parameters and I tell to agent gateway okay you're going to pull the image from this ripple on this image so whatever and you're going to pass those arguments to this binary right so I'm pretending that I am an agent gateway here and as

you can see on the gateway class I just add those parameter references uh saying like when you create a gateway using this gateway class right it will pull the images from here you're just going to use those configurations and using this gateway class name as in jinx So, uh, let's take a look into that thing working because it's kind of nice and we kind of have time.

Uh, cool. So, I have this thing running here. Let's take a look. So, if I do a get pods n agent gateway system, I have this inex gateway, right? This is our proxy and it's connecting to this agent gateway. Is it fine to read? I'm not sure. Yes. No, maybe. >> Yeah. >> Okay, cool. Amazing. So if I come here to this aac-it- a gateway and chex

dash bin slb bash because having bash on your proxy is a good idea and and and and I take a look into this uh proxy configuration you're going to see that basically what I have it's like an injinx configuration that adds everything from endpoints wildcard.com servers And I can start actually taking a look. So I can see like, you know, I have the upstream here, right? And

I have all of the servers here. http Yeah. So basically I have an engine that it's reading that control plane and all of the messages that it sends that are not gateway API. They are just pure uh proxy configuration messages and adding them here. So if I get a service here, let's see dash n agent gateway. I have this proxy here right inext gateway with this IP

and I can get http route here d ech oh sorry o I'm going to see that this proxy uh it it is using my uh actually this is the wrong one but that's fine. It's using this debug gateway. I'm just going to send I'm just going to change this to my inex gateway just to make the presentation better. Sorry. Keep edit http to sam.xample.com. Cool. Uh pat

prefix. Okay, cool. So I did that. You're going to see that uh it sent some messages here saying like, "Hey, uh because I was using that other gateway, those routes, they were deleted. So my gateway doesn't deal with that anymore. My debug gateway, just the other one. And I can now do a curl here. And you can see that this is working, right? So I have my

HTTP route uh attached to my Inex gateway and it's actually getting me, you know, it's working. So logs and agent gateway system uh in GX gateway I can see the message here on in GX log message sending all all of those things right so basically what I want actually to to to pass to you it's like uh if you are willing to write your own G API

implementation just don't suffer with you know the control plane we know that it's complicated I can see a bunch of git API maintainers around here as well and we kind of all agree that it's like it's not easy to implement a controller. We we had this discussion yesterday. It was kind of fun. Cool. So, what's next? First, anyone can write their own implementation rely on K gateway

or agent gateway model. Uh you just replace every K gateway here for agent gateway. I just didn't have time to update because they did that two days ago. uh and I am actually really looking forward to see people using that and abusing that model because I think it would be nice for the ecosystem to have more types of backends you know uh is also moving towards supporting

agent gateway using the same model. So uh I added uh the presentation is on scad and there are the PRs here uh so you can you can take a look and see how this is going so far supporting new features it's much easier u who was actually waiting for listener set on gateway API please just that we shouldn't have promoted that >> took so much effort what

are you waiting for TCP route yeah I can see something okay yeah so listener side is one of the hard ones to implement and back end TLS policy is also one of the hard ones because they are super you know related and you need to be sure that you are not conflicting listeners and all of those other things and there are some new AI features as well

right after all we are like on AI con right so everything is AI now and on our side I'm totally going to be kicked it off after this and on our side we just need to know how to configure the proxy right so uh and someone made me a question uh when I was doing this you know this driver and on the presentation Antonio actually made me

a question. I'm not sure if he's around like how you report back if your proxy doesn't support some feature. You don't. So that's that's one of the things the separation of things the separation of uh you know of concerns is actually I'm pretending to be agent gateway. So you need to be sure that at some point you add some hook some validation admission policy something like that

if you are really doing you know your own back end saying like hey listen uh I know that you want to use MCP route or whatever but this proxy doesn't support because it's an Apache 2.2 too from 2005. Cool. Uh so there are some references here as well. Uh sorry the links they are not here. They they are on the PDF but basically on all of those

models. Uh I I really uh I really uh recommend if you want to read them you know like uh agent gateway and they also have a nice model of controllers and reconciliation which is not based on controller runtime. So if you are willing to write all of those codes where like you have the control plane and the data plane not being even you know it doesn't need

to be gateway API you want to write something else on an easier way they they have those libraries and uh yeah it's uh feedbacks are welcome. I'm kind of on time. I was worried about like I had a lot of slides. Just be sure to go there and, you know, trash on me. It's I'm sorry. And uh yeah. Uh any questions? Do you want me to do

V code?