Visualizing GitOps: A Tour of Flux UIs in the Open Source Ecosystem - Stefan Prodan, ControlPlane

This talk provides an overview of the latest developments in the Flux ecosystem, particularly highlighting version 2.8. The speaker discusses the major updates, including the introduction of native Helm support and the enhancements brought by adopting Kubernetes server-side apply. They explain how Flux has improved resource management, allowing hundreds of Helm releases to upgrade in parallel with a focus on performance and security. Another significant feature is the new artifact generator, which alleviates issues related to managing large monorepos. The session also reviews various user interface options available for Flux, emphasizing the importance of community-driven UIs while explaining why Flux itself does not have an official UI. The speaker shares insights into a new lightweight UI being developed for Flux operators, designed with mobile-first principles and a strong emphasis on security and performance.

Hello everyone. Congrats for making it to the last day of CubeCon. Yay. I almost didn't made it. I was like, hard night. Okay. So uh today I would like to give you like an overview of what's happening in the ecosystem around flux with all the UIs that we have and I'll try to uh show you the UI that I'm contributing to. Uh but before we go into

that I would like to give you a short overview of u what we shipped in uh the last uh flux release. So we have 2.8 going GA. Um we do three releases per year um after each Kubernetes release and we usually skip the one in December because we don't want to you know spend Christmas uh dealing with client go not matching coign client go not matching helm

client go and so on. Um so what we did in in 2.8 The the biggest thing that we pulled off was we finally have Helm before support. Um I want to thank the Helm team for working with us all these months to make it happen. Um Flux as you know has um native Helm support. What this means is that we don't use Helm as a templating thing.

We actually use the helm go SDK inside flux. So everything has to be we rebuild all the flag all the helm actions and we improve the life cycle um inside helm controller and that means we are like tightly coupled inside helm controller with the helm SDK and if something changes upstream we have to make sure those changes are compatible with what we do in flux and our

main main concern is always performance from you know a helm cli perspective If if you use Helm to deploy on a cluster, you'll probably run it in a CI runner, right? You don't care if it uh takes uh one minute instead of 20 seconds or if it uses I don't know how much memory is there is just a CLI call is one Helm release that you are

installing or upgrading. Um flux is very different. We have to ensure we can upgrade hundreds of hand releases in parallel. We really care about you know the how many resources these are using how many queries to the cluster. Um also from a security point of view we need to ensure the charts are sealed inside the controller so they can't escape. So we have a different security posture

when it comes to integrating helm. So for us these kind of releases are a major disruptor. We basically focused uh almost 3 months on this migration and I'm very very happy with what's happening right now in Helm we finally have Kubernetes serverside apply. If you have been a flux user for a long time, you know that um in flux we've adopted serverside apply when it was alpha

in kubernetes and we we gave a lot of feedback to the kubernetes maintainers to make it uh better. It was u was a effort across um you know projects and having server side applying helm controller means we finally have a uniform way of how uh flux interacts with the cluster how we do merging how we take ownership of resources and so on. Um another great improvement uh

coming with serverside applying helm is the fact that we are the helm controller will do less operations with the cluster that means we can scale it even better now it will be lightweight it will use less resources and so on so I'm very very excited about this future um another big feature that got into Helm um and Scott which is also a flag maintainer. He's also a

hell maintainer uh helped us a lot uh getting this into Helm is the possibility of doing health checks for custom resources inside a chart. This was a big big issue for I'm guessing flux the majority of the flux users you are if you put something in a nit repo and you apply to the customization doesn't matter what's in there if it has a status um flux can

wait for it can hair check it can tell you what's wrong well if you put a custom resource in a chart good luck the chart says yes it's all okay release ready crossplane down right or whatever is in there. Um, so we finally have this capability, this unified way of doing health checking across all helm across all flux controllers. You can now have cell expressions into a

helm release to tell flux how to um, you know, determine the readiness of a custom resource. So even if you have a custom resource, I don't know a PG cluster for example or a cluster API cluster or something like that and you want to wait for particular conditions that that uh custom resource has to go through, you can teach flux how to do it and we have

um we have a cell library on our documentation. it. We tell you it's it's quite easy how to teach flux how to you know um have an expression that represents readiness for your use case. Um a third thing that we shipped in in flux. This is not part of the Helm SDK. This is something we've built on top of of of Helmv4 SDK is a way for

for reducing uh meanantime to recovery. If you are a flux user for a long time, I'm guessing you have suffered around this. You apply a change to a hand release. The hand release has I 30 minutes timeout because you deploy stateful sets or whatever you are doing there. It gets applied. Handle controller starts the upgrade. Then you realize, oh damn, I made a typo in the values.

You change that. You apply it. Nothing happens. You need to wait 30 minutes for the hell check to time out and only then your change goes there. And why is that? Because the the way the way flux controllers are built build on top of control runtime. Once a go routine is spin up it will has to do that job till finalization right what we've been working on

in the last months many months six months I think we also have this in customized controller is a way to signal the controller hey there is a new revision you should um cancel the ongoing hair checks mark the hem release as failed due to the cancellation So you know why uh that hen release entered that state and immediately start the next upgrade. So you can what what

this enables you let's say you have a dev cluster you iterate fast over changes that will be very beneficial you don't have to wait or maybe you have large teams high velocity they do commits all the time and you want to you you want to speed up the whole continuous delivery pipeline and enabling this will will uh will definitely help out. We encourage everyone to play with

this feature is behind the future gate. We plan to make this the default into flux uh next early next year probably Q1 next year. So please test it out, give us feedback. We we really want to solve any issues that you encounter uh around this new behavior and make it the default um in a couple of months. Uh another thing that we um we shipped is this

uh release inventory in the status and this somehow relates to UIs the way you discover what's being managed by Helm how you do that anybody knows Helm has storage has a Kubernetes secret right you know what's in the secret it's a tar So you unt it and inside the tar there are a bunch of yaml which are the template you need to parse them figure out was

managed and then show them in the UI right that means you have to read secrets maybe you don't want that for a UI and do all all this operation like discovering what's the latest storage untit all the stuff right so it's quite complicated what we uh what we shipped in 2.8 date is uh we helm controller of course knows what's managed at installer upgrade time so we

reflect that in the helm release custom resource under the status so this is also part of the unification of our API because customized controller did that since the beginning so now if you want to know this hand release what exactly manages in the cluster having the permissions to do get on a hand release is enough to know everything and uh in the inventory there are also CRDs

that flux manages outside of Helm. Everything that's you know managed by that entity will be in there and um we we build this especially to you know allow UIs to be very fast when they show the inventory but it's also you know a great feature during debugging with the CLI everything. So it's there from now on you can you can use this um new status field. Another

thing that I want to so there are many changes in 2.8. I will not point them all but what I want you to I want to remember everybody that we have a new controller called source swatcher and a new uh custom resource. This was introduced in 2.7. Um this is called artifact generator and we've built this to resolve two major pain points when you deal with you

know githubs at scale there is the case where you have a huge monor repo everybody has all the yamas in there all the charts all the teams are collaborating on one huge repo and the problem we have before artifact generator was any change anywhere in the because it's one source for flux the digest changes and everything has to be reconciled. If you have huge dependency trees, if

you have like hundreds and hundreds of flux customization, everything will do a dry run and it can take a while. It's quite annoying. It creates a lot of noise. Uh you can use the artifact generator to do this decomposition of monor repos uh based on paths. So you say hey I want to create an internal flux artifact only for the policies and then you use a customization

to apply this artifact that called policies that flux generates inside the cluster. Of course you can do that today with CI and flux push and decompose your repos into OC artifacts. we observed that uh for some large users moving from git to gitless and nci is a a long process. So before you get there the first step you can have these internal artifacts directly in the cluster

built by flux and they are like 100% compatible with OC artifacts are the same thing but they are generated by flux inside the cluster. So I can do this type of decomposition by just dropping one um custom resource in there. Another example here is the opposite. I want to compose things from multiple sources into a final artifact. And one of the challenges uh we see with charts,

right? We we have charts from outside organizations and charts from inside the work. Maybe for those inside the work, I have better control over them. I can set good default values and so on. But those coming from outside there is a lot of customization that has to go into them to adapt them to our needs. And here is an example where the chart comes from an OC

reposi let's say uh something vendor from outside. I have a g repo with uh better values. Let's say values optimized for myself. So I can uh say create an artifact from the chart, override the values with a file from git, then with another values file, merge it on top of the override. So you have all these um layered operations and at the end what what flux will

do will um rebuild the chart and present it to helm controller as a chart coming from outside. So, Hel is not even aware that all these changes happen there. So, yeah, please take a look at artifact generator. This is still uh um in beta. We we plan to do a GA release um quite soon. We had good feedback so far. So, if you if you try it

out, we'll we'll love your feedback on this. Okay, time for uh UIs. So I have I have some lists here with what UIs are currently into flags. But before going into this, I want to uh talk a little bit about why Flux does not have a UI and we'll never have it. We designed Flux as a set of controllers and loose couple APIs. Flux is now built

on this idea of the GitHubs toolkit. If we have a UI in CNCF flux, then it means that all the all the APIs will be shaped around it, right? The UI will influence how we design the software because we also own the UI upstream. And we made the decision early on, we are not going to do that. We are going to design the APIs loosely coupled and

it's up to the community to create UIs other experience around it. So that's the reason and so far there there are zero plans on having uh a UI official stamped in the flux organization tell everybody this is the thing that you should be using. Um and this decision enabled great things in the last years. We've seen uh many projects integrating into Flux, bringing the Flux experience in

their own UIs closer uh to users, right? So I think that's a great win for the ecosystem. let's say my favorite way of running uh a UI for Kubernetes that shows me everything I can do anything with it. My favorite way of delivering this will be through a desktop app. And we have here three options. Freelance. Anyone is using this? This is open a fork of open

lens. Open source. Anyone using is using freelance here? Nice. Yeah. So I I I've been looking at the flux plugin in freelance. It's really really good. It's maintained by u the people that are building freelance by the maintainers. Um it kind of has all the things that you'd expect from um a Kubernetes Kubernetes UI um that layered on top the flux custom resources and it also supports

I discovered by researching uh couple of days ago. It also supports all the custom resources of flux operator which was a big surprise and they uh from from all UIs out there I think freelance is the only one that's very very fast on adapting the UI to the latest flux release and I'm going to talk a little bit about this and why is this important uh we

evolve flux and this year we have removed beta versions from everywhere right so I was not surprised to see uh cap Pister and others having issues with the UI does not work fails because it it requests a version that Flux no longer supports. Um so the let's say the bonus point with freelance is that they uh update very fast which is something that you should ac um

expect from uh from a UI that builds around flux. Um the second one here I'm guessing everybody knows headlamp is the uh cube dashboard replacement is the official way of having uh dashboards and UIs for Kubernetes. so the headlamp uh um core maintainer is here and nice to see you. Um they are doing a great job. They also have integrated flagger into the plug-in. The plug-in in

my opinion is beta. I don't know is it beta or alpha or how would you call it? It's usable. Yeah, I think I think headlamp probably is the best choice if you go with a desktop app at this point because um it gives you this holistic overview of everything not only flux they besides Kubernetes I know you have other plugins in the pipeline and more will come.

So I think headlamp is going in a direction where um let's say popular controllers in the CNCF landscape will have some sort of a plug in there. So besides flux you'll have be able also to see I know S manager certificates and many other things that you have in there is uh tightly integrated with Prometheus as well. So that's that's a that's a great u great way

of of using it. I'm I don't know. I'm not sure I would run headlamp in the cluster with cluster admin. It can do many many things. So maybe keep it as a desktop app with uh uh some tight controls over the cube config. And I think that that works for everything in this list. Um if if the UI is generic and it's about everything, of course it

will give you a bunch of power to do very bad things to the cluster and flux will try to put them back. Uh all that stuff that you probably have experienced before with cubectl. Uh third one here, capacitor. Capacitor has uh gone through major refactoring. I've discovered the web app is not open source but the desktop is. I think it's alpha right now. I uh I still

have to play more with it, but it has a u a unique approach to how how they show all the flux interactions. The UI is focused on debugging pods and so on. So, it's a it's a great tool to experience and and uh you know um try it out. Another um category are terminal UIs. Terminal UIs are having a major comeback, right? Everybody's using CL code copilot.

Uh we are back on the terminal. So if you like terminal UIs, there are the obvious option. K9s has a plugin since years ago. It's stable. Uh the plug-in is maintained. Um it's very lightweight. It does exact all the CLI. So you have to install the flux CLI, flux operator CLA and so on. But there is also a version written by a flux contributor in Rust. Has

no xx looks and feels like uh k9s is flux 9s. It's in beta and it's a really really nice experience. If you if you go with terminal UIs, you should definitely give it a try. Give it a start. I I really like uh this uh this approach as well. Um so we we've talked about these two here which are kind of a single cluster view uh and

I know headlamp allows you to change the cluster right but you don't see multiple clusters in a single pane of glass right you can you see a hm release in this cluster and this cluster and compared between them okay so you have to select and build them but I think a multicluster UI is way way more requires way more things than just you know being able to

show uh things requires uh you know being able to def them them. There are there are a bunch of subtleties when you do multicluster and here we for for this purpose we always ship graphana dashboards right that you it's up to you to push all your metrics to a central Prometheus and from there uh use our dashboards and have this multicluster read only experience through graphana. There

is also a backstage plug-in that was maintained for a very long time. Uh now it's moved into the official uh community plugins. Uh and being backstage also can connect to different cluster and give you some kind of a multicluster view. Um so these are the multicluster options for for it only. Now one to six minutes left. I can uh do a little bit of a demo what

I'm building and me and other flux maintainers into flux operator. So we are building this very lightweight UI. Our target is I want to use it from my phone. So it has to be mobile first. Uh it's not it's not something that you have to you know configure or put on top of it is built into the theme. So you just have ingress on it and you

and you get it. If you're running flux operator now the UI is there you can access it. let's go and show some flux UI. So the first thing uh that we we had to build was single sign on. And why is that? I think for us was very important to build a UI that has the same security posture as Flux. Flux works by impersonating service accounts. You

uh assign service accounts to GD repos. else are let's say representative of a team. A team manages a git repo, right? So then you assign that team an identity inside the cluster through a service account that flux impersonates and it's of course subject to kubernetes arbach. So everything that um a person can do through git operations is restricted on the cluster but it's impersonated by flux on

the cluster under these restrictions. So the UI has to be built in the same way for us to call it a flux UI. So what we what we did here um we integrated with Dex to support all the IDPS. I have a bunch of them here. I can do GitHub. Uh what what happened after I logged in. If I go to my profile here, basically what we

what we do, we get the jot from um the IDP and this jot contains a uh an array of groups maybe depending on your IDP here can be teams or whatever but it's is there a claim that tells uh tells the fluxy that this person belongs to this group, right? And this group matches probably some Git repos is part of a team. It's a developer. You already

have that kind of arbback already in the cluster for flux to impersonate and do its own thing. So what we can what what the UI allows you to do as an admin um as a cluster admin through cell expressions is to say if it belongs to this group then look for a role binding or a cluster role binding with the same uh group or you can do

name transformations and so on. So the UI impersonates that group in the same way flags does. That means if a developer has access to three name spaces and only hem releases when they log into the UI that's all they are going to see right so uh this is part of the multi-tenency story when we talked about um flux in the UI and that's that was one of

our requirements um other things about the UI is the fact that it tries to give you a holistic view of thumbs up thumbs down approach um flux operator orchestrates flux. So it knows everything that's happening on the cluster without having to query the cluster. So one one major difference between let's say all the other UIs that you run locally and so on. They have to query the

cluster for every single because this one runs in the in flux operator. We can use the control runtime cache. So for example, we can allow you to search and filter 10,000 hand releases on a cluster in using 32 megabyte of memory. And it's like really really fast. Of course, it's a runtime cache. You can get some couple of seconds delays and so on. Uh maybe it's not

real real time as you'll query all the time the cluster, but it can scale at massive size. And this is also something that we really wanted to do with with the with this UI is being able to run it on massive cluster. It doesn't matter the number of you know how many H releases you have, how many apps, it should work the same and be extremely lightweight

and fast no matter how how big the cluster is. Um other things that we really strive here is support everything. All the CRDs have custom dashboards for each one of them. for example, I don't know. Let's go release the ham releases. where is the big one with Prometheus? I can feel. Oh yeah, this one. The monster Hchart. Everyone knows this one, right? have comes with a bunch

of things. So um we also have this graph that works for all the reconcilers. uh this heristy customization, a research set, flux instance, everything that reconciles something on the cluster will give you this kind of graph that tells you what's the upstream source in in this case is the um uh GitHub um of the um Prometheus community I think operator community something like that. Then is the

OCI repo that actually flux uses H release and everything that goes from there. Um we are focusing on these workload views where you can see uh the images that are deployed pods you can restart the pods you can have all sorts of rollout restart for uh stateful sets and uh you can create jobs from chron jobs. We have added all these you know things that people expect

from the UI to do fast debugging or restart the deployment not do it through git if it doesn't conflict with flux. So that's the the thing we we the UI is built around what actions can you do and flux will not override them right so things that are will will conflict with flux they will not work in this UI you will not have an action for it

um and all these actions that you see here are custom Kubernetes arbach verbs for example we have a verb called suspend if you want to give someone a group uh someone from your team the right to suspend something you have to create uh role binding for the verb suspend on that group right so we are not coming with our own arbback and anything all is based on

kubernetes arbback through this uh verb extensions um what's coming next for this UI we want to focus more on um the developer experience where they don't know anything about flux they want search for their app, land on a deployment dashboard and see there how that deployment got into the cluster. So we want to this is more of a view for um you know S sur platform engineers

but we also want to give specialized views for developers. We have this preview environment future for example. So we want to build a specialized view for preview environments where you it's all about the app is not about GitHubs anymore. So we want to kind of have this um change in our mindset a little bit and uh you know be more uh close to developers. Um we have

one minute left. Any questions? Yes. >> Okay. So the question is is this a multicluster dashboard? No, it is not. It runs in flux operator. Flux operator runs in each cluster. So you need to connect everywhere. Of course, flux supports hub and spoke. You can install it on a management cluster and from there you control everything. So that dashboard will show you all. Should you do that?

No. Uh run flags everywhere and uh yeah, you should access the UI like this. Any other questions? I don't know any chance that flagger might be included in that in the future. >> So we have an issue um many people are asking this question when will flagger get a UI here. It will definitely get a UI. We have some plans for flagger to do uh a refactoring

on it have the API brought to v1. So yes a flagger dedicated UI is coming to um to the flux operator UI. Thanks. there is the possibility to use uh API to develop custom uh UIs or not as plugins to this UI you mean or yes no we don't have yet a plug-in system uh given the fact that this runs in flux operator is quite complicated I

don't want untrusted code to be run there so I'm not sure if plugins will ever be I think uh maybe it's it's too early to say. No. All good. Thank you very much.