Building a Kubernetes Platform That Scales From SaaS To Self-Managed - Florian Forster, GitLab

This talk discusses how GitLab scales its platform from a single self-managed installation to a robust software as a service offering. The speaker, Florian, shares insights from his role as a Site Reliability Engineer working on the runway project, which aims to address scaling challenges associated with GitLab's Ruby on Rails monolithic architecture. He explains the need for a more versatile and portable solution, leading to the adoption of Kubernetes as a common platform that supports both self-managed customers and the SaaS model. The talk covers the deployment process using Helm charts, the importance of maintaining a good developer experience, and the future of GitLab's architecture with a focus on providing flexibility and managing the complexities of different deployment mechanisms. Florian emphasizes the need for careful consideration of vendor choices and encourages revisiting trade-offs as requirements evolve.

Thank you all for coming here. I know it's late. I'll be your final boss tonight. I'm standing between you and a beer presumably or the end of your day. At any rate, um I'm here today to talk to you about how GitLab builds a platform that scales from a single self-managed installation or single node self-managed installation to GitLab's uh ser uh software as a service offering. So

by a quick show of hands, who of you is running vendored software in production? By that I mean somebody else is selling it, you run it. Okay, that's I'd say a majority of the room of those. How many of you are running GitLab? That's appears to be the same number of hands. This is for you. This is good. little bit about me. My name is Florian. I

work as an SRE in the platform team at and uh I'm the tech lead for uh the runway project and I'm trying to solve this this big scaling problem. So a little bit about GitLab. GitLab is built around a relatively large one of the largest Ruby and Rail code bases uh that exists which we lovingly call our monolith. Um this monolith get is operated by GitLab itself

as a uh software as a service offering. So you can go to gitlab.com uh and buy a repository or buy an organization uh or you can uh buy this as a self-managed instance and then you get to set it up and run it yourself on your local uh hardware. This monolithic architecture, this Ruby on Rails monolith has served GitLab fairly well and this is why we are

where we are at the moment. But it is showing kind of signs of weaknesses uh signs of scalability friction and we need to in order to scale more we need to find something to to move away from that or at least in part move away from this uh uh architecture. At the moment the monolith bundles a bunch of things together that want to scale independently or that

want to release independently. maybe they have different compliance requirements and by bundling it all into a monolith we're really increasing coupling too much for its own So runway is what we aim to be the solution to this problem. Uh it's an internal platform and it was originally um built to deploy services that are kind of around the monith. We we call them uh satellite services. We have

about a dozen or so that are currently in production. Uh the the one that kickstarted all of this is a Duo, our AI agent coding thing We're about six engineers in the team. So that is a relatively small team for a platform of that complexity. And so it is uh very important for us to keep things simple so that we're not introducing unnecessary complexity. Complexity comes in

always anyways, but if you're not actively fighting it, uh you're going to drown in complexity uh sooner or later. Uh I want to be upfront where we are at because at the moment we use runway to do deployments for our gitlab.com, our SAS offering. We are working to integrating uh or integrating the release process all the way for self-managed customers and then beyond to GitLab dedicated which

is our uh single tenant SAS offering built on top of the self-managed stack. Uh but we're not there yet. So this is like the next half year or so of our work is going to be getting that self-managed uh on on board. The first version of runway was built on cloud run for a very simple reason. At the time the business need was velocity above everything else.

Like we needed to get to market and we needed to get to market yesterday. And Cloudr run abstracted away a lot of the operational complexity and allowed us to meet that that business demand at the time. Basically we built a system based on cloud run where an engineering team they hand us a container or they you know upload it into an OCI repository tell us where it

is and then runway takes over and does everything else. So they basically have a push on green sort of uh deployment mechanism where if it's integrated into the SCI and that typically is uh how they're doing it uh they just merge a new change and it gets pushed into staging and then later production automatically all in all in one go without them having to do anything with

you know appropriate safeguards along the way. Uh, this worked very well for our teams, but it didn't work well for our self-managed customers. So, GitLab and and a few other uh companies are in the situation where we need to kind of serve both deployment mechanism or both targets. We have this SAS offering on the one side with many users. Unfortunately, I'm I haven't been allowed to disclose

how many, but we have way very many users on gitlab.com. It's a massive scale in many companies relying on it. And on the other hand, we have our self-managed customers, some large, some small, some in between, but it really goes all the way down to just a single instance running everything from database and front end and git backends and everything. The self-managed uh market segment has sometimes

or some of them have very specific requirements uh that may be data residency requirements or other regulations and for those reasons for many of them cloud run is not an option. It's not a preference. It's not the the cost. It's simply not an option for them to use cloud run because it's not an option to kind of hand the data over to another cloud provider multinational like

that. And so we need to um need to solve this because in the past we essentially made this vendor lock in decision for us and now it's kind of we we by uh handing uh this the software to our self-managed customer we would would make it their problem. Uh that's not going to fly and so we need to find a solution for that. And so an important

point that I want to point out at this point in the presentation is that the the sort of vendored solutions meaning cloud run or similar offerings from from AWS uh they work just fine. They solve a lot of complexity for you. If that is kind of a good trade-off for you all good. it was for us at the time, but our requirements have changed and so we

needed something that is more versatile and more importantly more portable to well why Kubernetes is probably not something I need to evangelize here but we landed on Kubernetes as kind of the common platform uh to to run the the SAS offering as well as to to kind of help our self-managed customers run things. There there are two main reasons. The first main reason is it it's an

open standard. So you're not tied to any specific vendor to kind of provide the APIs. Uh you can use or the our vision is that you can install GitLab on GKE on EKS on your homegrown Kubernetes uh setup in in your home lab uh or in in our development environments. It's probably going to be something like uh KC or K3 something along these lines. Um we landed

on Helm packages as kind of the common denominator for packaging just because we felt it was like the def factor standard in the industry and kind of that what our that's what our customers knew how to deploy. All right. So I said that cloud run handled everything for uh the the v1 uh version of of the platform. the Kubernetes-based platform uh introduced like we made a conscious

decision to introduce an intermediate step where we generate a Helm chart for every service or every satellite service as a first step and then in a second step we deploy that Helm chart onto the the GitLab owned clusters for We didn't necessarily need that at the time. So we didn't need that to make the transition just from Cloud Run uh to GKE and EKS. But we made

the decision in order to be kind of future proof into to plan for an eventual self-managed support. in the charts are then basically in in a very similar fashion. So the the developer experience has not significantly changed. So they still have a CI pipeline that builds a container image for them and then the container image gets automatically pushed to the Kubernetes cluster for gitlab.com. Uh we're definitely

not going to do that for self-managed customers. They kind of need to kind of deploy it themselves. We we can't kind of take ownership of their cluster and it's also not something that our customers want. uh but for gitlab.com the the developers have this kind of rapid feedback cycle uh that is that is very useful for them. What this allows us to do is have a single

helm charge for something like uh the the Duo coding agent that we deploy for gitlub.com in that our self-managed customers are also going to deploy eventually allowing us to dock food what what we're building and catching problems much sooner than if we kind of had different deployment mechanisms or release mechanisms for gitlab.com and So with that setup now we kind of need to break away from how

engineers in GitLab have delivered software in the past. In the past when everything was built into the monolith we had these relatively long release cycles. So we're doing monthly releases of gitlab.com. uh we're doing like major versions uh every 12 months and that is kind of what our engineers gotten used to and we're now giving them the freedom to to kind of break away from that. There's

still going to be like a GitLab I want to call it like a distribution or you can kind of think of it like a like a Linux distribution. It's kind of different versions of these different uh satellite services bundled together into one version that is well tested and uh supported. But the individual teams, they are no longer uh tied to these long release cycles. They can move

much faster and they can innovate. ongoing work that we have is we we do generate these gitlab uh these helm charts for their satellite services which we're going to deploy ourselves and they're also being deployable by our self-managed customers. But some self-managed customers have expressed a wish to have like a single umbrella chart. Like we have the GitLab Helm chart, an umbrella chart essentially that gives a

single entry point and allows you to fairly conveniently install GitLab. Um they want to keep doing this and we need to figure out a way to support this. Uh how that's going to look like is currently a very heated discussion. Internally there are different ideas of uh uh keeping some of the things handwritten, generating part of it, generating everything, just sticking with what we have. It's not

entirely sure what we land on yet, but the the goal is is clear that these uh satellite services uh that we're supporting with runway, they need to be integrated into this chart in one shape or form. So choosing Kubernetes is a a true tradeoff that that you need to think about before you're making it. And we definitely need to think about because we're essentially eventually forcing customers

to make it if they want to adopt these these satellite services. It adds a lot of complexity that need to be trained and operations teams. uh they need to to develop some familiarity with it and that uh is frightening to to uh a lot of like operators out there. Customers now have the responsibility to provide us with the infrastructure needed. So if a satellite service say needs

a relational database and blob storage, we are not going to like we're going to provision that for gitlab.com. We are not going to provision that for self-managed customers. So their self-managed customer when turning up such a service needs to point us at say a possess database and potentially like uh GCS bucket or something along these In order to simplify that or help the customer uh achieve that,

we have the the GitLab environment toolkit uh get which is not confusing at all in like everyday speak. In the git environment toolkit essentially a an opinionated structured uh terraform codebase and anible playbooks that allow customers to set up uh the infrastructure provision the infrastructure that is needed by by We're also going to dock through that. Again, this is kind of in the future, but our single

tenant SAS offering GitLab dedicated is also built on top of GET. And so, we're also going to be dog fooding uh the the G uh provisioning infrastructure. Everything that we provision with runway are essentially kind of add-ons, additional features, uh not kind of the the core GitHub offering. All right. So, we we managed to keep the the good developer experience uh from the cloud run days. Uh

the devs are still productive we managed to keep this this consistency that kind of or manufacture this consistency that our our internal stance is using the same helm charts as our self-managed customers will eventually uh do avoid any any drift. And this isn't happening by accident. This was like engineered in All right, key takeaways I want to leave you with a vendor solutions are a legitimate trade-off.

They provide value, but they may also limit you or lock you in. And that can make sense or it cannot make sense. uh that is something that you carefully need to kind of look at and and make a decision for for the situation that you're in. Revisit tradeoffs when they stop working. the needs change and with that kind of the trade-offs that you've made in the past

may change and if that no longer works for the situation you find you're in today or potentially if you can see that this is going to change uh down the road uh it is totally fair to say it brought us so far but it's not going to be kind of a future road um open standards are a genuine advantage not just for your kubernes ernetes and the

ecosystem but also for for companies building on Kubernetes because this entire scalability that we can go from like a single machine to the the githlip.com SAS offering at scale that is essentially powered by Kubernetes. So the the orchestrator grows or allows us to kind of cover this enormous scale. Uh without Kubernetes that wouldn't be possible. Um, we have this sort of shared orchestration or the the same

orchestrator that we use uh for the SAS offering and the self-managed customer um which allows us to kind of eat our own dog food and ensure that everything is kind of consistent and made out of like one piece. All right, I'm sure there many questions at this point. I'm happy to answer any of them. Uh there's a microphone here in the aisle. I don't know if we

have any microphones walking around otherwise. >> Uh hello. Hello. >> You can hear me. So I'm serious from Nebus. uh we utilize Gitlab self-managed installations uh with our our own modifications and I want to ask you about the next plans of uh changing Gitlab approach to the deployment in Kubernetes. Maybe it will be splitted to much more services. Maybe you will move uh some site site jobs

to another uh installations. Uh maybe what will be in the next? Yes. >> Um, I can only partially speak to this. >> This is uh this part is owned by the the so-called delivery team and I I can't really speak for them. Uh the the general plan is to to move away from this very monolithic architecture and split this out into more like satellite services or modules.

Um, one of the goals is to have each of these modules be deployable independently from one another, but at the same time, we kind of need to be conscious about simplicity because a lot of customers rely on kind of simple deployment mechanisms. >> Yeah, I just remind you chart. >> Come again. >> It's not so simple. You provide a good picture of your unified hound chart. >>

agree. Yeah, thank you very much. Yeah, the the problem with the unified health drive like it it's convenient. I totally get it, but it's very hard to maintain because at the moment it's all kind of hand curated and that doesn't scale unfortunately. >> Yeah. We just decided to split it in as several parts to separately deploy. >> Yeah. So you what you can do is like configure

the helmchart to disable everything except these five services over here and then deploy only these five services that that is designed in this way by uh by design or this is made this way on purpose. >> but we internally we need to maintain the helm chart and at the moment it's like handwritten and it just uh explodes in complexity as you as you add more services. maybe

a good way to prepare >> which >> I don't disagree >> which uh doesn't work just one way to one more way to apply the same helm >> yeah I I don't disagree that would be a great great course of action to take >> I will wait really thank you very much >> yeah you're welcome >> u maybe it's a stupid question but uh >> no stupid

questions >> I um uh I've been using GitLab for a long time I'm most. But um uh what about the uh omnibus and source distribution installation methods? Are they are they going to disappear then if it's all going to be uh kubernetified kubernetes? no the omnibus package is not going to go away. So for those uh who may not know omnibus is essentially a a debian package

that is one package that you install and it brings just everything with it. It's a a very simple and convenient method of installing GitLab. It's it's not going to go away. It's way too popular for that. Um the plan is that new services that we add something like kind of the the do coding agent that they are going to be add-ons or modules that are going to

focus more heavily on Kubernetes where the common deployment model is going to be that you run omnibus and adjacent to that you run Kubernetes and then the kind of the optional like power user kind of services they would be in the Kubernetes cluster and the core functionality would continue to be supplied via by omnibus >> does sounds a bit complicated but >> we we kind of actually

run it myself but I know if if you from a strategic point of view if you going to support everything until uh I don't know when like >> I I don't disagree but we need to provide a migration path that doesn't leave our customers in the dust because we're already struggling motivating our customers to upgrade to the newest version. And if we make that extra hard by

saying this omnibus package uh that you've been using for years, uh now it's it's no longer something that that you can use uh that would be quite detrimental. >> I understand. >> Thank you. >> Yeah, you're welcome. Um, hi I'm from Metamos. We are in a kind of similar space that a lot of our customers run on prem and yeah there's a cloud offering but that's not

the main of business. >> Um, I'm curious how you handle backwards compatibility with all your satellites like can customers update the individual um individually or is it like one bulk update? How do you how do you deal with that complexity? or will there not be backwards incompatible changes? >> There's unfortunately no oneizefits-all kind of answer to this. The the best approach is if you have very strict

backwards compatibility requirements for APIs and then ensure that every communication goes via these APIs. That is easier said than done. developers frequently make changes that seem innocuous to them but then break somebody else's workflow or somebody else's uh kind of use of the API. Um there are some kind of static checks that you can do. So you can check that fields don't get renamed, that they don't

get removed, that constants are not changed, things like that. So there's some some static checking that you can do to to kind of work towards that goal. Um but it's it's not going to be 100% effective all the time. But that's kind of the the goal that you need to work towards that you have like all even the internal communication is as seriously backwards compatible as like

a a public external communication. >> You're welcome. Thank you so much. Uh, if you're interested, I have two bonus slides in case we're running short. Uh, I'll run through them and I am not angry if any of you decide to to go and get a beer instead. All right. So in addition to deploying what our our engineers have built into production to make their lives truly easy,

we need to extend the platform into the application via an application framework. uh the one that we've built we call lab kit and that allows us to make a lot of things easy for them or make it easy for our engineers to do the things in the right way. For instance, if you're using the labkit library, you get metrics and uh like a um a metrics scraping

endpoint that is set up in the way that we think that we should do metrics internally. Like this is very clearly like an package that we're using. It is out in the open. One of the the key values of GitLab is transparency. So you can go and inject it. You can potentially even like create pull requests if you so so choose, but it is very clearly geared

towards our internal use. And so if you're using as a as a GitLab engineer, if you're using the lab kit package and you deploy your service uh with runway into production, it is all uh tied up. It is all set up so that uh traces get reported uh to the right endpoint that metrics get reported to the right endpoint logs get written uh to uh kind of

logs endpoint in the right format with kind of the the standard field set that allow us to correlate between different kind of logging sources. um, again, it is open source. It's like MIT licensed like almost everything that we do. Uh, if if you're interested, go check it out. But it it may be a bit of a of an internal thing. What I want you to take away

is that deployments is not everything. You also kind of need to think in the other direction. You need to ex extend your platform into the application. And then another important bit uh that we we're urging our engineers to adopt is a a common set of CI tasks. These handle a lot of different things from uh running Go unit tests uh to semantic release uh to renovate to

update uh dependencies and and I'm pretty sure I'm forgetting a bunch of stuff like there are I don't know two dozen three dozen uh different uh jobs in there with that together you get a lot of functionality basically out of the gate, you just kind of need to remember to use the the application framework to use all the CI jobs and then to deploy with runway and

because as long as you have humans in the loops, they're always going going to be uh breakage and and deviations. There is a copier template that maybe I'm mentioning on the slides I haven't linked. There's a copier template that allows us to very easily set up a new repository with all of these things uh set up correctly. So within 15 minutes, 20 minutes, uh you're all set

up and you can potentially push to production. All you need to do is kind of write your business logic at that point. All right, thank you all so much. Enjoy your evening.