It (Only) Starts with Containers | Patrik Duditš (EN)

In this talk, the speaker discusses the complexities of deploying Java applications within containerized environments, focusing on the use of Kubernetes for orchestration. They explain the journey of transforming a Java application, encapsulated in a JAR, into a deployed state using YAML files, custom configurations, and the tools available for Kubernetes management. The speaker introduces Kustomize and Helm as methods for reducing redundancy in deployment manifests, and also explores GitOps practices with tools like Argo CD and Flux CD for automated deployments. Additionally, they highlight advanced solutions such as Pulumi for Infrastructure as Code and Custom Controllers, specifically mentioning Crossplane for managing custom resources in Kubernetes. The session emphasizes achieving a standardized Java abstraction layer to simplify the deployment process while ensuring robust infrastructure management using modern cloud practices.

[music] >> Uh yeah, glad you made it here. Uh my name is Patrick. I work uh with Azul, um previously Payara, and I spent last 5 years working on containerization of of of Java applications. And yeah, it's a long story to tell. So, uh if you want to reach to me, uh I'm p-didits uh everywhere imaginable. And of course, if you've got any questions, we can use

Slido uh for that. And and yeah, I will I will answer them towards the end. So, what do we want to speak about today? It's about all the things that need to happen for your production system to serve a response. And uh generally, you probably don't have just one of those applications, you've got many, and we need to put some order into it. Uh purely from from

developer's point of view, that's that's simple because, you know, you just write your your research method in a class, and then package it in a JAR, and then have some runtime that uh does all the connection handling for you. Uh and then you put it in container with your JVM and with some base image. And from developer's point of view, that's where the journey ends. But, yeah,

the the situation is a little bit more complicated. Um as I was writing those titles when I was preparing that presentation and read read uh what they say, I said, "Oh, this is nice. This looks like a poem. So, why don't we why don't we continue in that style?" So, "Hello, I'm an endpoint a method in a class sealed in a jar next to my runtime shipped

in a container inside a pod within a namespace. Deployment keeps me scheduled on the nodes of the wider cluster exposed by a service adjusted with a config map. My secrets are a secret and through ingress I reach out. One controller guides my packets, the second one gives me name, yet another approves me valid. There's more of us, but we're not the same. You hear me in network

calls, you see me in logs and metrics. You know me through past selves, older versions, remembered settings. Layer on layer object by object abstraction becomes reality. And that's like 6 hours in PowerPoint. I can't And yeah, you may think that this is over the top, but this morning uh Kubernetes 1.36 was released and yeah, I wake up, I read the release notes as one does when when

they wake up and see this. A poem in release notes of Kubernetes. So, I I start feeling there we've got a new trend here. So, anyway, abstraction becomes reality. What do I mean by that? It's a way to only say the important things and let the things that are easy to deduct to be filled in. And definitely we've got multiple layers of how abstract can we be.

So, on on the one end, we definitely got this On the lower scale of abstractions, we will have another language that also has a really interesting indentation rules. That will be YAML. And throughout this talk, I will be considering this kind of example situation because as I said, if you are shipping Java applications, how many of you are shipping Java applications in Docker container? Yeah, round about

round about half. So, you are not shipping one, right? How many how many separate deployment units will that be? Like less than 10? Yeah, less than 20? Over 50? Okay. Oh. Still someone. uh we'll consider that we've got like two applications that are quite similar, yeah? Because you will have some kind of common concept of an application. We've got two variants of them and sometimes I will

mention the third one which uses the same domain name as the first one, just just different context root to demonstrate how variable the various approaches are. The code you can grab over here. So, it's like bunch of YAML files and the sample And there will be also like small QR codes in the in the corner time to time to point to the specific directory. But yeah, if

you want to follow along, you can do that. However, these are kind of works in my cluster files. So, you if you want to try them, you will need to tweak ingress settings and so on. But just to see the syntax, that's that's enough. All right. So, if we deploy an application with YAML, roughly how much how much YAML code we uh we need to have. And

it does all all the things, yeah, from from the first part of the poem. So, we've got a deployment, a service, and ingress. What's interesting to consider is how much code we need to write for uh the second application to deploy. And the answer is exactly the same amount of code. And it will be quite similar. Yeah, so actually those are places where the things are different,

and even there they repeat a lot because they just repeat second instance second instance second instance. And it's like 14 line difference. This clearly is not an abstraction because with abstraction we want to reduce duplication. We want to reduce redundancy and and uh what we want to achieve. So, what tools do we have to improve The simplest of one will be customized. You've got a script already

with your Cube Cuttle. And it's a tool that can do predefined set of of uh transformations, prefix files, change labels, or then perform arbitrary changes within within the deployed resources. It looks uh like that. You will have some some base um file where all the resources are described. And then you've got a directory where you got your customization file with a key that describes the transformation and

any additional files it might need. So, in base you just say like these are the files that I uh want to have deployed. And in the customization uh you can start describing what the transformations are. So, this is the name space we will be deploying to. We will be prefixing everything with first instance. We will include labels in all of the things in in the services and

deployment specs and And we can generate a config map from a file, which is really convenient because it's gets a little bit awkward in plain YAML as you need to escape things, indent things, and so on. And uh yeah, the final rule is to set the host where there is no predefined transformation, but you really need to in uh the resource called ingress, replace specifically this path.

So, uh first rule within the spec and for the host, which you know, for the host attribute we change the value. Things get a little bit more complicated. Yeah, ironically there's 23 lines. Yeah, that's like a greatest greater diff than if you just replace 14 places in the original YAML, but we are not repeating ourselves at least. Uh things get complicated for the third case where we

want to change context root. Specifically, this example uh uses PyroMicro and the context root needs to be specified on the command line. And it happens to be like fourth parameter. yeah, we need to really know where our context root lies within the base base definition. And this of course is isn't good for maintainability. Yeah, because when base changes, an additional argument will be added uh, to the

JVM. And if you've seen Bruno Borges' talk 2 days ago, you'll see that uh, you will know that yes, you will will be adding and changing JVM parameters because that's a reasonable thing to do when uh, when deploying into containers. Then, your downstream, your applications, will need to be aware uh, of that and need to adjust their patches so it fits within your base. And yeah, this

is kind of locality problem, yeah? So, either for the for uh, settings or uh, or for context root, you will have lots this lots of this structural dependency on your base, which which isn't good. And that's not the only thing that's wrong with Kustomize. Uh, you generally cannot really have like a good reusable component that you can refer to and and just have few lines uh, and

describe it. Documentation isn't really good and basically it is considered feature complete. So, if you miss anything in Kustomize now, you will not get it there. Um, it also doesn't have any concept of a transaction. So, if if API server rejects your change, you are left with with a half-deployed thing. So, the things it managed to already apply will be applied, the others will be not. So,

we need something better. Uh, what could be better than YAML? A YAML in a text file which has like yet another format. Uh, so templating. Um, there is one templating tool, yeah, one that everybody would use, yeah. We use this Helm over here. Yeah, around all of the room who likes Helm? Brave. >> [laughter] >> But there are a few other things you could try if you

are not not happy with Helm. So, Jsonnet is a configuration language which is what would JSON with variables and and functions and prototype inheritance look like. But it's not really used all that much. But but can help you here. And if you want to see some more exotic project, it's called Yoke CD which has interesting concept because it doesn't define any language. It says I will pass

the parameters to web assembly runtime and you can write your templates in whatever you like, yeah, and you should just spit out JSON on the other end to make the resource, yeah. Also, yeah, a tip if you don't like YAML, don't forget every JSON file is a YAML file. you don't need to follow the the white space rules. If you can generate JSON, that's probably good and

many times better than than trying to find the indentation in in your template program. So, let's quickly uh roll through what a Helm chart looks like. When you create a default uh application with Helm create it will create a chart YAML which is where the metadata is. Values is a plain um map of different values uh that are can be used in the chart. This is what

you tend to customize. And then in template you've got uh the sources in and then go text templating language. And a Helm chart can have downstream dependencies which get downloaded into the charts uh directory as it is applied. So, the default values will have like lots of things that you tend to uh define for for a Kubernetes deployment. And it will help us solve the problem with

the context root there is like a a usual place in the values.yaml or of course you can define your own if you know that you've got you're only on single host and have only single path. And then reuse that in the definition of the command line you are invoking. uh instead of being structurally dependent like the in the opposite way, you can reach out to values and

easily uh some wicked definition of easy, uh you can reach into that structure and and reuse it. So, what um if you had uh multiple uh applications, you could have like a your default chart uh and then basically deploy two instances of it with different values. That's that's the usual way of of solving that. Or then creating um kind of library which defines >> [snorts] >> the

subroutines you can reuse in your actual Helm charts. Or there is this trick. You can use single chart twice as as a dependency by using alias. But yeah, this is really this is really uh an edge case and uh it has its own uh trouble. But, the result are nice. Yeah, if I want Yeah, if I really got like a two Java applications that get deployed at

the same time as kind of like not not exactly microservice way. uh yeah, you can use this. Uh there are limitations. Especially for this approach, if and that's that uh the the namespace for the for the subroutines, yeah, for the templates is global. So, if you've got two charts that define uh the same template name, Helm will pick one. And it will not tell you which one.

It just will do it, and then you will be uh yeah, diagnosing what the hell is happening. and another issue you will find quite early is if you've got dependency within your configuration values, there's no way to interpolate them. Unless the chart specifically does an eval on the particular parameter and particular location. you need to put Yeah, everything purposely. So, what we tend to do, or at

least what we did in in in my project, we will have a check if we wanted to do if we knew that two values need to be equal across different places in the value file, we will put a check and fail the build. especially if if we were under the control of the downstream chart. Right, so that will be that will be templating. So, what Yeah, how

can you get better? So, uh first step uh maybe is that you will be not running the commands the home home upgrade commands and the kubectl apply commands from your machine and you want a way of automating them. Uh the usual approach is to to use uh GitOps and these are the two projects >> [sighs] >> that allow you to do that. Uh one is Argo CD,

the other one is Flux CD. I will just quickly show, yeah, for a bit of interactivity, I will quickly show what Argo CD looks like. So, Argo CD can scan multiple GitHub repositories and then apply changes within those repositories into multiple clusters. And yeah, you get uh user management, uh audit, and all enterprisey things that uh you will need for for uh managing large-scale deployment. Uh this

is an example of those customize files that I showed. I'll basically just go and say synchronize and you'll see how these things got created in the cluster and they're coming alive uh and everything is working. And eventually I should be able to just go and open, yeah, and I was too fast and DNS didn't yet propagate. But ultimately uh this is a kind of view into your

application that you get. That's not the right button to click. Cuz that was the button I wanted. So, yeah, as long as your application is simple, yeah, and this one is like three deployments updated at the same time. Uh yeah, it is you you get a nice overview if your application is more complicated, the overview isn't that nice anymore. That still doesn't work. Never mind. So, Flux

CD is different. It's a little bit more abstract, have different sources, can like automatically upgrade uh Helm charts. Uh but also if you are just arriving into that area of Flux CD documentation is a little bit harder to digest than than the one of Argo CD. If you want to go outside uh of Helm templates, another good solution for you might be having infrastructure as a code.

which could allow you to define the Kubernetes state the way you want it in a program or in a config file. Yeah, and not only that, it can also handle the rest of the infrastructure. Many times you need I don't know a database or or an S3 bucket. And yeah, that can handle that as well. The usual suspect here will be Terraform. However, uh what we were

using uh was Pulumi, and we chose Pulumi because it has really nice uh property that you define the infrastructure in an actual programming language. So, you can write it in TypeScript, in Java, or in Python, or if you go in the Go if you would prefer. And as you are building up, the program runs, and you are building up these different resource objects, and then that gets

synchronized uh to the state of of your actual cloud account. And the good thing is you've got all of the obstructions of your programming language at your disposal. So, it is much easier to build uh either more complicated or more importantly better reusable uh resources. this is uh how it looks if if you're using Java to define and and you know, it For a Java developer, this

this reads quite cleanly. Yeah, so for creating a namespace, you do new namespace. For deploying a container application, you may really build a high-level uh abstraction where you only put the things that you want to specify, like what's the name of my application, where it gets deployed. Uh what's the host name? Same thing for the other one. And you can either capture the thing I didn't mention

at the beginning is that instance two needs to reach to instance one. And you can actually um And before we just mentioned that in a config file, like yes, your endpoint URI is this because we happen to know what will be the name uh of the application inside the cluster. Now, we can express it directly in a code and in actual value dependency that gets evaluated. your

situation is quite uh complicate and if you want to capture that precisely uh using Pulumi might be uh a way to go because you've got all of Java in your You can reach to the external resources, you know, you can have uh deployment plan in the database and look uh follow that as you're applying the changes to your infrastructure and so on, so it's an interesting concept

to uh to to Yeah, this is how the container app definition looks like. It's not important how many lines of code it is or that it looks like somebody threw a fireball on it because it's still the the the Kubernetes YAML, but all of done in in Java builder style, so. the effect on on on the main part where where you define your abstraction. That's That's what's

You can have more nice things with Pulumi as well because of course it can handle the rest of your cluster as well. Updating your ingress controllers, it can upgrade Kubernetes version in in a really neat way and and we were really happy with that. One line change will update all of the additional controllers needed in the cluster and so on. And obviously for your application you might

need those databases or managed identities. Uh there are downsides though. uh the state file for Pulumi so it's either um a cloud service where you are paying per resource per month or you can store your state in Azure blob or in S3 bucket. But it's kind of yeah, it's like additional thing you need to track. And that state file gets locked while the deployment is running. Things

like Kubernetes upgrades can take 30 40 minutes just to get applied on the API level. So it might block your throughput uh in how many things can happen at the And the programming model uses lots of uh lots of asynchronous monads. sometimes it is hard to put values together and and you need to like learn quite quite specific programming style. But yeah, that will be our our

next level of of abstraction. Can we Can we do any better? the solution for that would be using custom controllers. So, what custom controllers enables or to you is what you want to have as a Kubernetes custom resource. So, custom resource for those that do not know is uh you define those different objects in in Kubernetes, yeah? Like like the deployment and ingress and so on. They've

got some kind of schema. And you can define your own. You can uh send a JSON schema and say, "Yeah, this is Java application to to the Kubernetes API server." And then you can have a controller that watches for changes to this kind of resources and transforms that into another Kubernetes object. Those then have another custom custom controllers that perform changes in in actual infrastructure. So, it's

like from from the up to the top in Kubernetes. That's that's its major operating model. So, imagine we could have this, right? So, I just say I want Java application where it's called like this. I want it hosted uh at this host name under this context root, and this is like the additional configuration I want to pass to it. And the magic happens. Controller will do something.

And extend it like, "All right. Yeah, it is now done. This is your uh URI that you can use for reaching that application." And I think yeah, that's the that's the kind of abstraction that that everybody uh would like. Developers don't need to write much. The operations team will really like the the Kubernetes-ness of this solution. And um yeah, how do we make that happen? And the

project you want to look at at that case is Crossplane. Uh which is by their definition a framework to help you build control planes in in consistent manner uh and for >> Java native projects and so on. Uh what it means it manages uh the aspects of deploying those controllers, uh extending your custom resource definitions, and uh making sure that you've got the correct transformations applied according

to the version of the file and uh and so on. And you can write those transformations either declaratively in uh some kind of patch-based YAML language, or there's KCL, which is kind of configuration language, and also Python and TypeScript are supported. And there is an example of Java-based controller, but it is not documented, but it exists because what happens in the end like Crossplane orchestrates all those

different transformations using some gRPC calls. So anything that can accept gRPC can kind of participate in this. Uh interesting extension that the Crossplane will give you is the uh providers uh that can manage cloud resources. Yeah, so just like you have your custom definition for Java application, uh another custom resource that will describe your RDS instance or yet another to describe a bucket or or an IAM

role. So, that's also uh what will make the applications more manageable. As all aspects can be done in consistent of custom Kubernetes resources, and you will then have something like Argo CD to apply those resources into the following slides are just So, lots of small letters. I'm sorry for that. The custom resource definition will look like that. So, uh you will have basically a JSON schema definition

of what are the arguments you expect. What are the arguments you will be uh uh putting away. And this is the YAML patch file. Uh Details are not really important. If If you would like to see that in data detail, I'll record on the QR code in the corner. But, yeah, it's kind of similar to customize. You've got the resource template. uh create, and then how how

you want to pull the values from input, and place that into that template. But, of course, the difference is this is the kind of base file. Yeah, you don't instance doesn't need to know what the exact structure of the output is. Uh if you would want to see how KCL looks like, it's kind of JSON with variables. Yeah, similar to to to JSONET. um but you will

not uh need to yeah, see that very often. Also, if you look in the repo, there is like entire entire harness for testing that. Uh so, you can test like what the output of this is without needing to deploy it to cluster. That's something you don't get by default. So, uh yeah, that might be helpful if if cross-plane and KCL is what you want to pursue. Another

alternative, if you find this complicated because frankly it is a bit, you can write your own. So, you can take fabricate Kubernetes client and listen to the changes of the of the custom resource, perform the changes, then listen for the changes for the resource types that you that you created to know when they're ready, so you can report back. I'll be not showing example of that, but

if you would like to see one, there's this project I created called Doorman, which was like a very specific scale-to-zero controller that we created, and it does exactly that. Yeah, it has a custom resource for scaling to zero, and it will scale down deployments when there are no requests and then scale them up as as the request comes. So, yeah, if if you want to see the

mechanics of that, that that would be the project to check. So, yeah, this gives us like nice hierarchy of what's available. yeah, the the different different ways of of approaching abstraction within uh uh within that. Let me see if there are any any questions. And the Slido says there aren't any. So, yeah, but could it be any simpler? So, as I said, I was working on a

cloud runtime for uh a few years, but unfortunately that project is now suspended. So, I don't have an easier solution for you. What I can tell you is so what we just went through, that's that's like halfway of getting there. Because we were speaking about how do I uh describe my application, how do I deploy it, how do I make uh those resources in the in in

the target cluster, but you will need like the additional utilities running in the cluster. Yeah, you want uh your application to be reachable from the outside, you want it to have uh certificates, you want to collect logs and metrics so the developers know what is happening in the system, and you need to keep maintaining that infrastructure regularly. So, uh what I can do now is I can

go and explain the other steps that we need just frankly like yeah, we use this, we use that in a way that will go and analyze uh the poem again. Uh but for that let's just see like what was our approach uh here. Our system will be just using application binaries. You won't be creating Docker containers, you will uh upload a war file there or a or

a jar file of a Spring Boot application or zip uh folder and specify the kind of major configuration aspects, what JDK you want to use, how much CPU uh you want to give it to it. That will kind of imply how much memory it will get, what's exact uh garbage collector setting, and uh and other optimizations. Then the generic set of uh of properties that that we

should pass to the application and how it maps uh to domain and and context root. And we will be running a pre-configured Kubernetes cluster with those running services that I mentioned now, yeah, for for for ingress and log collection and metric collection. And you will get a nice UI that will tell tell you this is the endpoint, this is the metric the application has. You can get

the logs, you could request a thread dump or a heap dump from a running application. So, the way we done it and now we are getting to the analysis of the poem. I think you surely loved it in school, yeah, to um, break down the poem on what the author meant. since we didn't accept like Docker image on input, we will create our own Docker images, we

will pre-optimize it for the runtime uh, we will be using, yeah, so uh, application class that are sharing would be used to kind of pre-warm the class loading parts, which is what takes about like 60-70% of your startup time. Tends to be just reading through the classes, so uh, that helped a lot. And we will have like for combination of what server versus what JDK version roundabout

roundabout six to nine base that we will update every 2 weeks with the latest patches. And then we will find a way of how to apply the configuration to each runtime, for Jakarta EE we will have Payara Micro and uh, kind of inject the configuration at the right place. Uh, for Spring we found that the best way uh, there is this uh, environment variable when you can

create JSON and that overrides all other configuration sources, so that was the way of applying there. And for Quarkus we wrote an extension that you had to use. So, we configure things like having unified logging output across all all the runtimes. Right, then we would have a custom controller that we've written to create all those Kubernetes resources and report back to the management UI what's what the

status is. And the concept would be every namespace have a domain assigned to it as as in the host name. the different applications with with different bots would uh be living on the different context roots. We will have auto-scaling clusters. and supported both Azure and AWS over multiple regions. And yeah, one another aspect of the configuration that that you need to consider is that not Yeah, you

want to keep some things uh unaccessible. Yeah, for for somebody who is not an admin within the cluster and you would create that as a secret. Uh so, yeah, we had to a way of telling for each configuration property is it a normal property or is it a secret property and then we will treat it accordingly. And yeah, supported custom domains. As for ingress controller, what worked

for us was traffic and it worked for us exceptionally well. It will blend well with uh with uh Let's Encrypt. and in many cases it will be like more important more performant than the native load balancer you would get from uh from cloud. So, it definitely was faster to respond than than Azure application gateway, for example. We would uh DNS, especially if you used custom domains, yeah.

Well, no, custom domains is isn't relevant in this case. we would update external DNS, but we would have like a wildcard certificate for for our domain that would be like pyra.app. So, you wouldn't get uh I don't know if you noticed, if you're running an an application that that is protected by Let's Encrypt, the moment you renew your certificate, you get like a swarm of bots running

at your and and trying to probe uh what are you running. So, for our default domain, we would have a certificate like start.pyra.app. So, customers will don't be subject to scanning every time they they they create a new namespace. And it was a multi-tenant system, yeah. So, we would kind of confine uh the application so they can only speak within their namespace and then to the outside

world. And we would employ AppArmor to further restrict what a Java process can do. So, you really couldn't execute another file or or reach out to the other pilot parts of the file system. Then, to getting logs in, uh we we would ship the logs into cloud's appropriate uh place. So, it would be like Azure Log Analytics on on Azure, CloudWatch Logs on AWS, and also getting

like Prometheus managed Prometheus service. Inside the cluster, that required that we run Fluent Bit, that we run Prometheus agent, which was set up, you will Yeah, what user would get Yeah, this was all an implementation detail. Yeah, from user's point of view, you would deploy an application and the application runs. And you will get this web browser interface or or CLI uh to go in front of

that. We will support scale to zero with implementation similar to to Dorman few slides ago. And uh yeah, all of that was like large one large Pulumi stack that would bring up the world cluster up. And you will just go and deploy your application into it. So, this is how it works. This is how it can be simple. And maybe one day we'll be able to share

more of how exactly to achieve that. So, in summary, uh it is possible to create a standardized Java environment that will fit uh your requirements for for abstraction quite precisely. But there are lots of tools that that that you need to do to that you need to use to compose that stack together. So, to put it in more poetic way, it starts with the containers, but in

the end it's that's what's the poetry as or which is better known by uh its abbreviation PaaS. And that's all I got. >> [applause]