Scotty I need Warp Speed - Ways to improve the JVM startup/warmup, Gerrit Grunwald

In this talk, the speaker discusses the intricacies of JVM startup and warm-up processes, emphasizing the factors contributing to perceived slow performance, particularly in container environments. He explains the loading of approximately 1,200 classes during JVM startup and how performance can be affected by application initialization times. The session delves into class loading mechanisms including various class loaders and their visibility principles. Additionally, the speaker covers the execution engine, including the roles of JIT compilation and optimization in enhancing execution speed. He also introduces several solutions to address startup delays, such as CDS, app CDS, Project Leyden, GraalVM's native image feature, and Azul's unique approach to pre-compiling optimized methods. These methods aim to reduce startup times, improve application responsiveness, and address warm-up performance challenges.

Okay. Yes, now we can hear me. So, it was a coffee break. I hope you get some more energy to listen to this amazing talk. So, whoever got this um printed schedules, programs, throw them away. They are not worth anymore because uh unfortunately Roberto Cortez was not able to come uh from from from uh Portugal. So we had to find a replacement and I believe that this

is one of the best replacements you will ever find. S [laughter] So uh how many of you have watched movie called Eurotrip? Yes. And what is the famous song from there? Scotty doesn't know. Yes. So we are all Scotties and we don't know. And here we have amazing Garrett, our wonderful friend who's going to make more talks this conference. The more Garretts we have, the better the

conference. Yes, this is correct. And finally, after this talk, Scott Cotty will know hopefully. Well, at least he said so. So, please warm welcome Garrett. Thanks. All right. Can Can you hear me? Is it on? Oh, okay. Perfect. So, who knows Scotty from Star Trek? Because I figured out when I did the session that I'm probably too old because this is from the Star Trek when I

was a kid. There was Captain Kirk and then there was the B engineer which is Scotty, right? And always when he needed warp speed, Kirk said, "Scotty, I need warp speed in three minutes." So, and I thought this is the perfect title. And then I figured out, oh damn, a lot of people might not even remember that. So, okay, but this is Star Trek. I'm not a

fan of Star Trek, right? I'm a fan of Star Wars, but it fits. So, okay, let's start. And my name is Gumbad. I'm working for Azul, as you can see. We do JVMs and all kinds of stuff. Um, I love Java. Uh, I love this conference specifically. I was here at the first edition. Uh, this is now the 11th one. And who loves this conference? Yeah, it's

awesome, right? The gang of six did a great job. So, uh, this is all about JVM startup and warmup, right? And um maybe people don't know about it but JVM startup the JVM is often blamed for being slow starting slow especially in container environments is like everybody's like h so and I figured out that a lot of people have no idea what it really means when the

JVM start up. So let's dive shortly into it because we have the JVM startup itself right. So that means the JVM starts that needs already around 1,200 classes to load to get things going and then it needs to load your application right the JVM starts pretty quickly it's in the milliseconds and then we have your application that takes long time to load right all the classes initializing

and so on and then this is usually what we say it's JVM startup time to first response the first time your application comes back to you with an response right doesn't mean it's fast yet right we will see why. And then there is another thing many people really don't talk about which is JVM warm-up or application warm-up. And that's a totally different thing. For some it's important,

for others not so much. This can vary. So the red part can vary from milliseconds to hours. Right? So we have customers they warm up the application for hours just to make it hot. So but we will take a look at that too. So this is why people complain about Java is slow. Okay, this stuff happens outside of the JVM. So we have Java source code and

then we have the Java C compiler which is not really part of the internal JVM and then we have a class file which contains the the bite code, right? So that means the JVM itself doesn't even speak Java the language, right? It just understands bite code. So that's outside of the JVM. And then we have the JVM itself which contains of different modules. Let's name it like

that like the class loader subsystem. Then we have the memory subsystem and then we have the execution engine. And now we have the class file. Somehow it has to get into the JVM memory. Right? So let's take a look at class loading shortly because that takes already some time in the JVM. So this is the class loader subsystem. As you can see it has different parts like

loading, linking, initializing and then the loading again has different class loaders and then we have linking where we have verification, preparation, resolution and then finally initialization. So let's take a look at the class loaders because we have different ones. We have first of all the bootstrap class loader, the platform class loader, system class loader and there can be optional a custom class loader. someone using custom class

loaders here in the audience. Sometimes people do, sometimes it's needed. You don't have to. And there's some kind of visibility principle. So that means the bootstrap class loader can see down the platform. The platform can see the the system system can see the custom but they can't look up and there's a reason for that. And now if we take a look at how they are implemented, the

bootstrap class load is written in C++ this native code and then all the others are written in Java, right? So now if we get a request to load a specific class then let's say it comes in here and then it will be passed up to the bootstrap class loader and then the bootstrap class loader tries to load it from Java home or J mods. So that's Java

base object and so on and this is also the reason why we have this visibility principle and why it has to go up the chain first. You don't want to override the Java object class, right? And that's the reason why we first try to load it from here, then you can't override it anymore. And um if it can't find it in Java home somewhere, then it delegates

down to the platform class loader. Then this one tries to load it from the Java SE platform APIs. This is Java util, Java.net and and these kind of things. If this can't find it, delegates it down to the system class loader. And that one tries to load it from the class path or module path, right? So com.zul or whatever you have and if you have a custom

class loader and the system one can't find the class then it tries to load it from the custom class loader and this one can load it from wherever you like right where how you define it. So and then if either the system class loader or the custom class loader can't find it either it will throw class not found exception. These two can throw these exceptions. You might

have seen that at in your developer career at some point. So this is in principle how the class loading works. So this is the way of how we get the bite code into the memory of the JVM and you can see just the loading is already takes some time because we have to find the binary file first then we have to decompress it from the archive load

the actual file and as already mentioned the JVM itself needs around 1,200 classes files to load just to start the JVM to use it. And then if you have something like spring boot, is someone using spring boot here in the audience? Probably no one, right? See, and this comes with tons of classes. So that means you have to load all this stuff. Find the binary file, decompress

it, load the file. Then once we have that, we have to link it. Therefore, we need to verify if the bite code that we loaded is valid at at all, we need to do some preparation. That means initialization of static fields and with defaults and stuff like that. And then resolution resolve symbolic references in the constant pool. And once this is done, then we can initialize the

class means calling the constructor or the class loading constructor with the verification in the linking block. It could also kick off loading dependent classes, right? So that really can you load one class that depends on another one and so on. So you load just for one class, you load others too. So the whole process is quite time consuming. It takes some time. the class ning and um

it's a huge time by the way and therefore we have some project but I will come to that okay so that means now we have it in the memory we don't talk about memory here if you're interested in how the memory management works I have another talk later today about garbage collection just this is more about memory management then so that means we have the stuff it's

somewhere in the memory now now we need to execute it right that's the next big step So execution engine contains different uh so-called template interpreter. It it has also another interpreter but we don't talk about that here. It has two JIT compilers in open JDK. Someone using OpenJ9 from IBM Seamaru because this has just one compiler. It's a little bit different. Um where the Open JDK has

two compilers, right? C1 and C2. Then we have a profiler. We have different garbage collectors. Again, OpenJ9 has different garbage collectors than Open JDK. They work very similar, but it's different thing because it's a different implementation of the JVM spec. And then we have native method interface and native method libraries. The two compilers in open JDK work together since JDK9 officially. It's so-called t compilation and we

take a look at this too. And I try to explain it in a way that everybody understands. It's just basic understanding. So we start with interpreting byte code right. So that means we have the the bite code and an interpreter goes through the bite code line by line interpreting this stuff and we profile it and we let's say we count method calls method invocations. This is just

one thing. It's also if you call code in a loop this will also be uh counted and then there are specific thresholds in the JVM if they are reached right if the method was called let's say a thousand times then the JVM decides okay this method now is warm or lukewarm let's take it to the C1 compiler that's the first compiler that we have this compiler compiles

quite fast that's formerly known as the client compiler in the past and Then we have some profiling phase. So the compiled method and by the way a method is the smallest unit in the JVM that we use for compilation and optimization. So we profile that code again and then there's another threshold of method invocations. If this is reached then the JVM decides okay now this method now

is really hot. Let's make it fast because it seems that the application uses it a lot. So it takes the method with the profile information put it into C2. This compiler formerly known as the server compiler way more complex takes longer to compile and then there's also optimization involved. The C1 compiler doesn't do a lot of optimization. C2 does a lot of optimizations and then there's still

this gap right between super fast code and interpretation. And this is the optimization. Does someone know what it means? Maybe a few. If not, I will shortly explain it uh after this stuff because that means there is a a way to go from super fast code to super slow code. Again, sounds weird and it makes sense and we will see why. So interpretation uh we I talked

already talked already about the template interpreter and that's a very specific thing because even the interpreter in the JVM is already fast and the reason for that is we have around 256 possible byte code instructions in the JVM. I think JDK25 uses around 202 of this. And for each of these instructions at runtime, every time you start up the JVM, it will create a machine code template

for each of these bite code instructions. Right? So as mentioned 2566 are possible 202 roughly used in 200 in uh JDK25 and these will be stored in uh in a cache of the JVM. Creating these templates on this machine takes around 0.4 milliseconds. So it's super fast and you can even if you type in this java minus xlo col startup time you will see how long it

takes and if you are they will be stored in the code cache and if you're interested in the templates itself you can even take a look at those if you're interested in that then you really see the hex code templates for each bite code instruction. So that means now we have for each byte code instruction that is that we loaded with the class file we have a

machine code template for this specific machine. Why do we do that? If I run that on, let's say, an Intel i3, then the i3 CPU has the ability to optimize specific or use specific optimizations. Okay, the JVM knows that. It recognize the the CPU and applies specific optimizations that are possible on the Intel i3. If you now have an Intel I don't know what it is, 9

10 this can do more advanced stuff. So the JVM knows that and makes use of it. So that's the reason why at every startup these templates will be created. Okay. On the left side in the red uh headed block there's the bite code for an add method. So we have two i loads which means integer load. That's the bite code instruction. And then we have an i

at command at the two integers and then we return this value. So on the right side we see the segmented code cache. So that means the code cache in the JVM has three parts and on the left side the non-method code cache. This is where the templates sit and then we have the profiled code cache and then we have the non-profile code cache segment. So if we

step to the first row then this is the template that will be executed. This is the machine code template right? So and then we go to the next one then this template will be uh executed and then the add and then so on and so on. And this because it's an interpreter, if we caught it a lot, it will always go through these things. But because it's

already machine code, it's already quite fast, right? There is another interpreter which is a C interpreter that interprets uh takes the bite code and compiles it at runtime. That's slower, but it's more flexible for specific use cases. But this is the default, the template interpreter. Again, interpretation is okay. This is already quite fast but still interpreting code is not the best thing to do if you really

need performance. let's take a look at t compilation. That's the two compilers working together. Um the green line is our interpreted code running with a specific performance. Let's say that is one method that will be executed. Now we execute that a thousand times and then as I already mentioned it's warm. So the JVM decides, okay, let's take it to the C1 compiler. So we have a short

compile step and then we have around 10 times as fast code as the interpreted one. And then again after around 5,000 method invocations, this code will be taken to C2. And you see compilation takes longer in C2, around four times longer than C1. And the code is on average around twice as fast as the C1 compiled code. But this is where we get best performance at the

end. At the upper right corner, you can already see this is a staged process. It's very flexible because it depends on how your application is used, right? So it can really adjust every time you use the application in a different way, it will adjust the the compilations for that method. So this also means if your method isn't called a thousand times, it will stick an interpretation forever.

It doesn't matter because if you just call it from time to time, it's okay because compilation takes CPU time and memory. So if you can avoid it, we avoid it, right? It can also stick in C1 compiled code which is already fast. But if for some reason the method wasn't called more often, then it will stick in C1. But if it's really hot and used a lot,

then it will go to C2. Okay. So, and here we see the different uh segments of the code cache as already mentioned template sitting in the non-method part. Then the profiled codes is in the C1 code is in the profile segment and the C2 is in the non-profiled code segment of the code cache. So that means it will be compiled sticks in the code cache and then

the next time you call it, it will directly be taken from the code cache which is super fast. Okay, so this is tiered compilation. In seamaru, that works differently. In principle, the same idea, but the one compiler makes the decision. Oh, the method wasn't called often enough, so I just don't optimize it. And if it's called very often, I optimize it. So C1 compiler, fast compile, just

a small optimizations, uh, produces code, slow compiled code, it's 10 times faster than I interpreted and where it is stored. And then the C2 compiler slower compile very aggressive optimizations and also speculative optimizations which is something we will talk about in a second because that means the compiler speculates how the code will be used in the future and does optimizations according to that. So speculative optimization and

deoptimization which is also quite interesting. If we take a look at that code, right, you don't have to understand it's not very fancy. The part here is that we have a method that takes a value and based on that value, we calculate a bias and then we return the mathlo 10 from the bias plus 99. So now the JVM runs that code and it realizes that the

value was never greater than 9. So it's always the right path of the diagram that will be So why keeping the whole code right? So what it does it does this it removes the upper part and replaces it with some kind of a back door. It's called uncommon trap. That means in case something goes wrong we have a back door. We call this uncommon trap. But first

of all we set the bias to one. Doesn't matter what value you put into that method, it's always one. And then we further optimize because we can do like this that and if you know math, it's return two. That means whatever value goes into that method, it directly returns two. This is obviously way faster than going through the whole if then and so on, right? But the

problem is if it goes wrong, right? In case somehow suddenly the value is greater than n then we have to do something because that is not true anymore. Then we have to call the uncommon trap and that leads to the fact that the the code the optimized code will be thrown away. So it goes just in the trash and then we go back to the original version.

This is one solution and this is where we go back to this in the circle back to the green interpretation to the slow code. there might be a chance that in the C1 compiled code of that method that is still in the cache. So the JVM tries to use that one if possible. If not it goes back to interpretation. But this also means at the startup of

the JVM and I have a chart later on you see some really it starts performance grows and then suddenly drops and then grows again and drops again. This is the optimization. The compiler tries different things to make it super fast according to the usage of the application. This is branch analysis. There are different ways of doing speculative optimizations. But you get hopefully the idea. So all these

optimizations that we saw, they lead to awesome performance, right? The JVM is super fast once the code is hot. Uh the problem is um we still have some slow startup. So it takes some time to get to that super fast code right. So and therefore we have to identify some slow zones that we have which is typically class loading interpretation takes time compilation and optimization. So these

are the parts that we identify that are taking time until we have the super fast code. So what are the solutions to these problems? We have different ones. Um we start with CDS or app CDS. Then we have project leen native image which means growvm track not the drug but it also is addictive uh ready now and there is something like cloud native compiler IBM JIT server.

I will shortly introduce all of these features and then show you some measurements that I did and give you some advice where to use which one. Okay, CDS app CDS. Does someone know that this are already quite old? It came with JDK 12 and the CDS really came with JDK 12. I think it started in JDK 5 already, but 12 was the first usable version and then

app CDS came in 13. And the idea is I showed you the diagram where we have to load all the classes, right? That means you have to find the file, unpack it from the archive, load it and that takes time. It's file IO. If you have thousands of classes, it takes really a long time to load the stuff. What if we just instead of loading every time

we start the application, we load all the files, we load it just once, put it in a specific format that is, let's say, JVM memory friendly and save all of the classes in one file, right? So the next time we start, we just load this one file directly in an optimized format into memory. That's way faster. Obviously, this is CDS for the 1200 classes of the JVM.

This is what CDS does. So, because it loads always the same classes, why not share them, right? So, why not do this in one file? App CDS does the same thing just for your application. So, if you always load the same application with your service, why not do the same for your classes in your application, right? So, this is application CDS and um with the test with

the Spring Boot Pet Clinic, you can reduce the start time around two to three seconds just by using this feature, right? If you don't know that and you are not on JDK 25 or higher, probably not. Um, everyone from JDK 12 to 23 can use that and it it works. It's great, right? You just can look it up how to set it up. CDS, you don't have

to set it up. app CDS you have to right because this is application specific you have to train the JVM you have to at least load the classes once that creates a little archive and then you have to deploy this archive with the JVM and then it will load it the next time from that file um this will be replaced or was already replaced with with project

laden right so since JDK 24 we have project laden available but because 24 is a short-term support version you should stay on 25 and there you have it. um where it works. Remember this is the class loading principle and app CDS and CDS works uh here at that part right so it helps with the loading classes and these are the JS the JDK enhancement proposals that are

involved in that so you can look it up on the on the web if you're interested what they actually do um which leads me to leaden who knows about project leaden which is it's quite new but it's it's good this is like CDS on steroids, right? So they took the idea and they even used the same acronym CDS, but now it's not uh class data sharing, now

it's cache data storage. But the idea is pretty much similar. So instead of not only loading the classes, we also try to do the linking up front, right? Because that means the verification and all these things. And in the future it will also do something like AOT compilation. This is not growm AOT. This is you can think about you need you load classes and then you compile

it. Save the compiled class. Then next time you start the JVM instead of doing the whole compilation just load the compiled code back. That's [clears throat] very basic idea. This is not there yet. It will come in the future. But at the moment we just have um the loading and the linking part. So that means if we take a look at this diagram again then this is

where leaden works right. So it uses the loading and the linking and then creates this specific I think they name it AOT cache file which is just an extension of the the CDS file. And there's another advantage now in leaden where if we have the interpreter and it interprets the bite code, it writes the profiling information in a profile cache and then it goes to the C1

compiler and the C1 compiler also writes this profile information into this profile cache and then when the C2 compiler starts compiling it and can directly access this cache file from the start right from the last run. So that can help to optimize code directly instead of waiting until the profile is done. Right? So this all these things this is uh Jeep 515 and that al helps to

improve the startup time and we will see what that means in seconds or milliseconds later on. Next thing is native image. So this is Gravium. Who's is someone using Gravium in production here? Okay. a few gravium is a different approach. The idea is you take the Java code the actual Java code and directly you do a stat static code analysis and compile it down to a native

binary right ahead of time. It's not at runtime. So that means there's no interpretation, there's no JIT compilation, just in time compilation. There's ahead of time compilation here. So that also means that it's it's harder, right? Because you have to guess what methods do actually at runtime. Where the JIT compiler can do profiling, figure out how the method is used and optimize it, the native image can't

do that. There is one way of doing that which is named profileg guided optimization. This is you run the application once in the gravium jit compiler create a profile how it was used and then you create the native image based on that profile information. So it means you can do some kind of a one snapshot usage and then optimize it for that. It works for a lot

of things uh not for very let's say applications that will be used in different way all the time there it doesn't really work that well. Um yes there are some drawbacks using gravm. First of all reflection it's not really the best thing you can do with ahead of time because reflection run is happening at runtime. So that means the grav compiler needs to know all the classes

that will be used at runtime and he needs to know that up front. So if your application loads stuff on the fly [clears throat] not so easy, right? There are ways to do that but it probably needs more effort to get there. And also yes, you gain faster startup, but you probably move the whole let's say effort from the JIT compilation to your build process because building

that native image takes a lot of CPU and memory and this is probably have to beef up your your CI/CD system and debugging is also not so easy because it's native code and no Java code anymore but you get really fast performance in startup. So we have two different versions. So we have Gravium community and we have Gravium formerly known as Gravium Enterprise. Um the community version

you can use for free, right? No problem here. There's no profile guided optimization and also the compiler is not that powerful as the one in the former enterprise version. Unfortunately, you have to be aware of licensing because this is an Oracle thing and Oracle pulled the plug on supporting Gravm in their JDK and so it's a lot of hassle going on internally. So I can't really tell

you what the actual license is of Gravm Enterprise and how the support system is for that. I just know that it's it's questionable at the moment. So you have to be careful with the enterprise version. This this is for sure. The community version you can use. That's no problem. So crack um that means uh coordinated restore checkpoint. This means you have your application running in the JVM.

You run it until some specific point and then you just create a so-called checkpoint which you can think of like a snapshot of the JVM. You stop it and you save the complete state to disk and then you can restore and the con the application will continue running. It's not a restart. It continues running. Right? So you can uh get some uh super fast startup times with

this stuff and you still keep the JVM. So the idea is you start up your JVM, you warm it up and then you have a fast JVM. So you create a checkpoint here, save it to disk and then the next time you just run directly from there. Right? This is the idea behind this coordinated restore checkpoint. That means you need a specific JDK because this is not

part of the main branch of open JDK although it is an open JDK project. So this is a little bit more complicated. So the frameworks supported more or less most of them good because most of the people use Spring Boot they have pretty much uh the best support for crack. Helon has good support. Open IBM Liberty and all these things you see here they they support crack

so you can use it if you want. Um, the next one is probably pretty unknown. Does someone know already now? Ever heard of it? Probably not. Um, it's something that Azul did. And we have our own JVM with our own compiler. And the idea is a different one. This is specifically made for solving the warm-up problem, right? And the idea is I told you the JVM interprets

code, then it compiles code. Yeah. first with the C1 compiler, then with the C2 compiler, then it's super fast. [clears throat] So the idea is for each method that the compiler compiles, we save the method that was used to optimize it. So the optimization method that was used to optimize the Java code that's what we save in a file for each method with some kind of fingerprint

of that method so that we can identify it and the next time we start up instead of doing interpretation and compilation we directly take the method and compile it with the last known optimization method. So ideas as follows like in Leaden we have a profile cache we just have it for more than 11 or 15 years but um that's a different story and then the idea is

this the C1 compiler takes the method takes the fingerprint and say oh I optimized the method with this specific optimization method then the method gets more hot and it gets into our Falcon compiler which is more or less the C2 compiler on and it takes the same method and does a different optimization and it upsert this method type in the compilation log. Okay, so we have one

compilation log that contains all these optimizations for all of the methods that have been used. So the next time we start up the class loader loads the class and then it checks in this compile log, oh is this class with a method in the lock? If yes, directly compile it and use the the compile method that was used to optimize it. And with this, we can make

sure once you run your code and it's super hot, the next time you start, it will directly be compiled with the best possible way. Right? So this is the main idea. And we can of course also make use of the profile cache same as in laden. Okay? Now this is the chart. This is actual measurement of a JVM startup. You see the the green part is the

interpretation period. Then we have the purple part which is C1 and then the blue part which is Falcon or you can think about C2. You see these gaps in the in this steep curve the bigger gaps. This is the optimization where the compiler tried stuff it failed went back up and so on. This periodic gaps on the right side this is garbage collection process right? They are

super small but it's a performance drop. Okay. So that means we need some time to get to full performance. If we use this ready now approach looks like this. We just com need the time to compile the methods at startup. But then we are super fast. We are directly at the full performance. And where that makes sense um because people sometimes have no idea why do I

need to warm up my application, right? So let me give you an idea. Um, think about the stock exchange and the stock exchange when you trade some stuff and you would like to trade it at 8:00 a.m. on Monday morning. That means directly when the stock exchange opens, you would expect that the transaction is super fast, right? Or would you think, oh no, they just started their

servers, it takes some time until the JVM is really hot, right? So transaction take longer. No, it has to be super fast from the beginning. This is a typical warm-up problem. And something like the stock exchange, they warm up the application, the trading stuff for hours with using specific algorithms to do so to get away with this having the super hot code directly. Right? Startup is not

a problem. They can start up and if it takes a minute, no problem. If it takes half an hour, also no problem. But the code has to be fast. That's that's the problem here, right? This is a warm-up problem. So, and this one is made specifically for these kind of customers. And then this is a completely different approach. Cloud native compiler which is something from zoom and

IBM JIT server pretty much the same idea. Um you can think of instead of using if you deploy in containers who does microservices here I don't believe this must be more people but it's okay. So if you use microservices right so you deploy your JVM with your application in a container and then you run and start up the container which means you start up the JVM load

the classes compile the code and so on. So you go through this ramp up in each of these containers. So if you think about and this is the bump in CPU usage that you see on each of the containers runtime you start it up you see a huge peak in in the CPU spike because interpretation compilation optimization and so once everything is compiled it drops and usually

settles around 40 to 60% CPU usage that [clears throat] means in the end to run your code you just need maybe 50% of the CPU But to have a fast startup, you overprovision it with more CPUs to get rid of this bump in the beginning. That means if this is your node and we go through class loading, just take a look at the CPU usage and then

compilation, we use all the CPUs heavily and then everything's in the code cache and once it's running, you just need two of the CPUs. Okay, then you scale that out. You do that on every node and that means you waste probably a lot of CPUs and also memory because compilation needs more memory. If you do something like this, you use smaller nodes and you use one central

JIT compiler. This is not a JVM. This is hard to describe because you see this little squares where it says JIT. This is a Linux process. it just for one method it will be called to compile a method and then it will scale down to zero again so if there's nothing to do then it just does nothing it just sits there and this could be thousands of

JIT processes that can be started to compile methods and then scale down again we have a bigger code cache right that keeps all these methods because what is the problem with JVM if we start it in a container it does all the compilations and optimization stores it in a code cache we shut down the container and the code is gone. If we start it again, we have

to go through the same thing again. Okay, with this with this bigger code cache, we give all these different nodes like a memory because the code cache a central one keeps the methods the compiled methods. So if a node needs a compiled method, instead of compiling it locally, it asks the centralized compiler, do you have that method? And then the compiler says yes I have it and

just send it directly back. The overhead is just the communication in the cluster. And this is usually super fast because this runs usually in the same cluster and only the first execution takes a little bit longer because if a method wasn't uh compiled by the central obviously it needs to compile it first. But after that if you scale out all the other services they need the same

method will directly get it get it back compiled. This is the the basic setup. So you have on the left side the local JVM in your node and on the right side the CNC or If the JIT server is not available, it will compile and run it locally. If it is available, it will take the the code from the JIT server. That's that's the idea behind these

uh cloudnative compiler or JIT server. This is not for everything but it can help if you have services that scale in an unpredictable way and you have multiple services and you don't know when they scale. One example could be you you're a bank, right? And um because I heard that story two weeks ago in Amsterdam. There is a bank and they have a payment system on mobile

phones and then there was a soccer club playing in Amsterdam and in the break everybody was running to the booth and buying something to eat and everybody was paying with this app and so suddenly the whole service was spiking like crazy that was unpredictable and then after half an hour goes down again. So services like that because they couldn't predict that this would happen for this kind

of stuff. This works quite well. Okay. So now we have we saw all these different solutions. So let's take a look at some tests and to make sure that we are all in the same boat. I ran all the tests on this machine. I use Spring Boot 3.3. The pet clinic. I didn't use four. Sorry, but there's not really a big difference here when running that stuff.

It's a MacBook Pro M4. Um, I ran on parallels because all the stuff runs on Linux. You see all this stuff and I didn't do extensive warm-ups. And one thing I have to make sure if you test these things then make sure that you restart the virtual machine or the Linux machine you run it on because Linux caches a lot of things. So I can show you

tests where you run the first iteration and let's say you get 100 milliseconds. Then you run it again, suddenly it's 50. You run it again. 030 and you get really down to 25 milliseconds because the operating system is so smart and say oh I know that stuff I just cach it right so this is you have to restart the virtual machine to do that so startup um

first open JDK and open J9 I mentioned that already they are similar but it's a different implementation of the open uh of the JVM spec just I made some tests and as you can see it's more or less the same performance performance. So, you won't really see big differences here. Just to make sure because some people are on OpenJ9 and um I would just like to make

sure that people know, okay, performance-wise, it's not really a big deal. It's more or less the same. Okay, if you just use a and the fat jar, the Springwood pet clinic took on this machine 3.2 seconds to start. If I use a exploded jar, that means instead of using one fat jar, I just extract everything in folders reduces the startup time already by 1 second. Keep that

in mind. If you deploy in fat jars, use exploded ones. It's it's faster definitely. But this is just the JVM and the jar file. now using appcds and in spring boot there's an AOT feature. It was made for growvm but appcds really benefits from it. It went down to 1.6 seconds using just abcds. when I use leaden and that's quite interesting because when I I you saw

I used two cores and four gigs of RAM because this is more or less four gigs is might even be too much but I use that because this is a desktop uh Ubuntu that I used if you use microservices then you don't run it with eight cores usually except it's a huge service but usually have small services with small CPUs so that's why I use two CPUs

if I use two CPUs [clears throat] Leen took one or two and if I gave it eight cores then suddenly it dropped to 900 milliseconds. So, Leaden was the only one of all these things that I tested that really benefited from more cores, right? Where the others it's more IO bound, doesn't really benefit that much from or don't really benefit that much from more course, but Leaden

did. But because I saw it at Spring.io, they they showed the Pet Linux startup with 900 milliseconds using Leaden and I was like, "Wow." Then I thought, I I just got 1.2 seconds. So, I just increased the course and bam, there was the 900 milliseconds. So leaden you have to be careful if you use more cores it's faster if you don't you might get slower results. Um

Gravium community the native image startup was 400 like 500 milliseconds and you can see the enterprise compiler is better and I didn't use the uh profile guided optimization and it went down to 280 milliseconds. Right? So you can see there's a huge difference in the two compilers in the community and enterprise version. All the other things that you see now this is all different implementations of crack.

Um if it says instant on this is the IBM openj thing. If you all the others then there is crack using creo there's crack using warp different engines. This is not the point. The point is that you get more or less the same speed that you get with a native image but you keep the JVM. Right? So this is just the startup time. If I would only

show you this you would say wow crack is cool. Wait, because this is just half the story. I told you that many mainly for all of these solutions we needed to create something like a profile file or cache file something like that and you have to deploy that with the application and that means it increases the memory footprint for the for the artifact. Right? So let's take

a look at the memory footprint of all these solutions. Um the memory footprint. Yes, that's that looks terrible. I know. Um first of all, this is the JAR file 61 megabytes. Now the interesting part, this is the The JRE in Open JDK of JDK25 the Java runtime is 206 megabytes. So now my question is who's using a JRE for deployment here? Don't be shy. probably most of

you I hope you don't use a JDK JDK is around 400 something megabytes Samaru OpenJ9 is a little bit smaller then you see something called JLink this just 18 megabytes and just as a little reminder JLink is part of Java since JDK9 and it's a way to shape a runtime for your application what it does it just copies all the stuff that it really needs to run

your application. And for the pet clinic, I just needed 80 megabytes. Why should I deploy 200 megabytes if I can do the same with 80, right? [clears throat] And you can embed that in your CI/CD process. It just takes it's two lines in a script. It just copies stuff together. That's all. Super fast. So I use JLink because I would like to make it smaller. Sorry. And

then um this is the the files that I was talking about and this is uh the different files like the checkpoint in the first version was 200 megabytes. It's more or less a heap dump in crack. Then instant on same thing from IBM was 140. And then all these different things you see the AOT cach cach and laden was 130 megabytes. [clears throat] If you use uh

appcds this was 89 megabytes. These files you have to add to the deployment. All right. So this because if we now combine that we would like to reduce the startup time and also the memory footprint. So I just created a version like uh best combinations it's this appcds if you are not on JDK 24 upwards then abcds is the way to go then there is no other

solution here. If you use leaden as I said it can get further down if you use more cores but um then this was this uh solution then the gravm community [clears throat] and you see gravium enterprise is smaller even if the deployment artifact was bigger startup time is is faster but the the thing that I would like to see here is or show here is you can

get down below one second startup time with regular Java or even with a native image and the deployment artifact can be still okay from the point of view of the size, right? So you can really shrink it down. which solution is for you? This is the the big question. Now um first of all you need to know what is really your problem. As I mentioned there could

be a startup problem. This is typically something like you have a web shop. the user clicks on the icon or opens the app then it should be there super fast right you don't want the user be waiting like oh come on let's load the page right it should be there and running once the user has the page then it's he starts usually searching and that is not

performance uh dependent right because it can be oh I typed something so you have time to do something in the background so this is a typical startup um the warm-up problem I already mentioned is something like trading companies and stuff like that. [clears throat] There's even both. If you have a gaming platform, then usually you have multiplayer games. The game is already running. Now you have a

new user that would like to join the game. That means it should start super fast and he would directly interact with the others. It needs to be hot too. So you need both fast startup and warm up. Right? So these are the two things you have to keep in mind. identify your problem and as an overview it's like this. So you have the startup on the left

where you have CDS app CDS laden laden later on with the AOT compiled stuff can also [clears throat] help with warm-up. At the moment it's more startup time. Gravium community is probably just for startup. The enterprise version can help even with warm-up if you have profile guided optimization. Crack depends on when you create the snapshot or the checkpoint. If you take it very early then it's just

startup. If you take it very late in the process, then it obviously also helps with warm-up. If you uh combine crack with ready now, you have both because you can first of all save all the methods optimizations that have been used and then you create a checkpoint and then you have super fast startup with super hot code. It's possible. Cloud native compiler and jit server are more

on the warm-up side, right? It's not really the startup because we have this little overhead talking to the central compiler um to the code cache which takes a little bit longer depending on the communication in So let's take a look at the time. I hope I'm still good. Um because that's it. If you have questions, I don't know if we do we have time for questions or

are we out of time? I don't Oh, we are out of time. So um I will be here like I said I have another session later on about garbage collection. Just if you have questions come to me and ping me. [applause]