Tuning the JVM for Performance: 10 Optimizations Every Developer Should Know - Gireesh Punathil

This talk addresses the ongoing need for performance tuning in Java applications, even after decades of Java's evolution and its integration into enterprise workloads. The speaker discusses various aspects of Java's garbage collection, JIT compilation, and the importance of fine-tuning JVM parameters to cater to specific use cases. They present ten key performance tuning techniques, including escape analysis, thread local allocation buffers, and class data sharing, each offering measurable improvements in application performance. The talk emphasizes that understanding application characteristics through profiling is crucial to applying effective optimizations. Furthermore, it highlights the evolving nature of applications, where modifications may necessitate re-evaluation of performance tuning strategies, making it an ongoing process.

One of the question that comes automatically in the mind is Java is 30 years old and so is the enterprise. Enterprise has kicked in 90s and Java also in 90s and Java was lucky uh to adopt most of the enterprise world. So, the question is over the last 30 years Java has been um familiar with most of the enterprise workloads and use cases and Java knows everything

about the workloads. Why we are still doing performance tuning? Isn't it Java already knows all the best possible performance for these use cases? Or is it is it that Java is inherently weak that you need to tune after it has been released for your workload? Why why is that we need to do the Any answers? Mhm. Okay. Yeah, that's a that's a good reason. Any any other

reasons? Okay. So, that's a weakness in the JVM. Because garbage collection is part of the JVM itself. Yeah. Yeah, there was some other response. >> Flooded. >> Yeah. I think all of you answered correctly in parts. The correct answer or the best answer is that Java has been evolving. It has been incorporating all the use cases, all the best possible optimization technique, catching up with the technology

trends, catching up with the business your business use cases, the workload types and the deployment targets like the cloud and other things. But still there are hundreds [snorts] of thousands of different use cases, right? If two banks with all same design, same architecture, same percentage of the FDs and every transaction looks similar, but their applications will be different. Right? If you write a software for bank A

and bank B, they will not look similar. So the garbage collection characteristics, the the code optimizations, everything can be totally different. So JVM internally uses thousands of heuristics, hundreds of defaults, and these defaults are not going to work for all the spectrum of the use cases. They have done the best effort to catch the most common use cases, the characteristic workloads, but still you need to fine-tune.

So that's why we are talking about uh JVM fine-tuning. So what I'm going to cover today is I uh show my top 10 performance ticks. Uh it's not necessarily the best one for every workload, but based on the increasing order of the impact that could make on a characteristic workload. That's what I'm going to showcase, and then we'll have some summary or the key takeaways. So the

first one or the number 10 is escape analysis. What the problem? If you look at the Java application, the characteristic workload is a web workload. In web workload between the request and the response cycles, there are thousands of objects that get created. Tiny objects, it could be string passing, it could be number calculations, or it could be the HTTP header manipulations or processing whatsoever. All these objects

are created and destroyed in a jiffy of a second. So, do they need to be created in the heap? What is the problem with objects being created in the heap? Heap is heavily contended by multiple threads. So, you need to access the heap by putting the lock. You need to access the objects again by lock. You need to garbage collect the object with the lock. So, it's

heavily contended space the Java heap is. If you can somehow establish that these tiny objects are not going to escape the thread's scope, it could be just created, processed, and discarded. Why to create in the heap? You create it in the stack as if it's an automatic variable. So, that's called escape analysis. There is a specific flag in the JVM to enable that. For HotSpot, it is

do escape analysis. For OpenJ9, it's on by default. Anybody knows about OpenJ9? >> I just heard. >> HotSpot is a JVM developed by Oracle, earlier by Sun. OpenJ9 is a JVM that is developed by IBM. Uh though there are several Java JDK distributions, these are the two well-known JVMs. So, the Azul has a JVM, Red Hat has a JVM, Google has a JVM. So many other vendors

have JVMs, but they all use HotSpot JVM. And the other type is the OpenJ9 VM. So, they have subtle difference in the way they behave while they meet on a common specification called JVM spec. So, with escape analysis alone, we are claiming that 5 to 15% of GC pressure reduction on the JVM. Number nine, code cache sizing. So, this is about JIT code. So, as we know,

Java interpreter is inherently slow because every code needs to be interpreted line by line or a byte code by byte code and translate into the platform specific way because JVM abstracts the underlying platform. It doesn't use the CPU, it doesn't use the registers, it has its own Java stack, it it has its own variables and locals and things like that. So, that simulation is costly and it's

going to be very, very slow. So, because of that, we have just-in-time compiler which is a dynamic compiler. And once you compile all the methods, you need some space to keep it. So, optimization is good, but what if you don't have sufficient space in the uh perm gen or the code space, what happens? The JIT thrashing happens, the compiled code is removed and you go back to

the interpreter or you go back to the less optimized code and thereby hitting the performance, right? So, profile your system, understand what is the maximum size it should be able to reserve for holding the JIT compiled code and thereby get the optimal performance. So, by uh tuning the reserved code cache size, you're able to see around 2 5% throughput saving in the JVM. These are again taken

from standard benchmarks, might vary from your case to use case. Number eight, thread local allocation buffer. So, this is an extension or a flavor of the escape analysis that we talked about. While escape analysis is completely avoiding the heap allocation and put everything on the stack, thread local allocation takes a subset of that. You can still allocate in the heap, but does the allocation itself need to

be contended? That's the question. So, object could be shared by multiple threads, but does it need to be allocated with a contention on? So, it doesn't need to be, then you create in the thread local heap, and then later graduate the thread local heap to the main heap pool. So, that way what we are saving? We save a little tiny performance by avoiding the lock contention at

the allocation phase. So, uh empirical evidence shows that 5 to 10% pressure reduction on the GC side just by making use of the TLAB. Again, uh specific flag for HotSpot and on by default on OpenJDK. Class data sharing number seven. So, uh this is very very interesting, very critical performance uh characteristic. As we know, Java class loading is by far the most slow uh activity within the

JVM as a side effect of the platform abstraction. It requires, I think, around eight or nine or 10 discrete phases. Think about a class loading, you need to locate the specific directory where the class is in the by the name space, by the name specification. It could be a directory, it could be a JAR, WAR, EAR, whatever. You need to unzip the content, then locate the class,

load the binary, create a class in the JVM space, uh deserialize, and then you need to verify every bytecode because the Java security spec says that every bytecode should safe enough to run within the sandbox. The integrity of the JVM should not be affected, so on and so forth. So, every bytecode undergoes verification, and then all the static variables of the class need to be initialized. So,

some code execution, and the the interfaces, the abstract classes, the hierarchies need to be established before class A gets loaded. Class A's parent need to be get loaded first. So, all those hierarchies need to be defined. So, it's it's absolutely a slow, slow, slow operation to load a class. Now, think about a scenario where you are in a serverless or a function as a service scenario. You

have hundreds and thousands of class get loaded in the bootstrap phase, and if you're spinning up another container instance, you need to repeat the whole process again and again, right? So, what is the optimization that could be possible out there? Load in one instance, and keep it in a shared cache, so that it could be reused by successive instance of the JVM invocation. So, that's exactly CDS.

HotSpot call it as X share on. Open J9 call it as shared classes. this is a huge performance win, especially in the container environment. We see around 30% faster, mostly for startup. So, the use cases where startup performance is absolutely critical, you can go for the class data sharing. >> They said they did. Start with the data. Now, the containers they start, they share the data. Yeah,

I'm sorry, it's not got it. it isn't. Pick you up. How about then we tested actually on all the other options? >> So under the cover what happens? So when you switch on the flag the JVM creates a shared space like a disk. So all the loaded state of the class loaded state of the class would be a structure like you know JVM is written in C++.

So it could be a C++ structure or a C++ class which represents the Java class under the cover and that whole thing dumped as a raw memory image in the shared space. So the other JVMs when they boot up they know that we are on a shared mode so don't load from the disk through the traditional way. Go to the shared space and get it loaded. >>

It can be done in the >> Yes. So >> So I think here the pause about third. >> Right. Yeah. This is this is something which is much more suitable for the containerized Kubernetes kind of world. >> Okay. So you can >> I think there are there are ways to specify the shared location and there is a default as well. So obviously I didn't mention the whole

details 30 minutes session. there could be default. It could be the current location where the JVM is launching from or it could be a specific location like your class path. That's possible. Yeah. Container awareness this is again critical. we know that Java heap number of CPUs parallel GC threads parallel JIT threads, all these things you are not specifying, right? JVM takes a lot of assumption. Where do

JVM takes assumptions from? It queries the operating system. It looks at how many CPUs this system has. What is the total RAM capacity of this system? And based on that, it makes assumptions, it makes heuristics. Now, uh when you are running on a container box, um there is a clash, there is a conflict because container is is a subset of the OS. It doesn't get the whole

RAM, right? It's assigned based on what flag you passed at the container um uh spinning time. Now, unfortunately, Java queries directly from the platform, and it doesn't understand I am running on a container. So, if you have a 2 TB memory in the in the system, but the container was spun with uh say 500 MB, Java assumes that you have the whole 2 TB, and it goes

for big Josh, and it it What is the consequence? You get OM killed, right? So, um obviously, the solution is to make the JVM container aware that I am running within the C group with these limitations and these resource allocations. So, with that, the container becomes like a first-class operating system like, and >> [clears throat] >> Java performs well within the container scope alone. So, it's not

about the performance win or performance improvement per se, but it's more about the stability. No OM killing with the container awareness. So, HotSpot has a specific flag, but I believe that this is the default in the modern HotSpot JVMs, and in OpenJ9, it has been uh default since the beginning. Concurrency and parallelism. So, obviously we all know about virtual threads, but virtual threads have limitations. Anybody know

about that? >> Like >> Sorry? >> Definitely. Thread pinning. I mean, you can create millions of virtual threads in a in a box with a three CPU, five CPUs, or one CPU. No problem. I created million virtual threads with no no impact on the underlying system. Whereas in in that same system, you can create only 100 real threads, and your system will start jamming, right? So, it's

cool on the surface, but when you really exercise the platform specific threads, you will start seeing the pain. It gets pinned on the one particular thread, and then from within the JVM, if you look at like the GC threads and the JIT threads, um they need real threads. They cannot be virtualized. Because you need real CPU to perform the task. If it is multiplexed between too many

virtual threads, you're not really doing the task. You're just switching between the threads. So, virtual threads is ideally great for IO-based workloads, where there is a huge amount of waiting. But if you need real task to be executed in parallel, it's the green thread or the OS thread is the ideal one. So, now the real problem is in any any runtime, not just Java, Python, Node.js, everybody

has internal threads for GC and JIT, and there is a finalizer thread, there is a signal watcher thread, there is so many other threads which we don't know about. So, you can't make an assumption about the right number of threads that you need to parallelize your application with. Most of the time, what happens, you end up with either signing so many threads, in which case there will

be heavy contention, starvation of CPUs, or you will have very less number of threads, in which case you will have underutilization of your CPUs. So, the best thing is to apply this thread sorry, this flag by which JVM calculates what is the number of free threads and make use of those threads optimally, so that you get the best performance. So, the the promise is 20% improvement on

the throughput. Number four, heap sizing. So, this has been the ever since Java V1 was there in the market. We have two flags X M S and X M X, and what is the best practice? What is the ratio of X M S with X M X? I mean, this is one of the first Java lesson, but we always make mistake. Uh the the the guiding principle

or the best practice is to keep both same. Then, the question is why do you Why do you have it? Right? That was a design limitation, or the original thought was not great. The problem is if you have a 1% of the total heap as X M S and 100% of the heap as the X M X, what happens is your memory management system goes for a

toss. Every time you increase the heap, you know that heap is going to be increased. Your workload is going to be piling up objects, and your your heap is going to be an increased, and then you need to keep so many bookmarks and pointers and other internal records to say that what was the previous size of the heap, what is the increase, what is the percentage, where

do I keep the objects now, where did I keep the objects previously, and where do I move so many other things need to be known within the JVM. So, forget everything, keep both same. Um both the JVMs use the same flag XMS and XMX, and um just by doing this alone versus um different XMS and XMX, we see 30% improvement on the overall performance. So, this is

very subtle, very easy to tune, very simple, but people do make mistakes. what I can say is that 1990s is when the XMS XMX came into the picture. So, that memory, right? So, you think about um 4GB memory. The moment you set XMS as 4GB, the whole of the 4GB gets committed into the space, right? So, that means um no other process will be able to allocate

memory. So, 1990s you 4GB memory was pretty big, right? These days it's not. So, um the the point is don't allocate the whole thing. Just reserve XMX, meaning don't commit, just reserve. Just reserve that much space for this process, and don't commit the process into you. That means other processes are still able to make use of your memory through a technique called paging and swapping. Paging and

swapping is a technique by which other processes, if they need, can take memory from your your process, right? But if you do XMX equal to uh sorry, XMS, then that much memory need to be committed upfront. Right? So, that was the difference. But these days, those premises are totally changed. I mean, 4GB is is nothing, isn't it? These days, we are talking about terabytes of heat. Thanks

for asking that question. So, obviously, GC number three GC plays a huge role in Java. In fact, huh, uh when you study Java, one of the first or second chapter in the Java book is GC, garbage collection. It's not that Java invented garbage collection, but Java popularized garbage collection. Through Java, we learn GC. Now, what is so much important about GC? GC comes with various policies, various

strategies, right? Uh and it depends on your requirement. Just like you rightly pointed out in the beginning, what is your requirement? Do you need high um throughput? Do you need low latency? Do you need um memory efficiency? Do you uh faster startup or low memory footprint? There are so many different requirements based on your application's needs. You may be running on an edge device, an IoT device,

or a cloud deployment, or a or a data-centric, or a security-centric, or a IO-centric, CPU-centric working. There are so many different types, right? Federal uh use cases. So, based on that you define your strategy, and based on that, you pick choose the right GC policy. The the exact requirement that I told talked about, I I don't fully remember the metrics between G1, Shenandoah GenCon, and balanced GC.

But each of these attack one specific trade-off. Say, for example, GenCon uh attacking the low latency. Whereas Shenandoah, I believe, is about um pauseless GC. I mean, don't have too many pauses. So, it attacks on the high absolutely based on your policy or so your strategy or your requirement, you need to pick choose the right policy. now, that's a good good analogy, right? You have a pizza

and you need to distribute it to say 10 different kids. Every kids love pizza, but no kids no two kids will love the same flavor. Somebody wants more spicy, somebody wants more cheese, somebody wants more toppings and things like that. So, that is why we have different policies. Now, without performance tuning, you go with the default, definitely you're not going to get the best performance. So, pick

choose the right policy. If you are on a hotspot, choose between the three. If you're on OpenJDK, choose between these two. Of course, you can't sit in OpenJDK and pick choose the hotspot policy. That cross uh tuning is not possible. Now, you see 20 to 50% latency improvement is what we are getting just by tuning the right GC. What would be number two? It breaks. Yep. JIT.

Just-in-time compiler. So, as I said in the beginning, Java starts with interpretation and it looks at the uh methods that are keep coming for execution, which are the hot methods. If you have billions lines of code, not necessarily that every line is executed in every deployments. Every line is executed exactly the same number of types same number of times. So, the ones which come every now and

then is called the hot methods. Within that, there could be hot loops. A loop runs for million times. A loop runs for just once. So, there could be differences. >> the the earlier premise 1990 premise was to go ahead and jit everything. What is the problem with that? Very slow startup, yes. And then, it bloats up the jit code cache, right? For every piece of Java code,

you have that many pieces of byte code, and you have that many pieces of jit compiled code. Waste of time, waste of space. So, what is a modern strategy? Modern strategy is allow it to be interpreted, then profile it, look at which methods are coming for execution with what parameter, what is the loop count, though? Which objects are getting contended? Where the objects are getting allocated? How

long the objects are living? There are so many things that you can profile from a running JVM, and based on that, you make a compilation plan, and then, don't just compile based on this, you make a incremental compilation. You start with a um very low optimization called AOT, ahead of time, then cold, then warm, then hot, then scorching. So, there are five levels of compilation, scorching being

the best or the most optimized variant of the code. And in HotSpot, we call it as tiered compilation. In OpenJDK, it's on by default, but they have graded levels of optimization uh density. So, this technique works best for most of the modern workloads. All right. So, what would be number one? Data structure. Uh-huh. Anything? Anybody else? Okay. Environment tuning. Like what? Eight. Sorry? Eight core. Thread dumps.

Good. It's not a tuning option at all. It's JFR, Java Flight Recorder. It doesn't give you any any improvement. I don't have a number on the right-hand side. It's an observability tool, right? It doesn't help you tune anything, but it it is the starting point. It monitoring your application to profile what kind of characteristic your use case has. Say for example, you say I have a banking

application. But, does that define what is the resource need? What is the concurrency need? What is the optimization need? What is the GC need? No. It just says I'm it's a banking application. There would be several concurrent users. To understand what is the finest level of JVM use cases, JVM tuning options, you need to do the profiling. JFR is a 360° view observability tool inbuilt into the

JDK, right? It gives you metrics such as the most lowest possible metrics possible, the CPU each thread is using how much CPU in the last 1 minute, last 1 second, and last 1 day sort of thing. How much of memory is actually being committed? How much of memory is reserved? How much of memory is paged out and paged in? What What are the methods that got jit

compiled? What is the environment setting? What is the GC pressure looking like? How many threads are running? How many threads are doing concurrent GC? So on and so. There are hundreds of metrics, right? Use these metrics, understand your application's needs, and understand bottlenecks, right? Where are the conditions that is happening? And based on that, apply the flags. There could be hundreds of different tuning flags. By applying

all the flags, do you get the best performance? No. You You apply the right flags for your use case, and you get the best um both both the JVMs use the same flag for starting the uh flight recorder, but in addition to that, HotSpot has something called a sync profiler, which is a very low low overhead profiler, whereas OpenJDK has several inbuilt tools like Java core, which

provides the lightweight uh in-inside view of the running JVM. Heap dump is the dump of your Java heap. System dump is the dump of the whole process. And then there are several tools to process those things, and with that, you you you don't get the best performance, but you get the best insight to apply the performance tuning. So, that's pretty much we have today. Oh, it's not

moving. Okay. So, in summary, what did we learn is first, measure. Don't apply any flags. Don't any optimization technique right from the beginning, because we don't know what the use case is looking like. And then, don't tune randomly, tune from the top level. Like set the GC policy. Before setting the GC policy, you profile and tune something else, and then you get a wrong number. But, the

big ones first, and then drill down to the bottom. And then, we already know that the defaults are so generic, then whereas the workloads could be pretty unique. So, you need to be aware of that. More flag doesn't mean more performance, and the most important thing is the last learning, right? You performance tune your application, you get the best one, you met your SLA, you are happy,

your clients are happy, everybody are happy. But, are we done with that? No. In the next release, after 3 months, let's say you just changed one line of code in your application. Now, the application characteristic could be different, right? I mean, based on what line of code got changed, it could totally change the whole spectrum of characteristics. So, it's an ongoing activity. You need to perform the

same thing again. So, monitoring and performance tuning is part of the software development life cycle. It's not a one-off activity. These are my call for actions. Java is a very proliferating, very welcoming open community, both OpenJDK and OpenJ9. Both are open source projects. There are a lot of options available, starting from the JEPs, Java Enhancement Proposals, through which the major feature come into Java, to small-scale bug

fixes, feature enhancements, documentation, test case writing, and um sort of things typical of every open source project. So, onboard any of these uh project areas, contribute in ways you can, and enhance your skills. Uh open project open source contribution is very very critical in these days because 90% of the code is now running on open source. And having a strong open source profile is absolutely critical for

your career. It doesn't stick with your company. It's it's a public profile. GitHub profile is a public profile, so don't be shy away from that. So, that's my message and call for action. Thank you so much for listening. Really um appreciate that. Have a wonderful day. Thank you. >> [music]