Asynchronous Programming in Spring: Past to Present -Venkat Subramaniam

This talk explores the power of virtual threads in Java and their integration within the Spring framework to enhance asynchronous programming. The speaker explains the concept of non-blocking calls and their significance in improving application scalability while reducing resource consumption. Listeners learn through examples how virtual threads allow a task to yield execution to another while waiting for I/O operations, thus optimizing thread usage. He demonstrates the practical implementation of a Spring Boot application that incorporates these concepts, highlighting how minimal code changes can greatly boost performance and efficiency. The talk emphasizes the balance between increased throughput and the pressing need to manage downstream systems' capacity when adopting these new features.

I want to talk about um uh uh interplay of two different things together, which is pretty amazing in terms of uh how uh virtual threads are very powerful and how Spring is able to make use of it. And and it's pretty remarkable in terms of many rarely in in life you get to see a concept that you can use with almost no effort. That is one of

the beautiful things about concepts like that and virtual thread is one of them. So, I want hopefully illustrate that in this example in here. So, before we go that route, let's first understand what does it mean by asynchronous programming? So, essentially, what this means is, so when you talk about asynchronous, uh means non-blocking calls. So, that's basically what it means. It's a non-blocking call. But, here's the

problem. If you're making a call and that needs to return a result back to you, you have to block and wait for the result. For example, if I say I would like a cup of coffee, please. I cannot say I want a cup of coffee, please and just walk away. I'm not going to get my coffee. I have to wait for the coffee if I want to

get the coffee, right? So, in a sense, you can say every single task that you can say that requires a result has to block uh for the response. So, essentially, you cannot just make a call and run away. You have to block and wait. So, then the question is, wait, if asynchronous is non-blocking, but if we have to make a call and get a response, we have

to block, then what does it mean by asynchronous or non-blocking? So, essentially, asynchronous means the task will block, but the thread executing the task, uh you know, the task will not. So, essentially, non-blocking is really about the thread of execution, not the task itself. So, you want the task to wait, but you don't want the thread to be waiting for it. So, the question is, how can

we Well, first of all, why should we really use asynchronous The reason really is, you can say, this is this has an uh impact on the architecture. So, essentially, a a synchronous programming means essentially means higher scalability for uh fewer resources. So, think about this for a minute. What do threads in Java do? Threads in Java, when you call a task, the task is assigned to run

on a thread. So, the thread is executing the task, and the task says, wait, I need to perform an IO, or I have to access the database. I got to make a remote call. And the minute the task is making a call, the task blocks, and the thread blocks as well. So, when the thread is blocked, what do you do? You want to perform more actions. If

you want to perform more actions, you end up creating more threads. Now, the more threads you create, the more tasks can run, but there's a limitation on the number of threads you can create on a system. So, if you max out the number of threads you can create on a system, your scalability is limited. So, then what do you do? Oh my gosh, you can only run,

let's say, a thousand requests on this machine, but you need to run 10,000 requests. So, you're like, wait, I can only run a thousand requests concurrently on this, but I need 10,000 requests. So, why don't we go add more instances in this cluster? So, every single machine in the cluster, instance in the cluster, can run a thousand requests. Now, I can have 10,000 requests running by having

these 10 different instances. Now, what just happened? You're running not one instance. In order to scale, you have to run multiple instances. This is great news, right? For the cloud providers, because they can keep billing your company more and more, and they are very happy. So, I would say this the problem here is three E's. The first E is that the economics. You have to spend more

money sending to these companies. You're going to lose out on that. The second thing is environment. You are using more machines, and they have to be in these centers running more processors. So, you have the problem with the environment as well. So, you're losing on spending more money. You're burning more of the, uh, you know, fuel that is needed, electricity maybe you're consuming. And of course, it's

going to mess up the environment as well. That becomes a problem. But the third E, if I have to write a program that runs on one but I have to write a program that runs on multiple machines, which is more difficult? The multiple machines takes more effort. That's the third E, more effort. So, you spend more money, you put more effort, and you mess up the environment,

all because why? Because threads don't do a good job. Threads get stuck when you call a task. So, what if we can make the threads not get stuck? So, what if a thread can say, "Hey, if I have an IO, I will not hold the thread hostage, and it can go off and do other things." Let's understand this with an analogy. Let's say you go to a

nice restaurant, and when you walk out of the restaurant, the waiter comes to you and says, "What would you like to drink?" And you say, "Hmm, I'm not sure what I want to drink. What do you recommend?" And what does the waiter do? The waiter says, "Oh, here are some drinks, and here are the names of it. Why don't you think about it? I'll be right back."

And immediately leaves to attend to other things, right? Now, what if this waiter instead says, "Oh, you're not sure what to drink? Let me help you." And pulls a chair, and sits next to you and says, "How's your day going? Oh, by the way, I like your shirt." Now, what's going to happen? If the waiter sits with you for a long time, the other customers are going

to get angry. They don't have anybody to serve them. Well, the restaurant will have to hire more waiters to wait these other tables. So, here is the irony. In a good restaurant, what does the waiter not do? The irony is waiters don't actually wait. They make you wait, right? And that that way they're being efficient in switching between multiple tables and serving these tables. So, this is

where the non-blocking really comes in. But, how do we benefit from this non-blocking? Let's take a look at this with a little example, and drive towards it. So, let's first of all, look at creating a little Spring Boot application the old way, and see how we it can work. So, here I have a little application, nothing really going on right now, and it's got a little demo

application. I just use a Spring Initializr, and I brought this demo app. That's all I have. Now, what I'm going to do here is that I'm going to go over to this and say, "I want to create a controller." Now, what am I going to do? This is a very simple program just to illustrate this. I'm going to give a sentence, a bunch of words, and I'm

going to ask it to count the number of spelling errors in that word. Now, in order to make this a little bit more interesting and realistic, when I give the sentence, it's going to split that into words, and for every word, it's going to go to a remote server and check for the spelling and come back. And as you can see, that's an I/O operation, it's going

to take time and get the response back, and as a result, it has to perform this operation, which is going to be time-consuming. So, here is a class I'm going to create, and the class I'm going to call this one is a controller called Let's call it as a spell uh check controller, right? That's what I'm going to call it. So, here's my spell check controller. And

and what does the spell check controller do? First of all, it's a REST controller, as you would imagine, because I'm going to make a RESTful call to this, and I'm going to perform a request mapper here, and the request mapper says, "I'm going to receive requests into a spell check." So, this is going to be receiving a request into a spell check and say, "Hey, when you

make a call, I'm going to perform this operation into this, you know, URI, which is the spell check." Okay, great. Now, I want to create a function. We'll call it as a public int, and we'll call it as a count incorrect, let's say, uh you know, words. And this is going to take as an argument, you know, maybe we'll call this one as a sentence that I'm

going to pass to this as an argument, right? So, this is going to be a string and that's going to be a sentence, but this one is going to be a request body we're going to get it. So, we'll say sentence and this is going to be as you can imagine, this is a request body that I'm receiving as a sentence. So, what is this going to

return in this call? Let's take baby steps go and going towards it, right? So, I'm going to just return a zero. There's no spelling errors. Let's assume that for a minute. So, this is going to be after all, this request that I'm going to save into a check, but it's a post call I'm going to be receiving. So, I'll say post mapping and it's going to come

into a check as a URL. That's where That's where I'm going to receive the request as well. So, let's start with this baby steps first of all and see how we can proceed with So, that's our code. I'm going to start with this particular code as you can imagine. So, I have a little script here called and run server and a service and all that is doing

is it's calling Maven and saying run the Spring Boot application. That's all it's doing. I just put that in a script to make my life a little easier. Okay, that's great, but I want to make a call to this particular request. So, I have a run.sh. It is performing a curl and says this is a test. You can see there are two spelling errors in there. So,

I'm going to run that right now in here as you can see and that's saying zero because that's what I implemented. So, that part is working great, but I wanted you to tell the truth. It isn't right now behaving like AI even though we're not using AI, right? But I wanted to tell the truth. So, I want to be able to check the spelling and get the

response. How am I going to do that? Well, in order to do that, what we're going to do here is to use a service. So, I am going to say this is going to be a spell, we'll call it as a spell check and the spell check is going to say count. We'll call this one as incorrect correct words and we'll pass the sentence to that particular

incorrect word. So, I'm just kicking the can down the street and calling this other function into the spell check. Now, the spell check itself is going to be a class that I'm going to be creating, and this class spell check, what does that do? It contains, as you can imagine, this particular static method that I want to implement, and it says, "Given a sentence, I'm going to

return it." Well, this one requires just a little bit more effort. Remember I said I have a web service I want to talk to and get the response, so we are going to go work on it and get the response. But, before that, I I'm going to make myself a little bit easier here. So, let's go ahead and say here's a private, and we'll call it as

a boolean uh is spelling uh spelling, let's say uh correct, and this function is going to take a word as an argument. So, what does it do with the word? I'm going to say client is equal to let this be HTTP HTTP client. new HTTP client. So, I'm just creating a client request object, and I'm going to say request is equal to, and I want to create

a request on that object. So, HTTP request, and then we will say in this case uh new HTTP uh sorry, this is going to be a request, so it's a new builder, and on that one we'll make a request. This is going to be URI, and and where is this URI? So, URI. uh in this case, we're going to say this is this is going to be

making a request to our create function. So, URI.create, and let's provide a HTTP uh you know, a colon, and this is going to be a URL www. dot, let's say Agile Developer, I'll say uh dot com uh slash spell check, and then question check, and then I'm going to provide a equals to percent s.formatted, and we'll provide the word to it. So, this is going to make

a request to that particular URL that is going to bring us the uh response. This is going to be URI, rather. And then, of course, we're passing the URI to get the response from it. So, once I make the request to it, I'm going to call a build function on it that gives us the request. It's time to make the call to this request. So, response, we'll

say, is equal to bring in the request right now. And we are going to say client a client in this case uh so, a request is going to be Let's go ahead and say request Oops, let's try this again. A request, and we're going to make a call to that particular client. So, client.send and do a request on it. So, once we make the request, I'm going

to say HTTP. This is going to be a response.bodyHandler. So, we'll say response, and this is going to be the body handlers, and this is going to be really a string object. So, we'll say off string, and then we'll say expected results are a string type. Once I get the response, I'm just going to return it, right? So, we're saying here return response. And this is going

to be on the body of the response, and then we can simply say a dot in this case equals, and if it is equal to a true, then that's a boolean true, otherwise it's a false. We're almost there, except the sender is saying you need to handle the exception, but I'm going to be a little lazy right now, so I'll put a try block right here. And

at the end of this we'll close the try block, and we'll simply say catch, and I'm going to catch any exception that's thrown my way, and I'm going to simply say throw new runtime exception, and we'll just bubble this up to the caller, and and be done with it for now. So, that's our spell check is spelling correct. Assuming I wrote that correctly, what do we want

to do to count the number of spellings in this uh incorrect spellings in this case? So, I can say this is going to return an integer and out of this this is a stream, uh if you will. And the stream of and we'll take the sentence and split it across a blank uh word, so that gives us each of the words in the collection. So, given a

stream of that, I'm going to do a filter and here is a word that's being given to me and I can say is spelling correct, right? That's what I'm interested in. We can simply turn around and say this is spelling correct and ask for the Excuse me, on that one. So, once we ask for this, oh this is going to be the uh spell check, right? And

then it's a static method we're calling on it and then we'll do a count on it and that gives us the integer value and we are returning it. So, is spelling correct is going to take a word, which is what we are passing right here to this particular word as well. Uh that's Let's make that a static method also because after all, we are not using any

state and we can simply use that information in here. So, let's see what's going on here. So, we got the filter on the stream and so, once we get the stream, we're going to get the integer out of it and return it. Well, this is supposed to tell us how many request how many words are of wrong spelling, let's see if that actually works. So, I can

go back and run the service right now and let's go ahead and make this call and it is telling us in this case, you can see a little slower because it's going to make the call and get the response back from the call and and eventually, once it finishes, it should tell us how many requests it was able to perform and it's two words are incorrect from

the whole thing. But, I want to take this to a little bit more fun level here. So, let's go over to the controller and in the controller, let's go ahead and say output and this is going to be a request that's coming in. This is purely for our purpose, right? And the output I'm going to say a thread.current thread and this is going to be in the

controller, right? So we'll say controller, so we'll say request and you know, this is going to be running under so request running in and then the thread of execution. So if I go back here to illustrate this particular thing, you can see what's going to happen. So we run the server but I'm going to go ahead and start multiple of these calls in here. So this is

going to fire up many many requests. You can see all those requests are running concurrently and making a call to execute it as you can see. And you can see the result is being printed from all those calls. However, I'm going to make this a little bit more interesting to make this clearer as to what it's doing. Now, you can run this at at full fledged and

try to illustrate it, but the problem is it's going to be getting very confusing when you see all these threads. It becomes hard to understand it. So we'll take this a bit unrealistic illustrate this problem and so what I'm going to do is the following. I'm going to go to the server right now and in the server I am going to go to this file which is

the resources application.properties. So in here in this properties file, I want to tell it that I want to you know, control it to a certain number of threads. So this is going to be server.tomcat in this case dot and I can say how many threads I want to use. So threads is equal to and a very rather unrealistic value of two. We'll say dot max equal to

two. So this is saying only provide two threads at any given time, right? So what's going to happen if you have two threads? If If want and again keep in mind, right? It doesn't matter if it's two or 2,000. You it's depends on the number of requests that's coming in. So, if there are two threads, what's going to happen? Let's say you make a request, this thread

is going to go, you know, address that. He makes a request, the second thread is busy, you make a request, what's going to happen? Yeah, it's going to queue up, right? It's got to queue up and wait. So, it's going to queue up and wait, and when this finishes the request, the other one can execute. So, what that means is if you have only two requests, they

just finish and you're done. But if you have 10 requests, eight of them will wait and two of them are getting executed. So, there's going to be a time waiting in the queue, plus the time for execution of the call itself. Given this code, you cannot minimize the time of execution. It's got to take the request, send the request to the remote server, get the response, give

it to you. But if there are not enough threads, you're going to wait in the queue. This is like going to the doctor's office, right? The doctor sees you for 5 minutes, but you wait for an hour because you have to wait to get in. That's kind of very boring, right? But that's kind of what's going to happen in this case. So, if I were to go

back and run this code right now, so you can see the server only supports two threads as we talked about. So, I fire up the request. Here Here you go. There are two requests while the others are waiting, as you can see. When two of them finish, right there, you can see two more requests are being handled. So, it's waiting on those two to finish. When those

two finish, you can see one finished, the second finished. So, you see the point, right? So, it's just progressively implementing. The others are just waiting. It's like, "My gosh, I'm just waiting here doing nothing. I don't have time to execute because it's all busy." So, what is the moral of the story here? This is like the waiter in the restaurant who's sitting uh with the customer in

the table while you're waiting outside. I'm not even getting a a seat, but there are so many empty tables, but there's nobody to take me from here to there and you know, make me sit, and it's frustrating. Okay. So, I want to make this faster. What can I do to make this faster is the question. Well, let's step back for a minute to understand what we can

do. So, this This was a Spring Boot application that you were programming before. And you know that in Spring Boot in the controller, you can do a couple of different things. So, rather than this code, you can write the code a little differently. So, what am I going to do? I'm going to say a public, but not a int, but a Mono integer. Now, in Spring, you

had two things that were available to you. What are those? The two things that were available to you was Mono and Flux. What's the difference between Mono and Flux? Mono is you can make a call and it will immediately return a future to you. And your request is no longer going to hold the thread. But the Mono is a handle that you're holding onto, and it will

push the response to the Mono to you. So, this gives you a non-blocking API. So, if you have one request, I'm sorry, if you have one response, a Mono is a way to do make it asynchronous. So, if the response type is A, you make it a Mono A. If the response is int, you make a Mono integer, right? Now, what is a Flux? Instead of a

Mono, you can use a Flux. Both Mono and Flux are asynchronous, but a Mono is a single response. A Flux is a series of response. So, if you have multiple data to send, you can return a Flux, and you can do back pressure and keep pushing multiple data. So, in this case, we only have one data, right? Because you're asking how many letters how many words are

incorrect, that's just one response. So, it's a mono of integer. So, in this case, I'm saying I'm receiving a mono of integer, as you can imagine in this case. And and the function name is exactly the same as we had before, no different at all. So, I'm going to copy this over uh to here and I'm going to uncomment it, but then I'm going to change this

to a mono of integer, right? Because everything else is the same up to this point. But what am I returning from here? I'm going to return a mono. And you can say just or empty. This is to say I'm asynchronous, but it will return the response asynchronously. But if you already know what to return, you can just return it right here. But in this case, I don't

know what to return. So, I'm going to say from future. And and what am I going to do within this from future? In the from future, I say go ahead and send a supplier and do a spell check. Count words you know, count in words. And I'm going to then call that function a synchronously and send the response back to the caller. So, what does this do?

This is going to take the sentence and send the response back to the caller, right? But it's complaining to me why? Because it really requires a supplier that turns a turns a completable future. So, to uh handle this, I'm going to change it to async right now, like so. So, async is a function I'm going to implement. So, then the question is, what does this async function

do? The async function returns a completable future. Given a sentence, it's returning a completable future. So, what does a completable future actually do for us? A completable future allows us to uh really return a response asynchronously without blocking. So, I'm going to say return completable future dot and you can say supply async. And to the supply async, I'm going to then make the call here and simply

I'm going to say, "Hey, let's go ahead and return an integer on the stream dot Let's say off. Here is the sentence dot split on empty words. So, that gives us the stream of data as you can see. And then I say a dot and this is going to be a filter after all. And given a word, I'm going to simply call uh you know, not is

spelling correct. So, we can say it Sorry, I did something wrong here. I just realized. This has to be a word uh and this has quite got to be a not uh of is spelling correct. It's just that I had two words uh out of the four incorrect, so that gave us the correct words, but I want the incorrect words. So, here again, we'll say is uh

is spelling correct and we'll pass the word to it as well and then return the response. And then finally dot count and we will simply return the response of that count in here uh to the caller. Okay, so this is an asynchronous function that we wrote. But notice the difference. This was a nice little synchronous function we wrote in the top. Now, we are using a completable

future to make it asynchronous. Now, you may look at this and that's not too bad, but the more complex things you do, the more complex this is going to become as well as you implement it. Okay, but what is the benefit so far? Let's go back here and run the server right now and you can see the server has started up, but I'm going to go back

here and do the start and you can see all those requests are running exactly at the same time. Remember, only have two threads available. But, what happened? The threads here, as you can see, is receiving a request, and the minute the thread receives a request, it puts it on a on a background thread, which is asynchronous. That's your That is your completable future, and immediately returns it.

So, this is why when you see it, notice it is 44 and 44 and 44 and 44, and similarly 45 45. So, you can see all those really indicate to you that it was wicked fast in taking the next response. So, what is this doing? This is like you were you're in the doctor's office, and the and the, you know, nurse comes to you and says, "Hey,

you." And takes you in, hands you off to another person, and comes and gets the next person. So, you are in the not in the queue anymore, you're being serviced. But, where is it being serviced? These are all background threads that are running, so that's non-blocking. So, it is not blocking the request thread. So, this is basically making it asynchronous, so the request threads are saying, "I

get a request." This is how it worked in the past, right? I get a request, I go do the work, I perform the IO, and when the IO is happening, what does this thread do? It is just sitting there and just watching it, right? Doing nothing. And it's like, "Oh, I'm just waiting here." It's making a request. And then the other thread comes in. That makes a

request call. And what do these two threads do? They just sit there by the coffee machine. "Hey, how's your guy doing? Oh, he's making a request over there to IO. Well, I'm waiting on this one." And they just sit there and talk politics and do nothing, right? Whereas, in here, the request comes in, the request thread says, "Oh, thanks for coming. Hey, you, run this." And runs

off to get the next one, and you run it. So, it's not blocking, it's just handing over to another thread, and then when the response comes in, it can take the response and return back through the Well, this is the way we did this in the past where we can include asynchronous programming. But, let's step back and talk about this If you look at the code, did

we have to make significant code change, or did we not have to do any code change? What did you notice? That was a lot of code change, isn't it? Now, imagine you're an architect. And as an architect, you are solving problem based on what and you're not still sure about what the expectations are going to be. And one of the principles we like to follow as a

good architect is this principle called last responsible moment. What is last responsible moment? You don't do things until you no longer can avoid doing. Well, a last responsible moment means, "Oh my gosh, I realize I have to really handle this now. I cannot do it later, and I have to do it now. I didn't have to do it earlier, so I'm glad I didn't do earlier." When

is a last responsible moment really useful? It's really useful if you're able to accommodate that fairly easily. But imagine you have four, you know, 25 controllers. Let's imagine the 25 controllers are calling into multiple functions in the background. And as an architect, you said, "Don't worry about it, folks. Write the simplest code possible." And they've written everything, and two weeks before going to production, somebody comes to

you and says, "Uh we have we have a problem. This is not scaling. So, we need to really make it asynchronous." What would you do as an architect if you get to know that 2 weeks before? That's right, find another job. Right? Because you are not going to sit there and face your team because your programmer came to you and said, "Are you kidding me?" Because we

have to make change to all this code. You're like, "Why?" If you're architect who would write code, you know what it is. But if in your company the word architect means I don't write code, you're like, "Why do you have to change all that code?" And then they're rolling their eyes. This idiot doesn't write code, he doesn't know what he is doing, right? And now they have

to make all this code change, and that's expensive. And they have to understand if this is working, got to test it, you got to do what errors you're introducing. So, what do you do as an architect now in the next project? Remember, you quit here and went to the other side. And as soon as you join that other company, you're saying, "We are going to make it

asynchronous." And they're like, "We don't even know what we're doing." But I have a scar from my previous job. We had to make it much late, and that cost us, so we are going to do it now. So, what does your team do? Here we go, turn everything into asynchronous. And is that a fun way to write code? When you don't need it, you are still writing

it. And that is the consequence. So, you are not able to postpone your decision when your change is enormous. With me so far? All right, let's just pretend we didn't do this for a minute. Let's go back to this code. Let's just get rid of it, and let's go back to this code. Let's quietly get rid of this, right? So, we don't have that completable future code

we wrote. This is the original synchronous code we wrote. So, I go now to the controller and we'll pretend we not did not write this as well and we'll simply go ahead and uncomment this. That's our original code as well that we had. So, now let's go back here and we still have two requests that's in here. That's all we have. And let's go back here, run

the server now, and let's at the same time, let's go ahead and fire up this request as well. And you can see it's made those requests, but sadly it's got done very slow, right? It's extremely slow and it's doing two requests at a time because we restricted that. You can see when it finishes one of those requests the other one is able to handle, it is waiting

in the queue and you can see that right away. It's extremely slow. So, one of your team members comes to you and says, "We got a problem, architect. The The code is not scaling. It is too slow, right?" So, right there. So, but you are an informed architect, right? You're not a PowerPoint architect. You're a hands-on architect and you're like, "Relax, dude. Don't worry about it. We

can take care of it." And they're like, "What would you do?" And you say, "Aha, that's why we moved over to Java 21 or later." So, what happened in Java 21? In Java 21 they introduced this concept of virtual thread. So, what does a virtual thread actually do? So, a virtual thread uh it it essentially says, "I want to be non-blocking." So, imagine this for a When

you have a task and you have a thread executing in a task when the thread when the task blocks the thread is blocking. And when the thread blocks when the task is blocking that is a poor utilization of the thread. That leads to creating more threads and create scalability issue. So, let's rethink about it. But before we go further with that, let's think of a fundamental way

in which we solve problems in computer science. So, if if I were to give an award to somebody, hands down, I would give award to one That That person said something absolutely profound that fundamentally affects what we do and how we do it every single day. So, who's this person? This is a person named um uh he he was a British computer scientist and he was he

passed away years ago and his statement was that in computer science, you can solve almost any problem by introducing one more level of indirection. Now, just think about that. How many times we have benefited from that, right? So, in computer science, we can solve almost indirection. We have done this in many places, right? In C programming, what have we done? We have introduced uh uh pointers. So,

rather than calling a function, you route the call to the function using a pointer. By varying the pointer to point to a different function, you end up calling a different function. In OOP, we use polymorphism. Polymorphism is a level of indirection as well. So, so this gives us the ability for uh being uh you know, being able to dynamically vary the method calls. So, it turns out

that virtual uh threads are a form of indirection. So, how is that? So, old way of doing things, a thread uh runs the task. But, in the new way, the thread runs the uh thread, which runs the task. So, essentially, you have a virtual thread, and the virtual thread, as the name alludes to, is the one that runs the task, but the thread is running the task

via the virtual thread. So, essentially, you could look at it a little bit differently, right? You can say, uh in the old way, a task runs on a thread. Whereas, in the new way, you can say, task runs on a virtual thread, which gets uh you could say, mounted on a do? When there is work, you can say, when there is work, a virtual thread uh gets

mounted on a thread. On the other hand, when there is no work, in this case, a virtual thread, uh you can say, gets unmounted uh from a uh a thread. So, essentially, it is not holding the thread hostage. So, it says, "I have work to do. I'm going to mount on the thread, and it's going to execute it. All right, I'm going to block. I don't have

work to do. I'm going to get unmounted, so the thread can do go out go do other Now, why don't we take a look at this in action, and get a feel for it, and then we'll come back and see how we can make use of this. So, what I'm going to do is to understand this with a little example. So, let's say, we want to implement

a little task. So, we'll say, a public, let's say static void do work, let's say index over here. And in the index, I'm going to say uh a thread. Co- First of all, let's let's output. And in this case, I'm going to output the index uh plus and we will say uh entering. And after we say entering, let's go ahead and say in here uh a plus.

Let's go ahead and say plus uh thread. current thread, right? So, we are displaying the thread that's executing it. Then, let's go ahead and say exiting. But between those two, we'll say a try uh thread. sleep uh let's say about 2 seconds and we're going to sleep for 2 seconds right there. So, let's stare at this function for just a minute. It's not doing any real work.

So, you call do work, it prints the thread, it sleeps for 2 seconds, and prints the thread as it is exiting. So, let's go back here and say uh over here uh for let's say int I int I equal to zero, let's say I less than 10 I plus plus. And we are going to say uh a new thread. We'll say new thread and we're going to

call this sample do work uh and we're going to pass uh index to it. So, let's go ahead and say this is going to be the do work method. And to it, we pass a index right there. And what is the index? We can say index is equal to I that we want to pass to it. And then once we do, we'll do a start to start

the thread of execution. So, I'm just starting these new threads, right? So, once we start them all, pardon me, we will then say over here uh thread. Let's let's sleep. let's give it about 10 seconds of sleep. Uh and and then once we do, we will then, you know, display the result of it and say done. So, this is going to be at the very end of

this, we'll say done, right? So, nothing really exciting. So, this is starting 10 threads, each of the thread is calling the do work, the do work is sleeping and returning the result back to us. So, if I go back here and execute the code, so we'll say run Java in this case and execute it. You can see it's waiting to execute and finish it, and then finally

it's going to end it in the end. But to make it a bit easier and and more fun, let's go ahead and fire this up and say, I want to take the output and sort it and and visualize it. So, let's run this and take a look at what it's going to produce. Now, remember we started 10 10 of these. But when you started a thread 10

of these, there are 10 threads executing, but look at the result of this call right now. So, what happened? One is entering using thread one and exiting using thread one. What about four? It's enter using thread four, exiting using thread four. What about seven? Entering through seven, exiting through seven. So, what did we learn from this? When a thread gets allocated on the task and it says,

I will stay with you until it is fully done. But what is much worse? What is much worse is when the code is running, you can see When the code is running, you can see the thread calls sleep. What did the thread do when the task sla- says, I want to sleep for 10 seconds? The thread says, that's a great idea. And then the thread sleeps with

the who is a who is a a parent in this room? You got children? Others are very young people, don't have children? Niece, nephews? Come on, you got nieces and nephews, right? Cousins? Yeah. When a little child in the house says, "I want to sleep." what do you do? Do you say, "That's great, baby." and you sleep next to the child? Do you do And if you

do, nothing ever gets done in the house, right? So, what do you do as a responsible parent? You're like, "My goodness, finally the little devil is going to sleep. I'll go get some work done, right?" That's what you do, right? So, when the child sleeps, you rush to get your actual work What does this thread do? It's like a very irresponsible parent. Right? This is like me.

That's why my wife never trust me with anything responsibility. Anything useful, she takes care of it. Anything that's not important, she's like, "You can do it, at least this, right?" So, if you leave the kids with me, I'm like having a good nap. The kid is awake and playing, but I'm sleeping, right? So, that's what this thread is saying. This thread is like, "Wow, this is great.

I don't have to do work. I'm going to sleep." Now, imagine you have a thousand threads and they all make remote calls, what's going to happen to those threads? They're just going to sit there and do nothing. Right? How does that feel to have a thread that does nothing? You know the feeling, right? This is like some people at work. They come to work, but they do

nothing until they go home in the evening, right? That's kind of what the threads do, nothing. And that's a poor utilization, right? So, what can we do to deal with that? So, in this case, I'm taking this code and I'm changing just this one line. And I'm saying, "The dot start right here virtual thread. And I'm starting a virtual thread rather than a regular thread as you

can see. That's all I did. I changed from a regular thread to a virtual thread. Now, once I change it from a thread to a virtual thread, what happens? Well, remember your task is scheduled on a Your virtual thread gets scheduled on the real thread. It is executing. You call the sleep. The minute you call the sleep the task says, "Oh, wait, wait, I want to sleep."

And the virtual thread says, "No I can sleep here with you, but let's let the thread go." So, it unmounts from the real thread. And the minute it unmounts from the real thread, the real thread is away to do other work and the virtual thread is just waiting for the task to finish. When the task is done the virtual thread says, "Oh, I need to run. Excuse

me, can you please give me a thread to execute?" And any thread in the pool is taken and it it gets mounted. Now, you may say, "Oh, but wait a minute. If the threads were held hostage we ran into trouble did we just not shift the problem from the threads to virtual threads?" Well, remember threads are considered to be lightweight. Yeah, they are lightweight, but they do

occupy significant memory. Virtual threads are not lightweight. They are called super lightweight. They take very little memory. And because they take very little memory, you can have a lot of them. And how many? We'll see that in just a few minutes. But after we execute this, let's take a look at it just to see how that works. So, I go back to the code right now, and

I'm going to execute the code right there this time. Remember, it's running on virtual threads now, rather than on the threads. So, it's going to execute, it's going to sleep, the task has to sleep. So, that's going to take time as it did before, but look at what just happened When you look at one, one is executing on 28, as you can see. And it is 28.

Why is it 28 for one? Because that's your virtual thread. However, your virtual thread was scheduled to run on thread two before the sleep. And the minute you hit the sleep, what did the virtual thread do? The virtual thread says, "Task, you want to sleep? I get it. Hang on a second. Hey, thread, thank you so much. You can leave now. I'm going to just wait here

doing nothing." And the thread says, "Oh, thank you. Uh good working with you. See you later, right?" And the thread is gone. And the virtual thread is just waiting for the sleep. When the sleep is over, the virtual thread says, "Hey, I need to run. Could you please give me a thread to execute? I want to mount." And the JVM says, "Sure enough, here is thread five

for you." You can see the mounting and unmounting, mounting and unmounting. Let's take one more. Here is four. It went from five to two, as you can see. Here is six. It went from seven to one, as you can see here. it doesn't mean it's got to be a different thread. It can be. Look at what happened here. It mounted on 11, and it unmounted, it's at

thread 11. Thanks for your service. I'm going to wait here, you can leave. What did thread 11 do? Bye. See you And it goes away. And there was a call. Hey, we need to get mounted. Can you Can you do that? And it comes back and says, "Oh, you again?" Right? This is just purely coincidental. This is kind of silly, but it's kind of also funny when

it happens. I travel a lot. So, there are times when I'll be traveling international, and because of the schedule could schedule sometimes, I would literally go home for a night, and I would turn around and travel. Uh occasionally this happened, maybe once a year. So, you know how in in flights there is a flight attendant who stands by the door, and as you're walking in, they smile

and they greet you, right? Well, you're in a flight for about 8 to 10 hours, so they see you for a long time. So, I arrive, I go home in the for the night. The next day I'm coming, and purely coincidental, the same flight attendant is returning back on back on the next flight. Then this flight attendant is standing there. Hello there. Hello there. And they'll say,

"Hello there." And they'll take a do a double take. It's like, "You." I'm like, "Yeah, it's me." And they'll say, "This is weird. Didn't you just travel yesterday?" I'm like, "Yeah, let's not talk more about this." That's exactly what that happened here, right? It's the same thread was reassigned to it. That's purely coincidental. Uh and that can happen, too. But here is the deal. What's the benefit

of this? The benefit of this is, let's go back to this code and take a look at it. Notice I say, "Max is equal to I'm going to say 1,000, and I'm going to change this here to uh a max, right? And then I'm going to run this as threads, as you can see. So, when I run this as threads, what's going to happen? I'm creating 1,000

threads, as you can see. So, I'm going to just run this code right there. It's created 1,000 threads. It's running the code and it's going to finish it. No big deal. But, what if I said I want this to be 15,000 15,000 threads, right? So, I run the code and it's starting those 15,000 threads. Did you notice that? It failed right there. It gave an error out

of memory error. It couldn't handle 15,000 threads, as you can see, right? Now, let's go back to the exact same code and I'm going to change this code from a to a virtual thread. That's all I did. So, let's go back and run the code right now. Notice it's like no problem, but can you give me something more interesting, right? So, just to make it easier on

ourselves, I'm going to remove these uh system out so we can just run this and get a feedback. So, if I run this right now, you can see it is running and it's waiting on the done. Not a problem, right? It's going to just go run through. Let's go back to this for a minute and let's go ahead and say, right here we can say output. Let's

say Let's say here started, right? And I'm going to say started and display that it's been started. So, if I go back to once we start it, if I go back to regular threads just to illustrate the point, I'm going to use a regular threads. So, if I run the code this time, notice it's starting all those threads. It failed right there. Out of memory error, right?

But, if I go back here and instead of the regular thread, I'm going to use virtual threads right now. So, now that I use virtual thread, you can see in this case with the virtual thread, when I run this code, it makes no problem at all. It says started. Not an issue. You're like, really? Let me go back here and say, not a 15,000, but a 50,000

threads. So, if I run 50,000 threads, it's like no problem. Wow, it doesn't have any problem with it. What about 100,000 threads? So, I'm running 100,000 threads right there, right? So, we'll execute this with a 100,000 threads. So, here we go. And it's like, sure, no problem, started. How do you feel? You feel a little tense? What if we change this to million threads? How does that

feel? You're on the edge of the seat, aren't you? You're like, oh my gosh, are you serious? A million threads, right? A million virtual threads. And it's like sure, no problem. Can you give me something more interesting? Because it's super lightweight, it has no problem running it. So, we talked about what do virtual threads do, and we saw that example. Now, typically, you wouldn't be creating thread

after thread after thread. If you're writing pure Java code, you can go to executor services, and in executor services, there is a special function called uh you can say when you start executor service, give me a new virtual thread per task. So, there's a new virtual thread per task executor. So, every time you schedule a task, it'll pop up a new virtual thread for you to execute.

What does a virtual thread do? It It mounts on a thread, executes it, when it gets blocked, it unmounts. So, sleep will unmount. IO call, unmount. Call a web service, unmount. Perform a database call, unmount. So, anything that doesn't use a CPU, and you're performing an IO, it'll unmount. And when it unmounts, the thread is ready to go do other work. And when it's ready, a thread

is given, it mounts and executes it. You say, "All right, but in the code we saw, we did not have to make a lot of code change, right? Only thing we had to change, the other function was exactly the same. The only change we made was instead of thread, we used virtual thread. Similarly, if you're using executor service, if you're using new fixed thread pool, you would

say new So, a little bit of code change. That's fine. But Spring, as awesome as it is, decided not to even ask you to do any code change at all. So, let's see how this is going to work. So, here is the code we saw in Spring a few minutes ago. Notice, is spelling correct is making the call to the service. Here is the count incorrect words,

and that's making a call to that service and returning the result. this is not fiddling with completable It is not doing anything complex. It is just having a good time performing a synchronous call as it is, right? And to emphasize this, I'm going to close that file. That's gone, right? That file I've closed it. Here is the controller. Notice the controller is returning an int, not a

mono. So, this is also a simple code, but you wrote it as a synchronous call. That's all we did. So, now I'm going to close that also to illustrate I'm not changing any source code at all. So, let's go back here, run that server right now, and the server is running as you can see. I make the calls over here, and you can see that's really slow

because it performs the two requests, and it's waiting, and all the other requests are in the queue just waiting. It's like, when is this going to get done? And then it gets their turn, and you can see that two at a time are being triggered as they finish execution. Now, your team comes to you and says, "You know, our Spring Boot application, it's it's not scaling well

because the threads are being occupied, our request is taking time. What do we do?" And you say, "Don't panic. No problem. We can make this change." So, what are you going to do to make this better? So, what we're going to do is to go over application.properties, and obviously, right, you wouldn't set the thread to two. That was just a contrived example. But, what we're going to

do instead in here is to say, "Hey, Spring, do me a favor." So, this is spring {dot} right in this case, uh threads {dot} virtual So, virtual {dot} enable. So, enabled is equal to true. So, all I did is to say spring {dot} threads {dot} virtual {dot} enabled is equal to true. That just that one property we modified, right? Once we modified this, we go back and

start the server this time, and the server is running. We fire up all this request, and boom, notice where those requests are all running. They're all running under the virtual thread, and they got mounted into worker threads behind the scenes and so they all get fired up and they are executing So even though your request threads are limited, you said don't run them in a normal thread,

run them in virtual thread. So what's the benefit? Spring maintains a pool of threads in the background and you're it gives you virtual threads instead of threads. So the virtual threads are like, "Come on in, let me help you." and starts executing code. You're doing work it's doing and the minute you perform an IO, it unmounts from the real thread and it waits. And then the real

thread of course can go do other things and then this can finish the job and it can return it. So you can see this is going to give us a better throughput, a better scale, but what did we have to do to fix it? We had to write that one line of code. being in this room, you should promise me one thing. You know it can't didn't

take a lot of But promise to me that you will never tell this to the boss. Right? This is just between us, a secret. So your boss says we have to scale it. Don't smile. Make a grim face. You say, "Scale it? You know it's going to take me 3 weeks." And it only works by the beach. And so take your trip, hm? On the way, add

that line and you know what to do for the next 3 weeks. And tell your boss when you come back, "It scales now, but it requires maintenance every 6 months." And that's just a secret between us, right? And that way you can make it scalable without actually doing a lot of So, what this means to me is that look at the nice little food circle we have

come through. To me, Java 8 made the programmers productive because we were able to use functional programming and we were able to write more elegant code. So, Java 8 made programmers productive. Java 21 made programs productive because you're able to scale the application but with absolutely minimum effort. And in the in the case of regular Java code, you just had to decide a different, you know, threading

model from thread to virtual thread. Your executor service can be modified. In the case of Spring, a little configuration had to change and it was able to provide that as you can see. So, so the beauty of this is you're able to scale your application without much effort. you this is great but remember, for everything we do, you have to ask the question, what are the consequences?

The consequence now is your your application was queuing request and processing it when it was doing it sequentially. Now that you turned this into a virtual thread, you don't have to queue, which means you can push more request in the in the downstream. If your downstream is not capable to handle that load, it's going to choke up and die. So, be careful before you turn this on.

Sometimes good things can be problematic. So, if you're going to turn this on, you may want to consult with the downstream team and say, "Hey folks, we're going to start sending more requests your way because now we are capable. Are you able to handle it?" If they say no, give them time to handle it and while they are working on it, put a queue in between you

and the downstream so that when they fix it, you can remove the queue and it can go. If they say yes, go ahead and fire up the request. And if they said yes without really thinking, that's their problem, right? So essentially, know that you're going to send more requests downstream and that could be a something to really think about as well. Hope that was useful. That's all

I have. Thank you. >> [music]