About this talk
In this talk, Carlos introduces Undici, an HTTP client for Node.js designed to enhance performance and usability. He compares its capabilities against traditional HTTP clients like Axios and Got, citing key performance benchmarks that position Undici as a superior option. Carlos explains how Undici diverges from the typical event-driven model seen in Node.js, opting instead for a state machine approach that reduces overhead and improves efficiency. He discusses features such as connection pooling, DNS caching, and a built-in mechanism for HTTP/2 support, emphasizing its ease of use and customization. Finally, he touches on the potential for HTTP/3 support, indicating promising developments in that area.
Full transcript
Hi guys. Well, I see Let's see if everything is I'm just going to wait a little bit more for people to get the the headset on. So, hi. Hi everyone. My name is Carlos. I hope that everything is going good. I hope that I have you enjoying the the conference so far. So, for today I'm going to be talking a lot of a little bit about Undici.
I'm not sure how many of you have heard of it before, how many of you have heard about uh this specific or particular library. And I'm going to be talking a little bit I will try to make it short. I will try to make it easy for you um about something new that we have been recently starting working on. So, without too much else to say and
to close this small window right here, sorry. There you go. Uh well, let's get started. first of all, I think that everyone has already been using or has used before uh any kind of um HTTP client in the past, right? I think everybody knows it. You have been using it maybe in a server-side application, maybe in a web browser. I don't know. well, most likely you have
been using either Axios. I don't know how many of you knows Axios, how many used it in the past, how many had experience with it. Nice. How many of you have used Got? Got? No one? Almost no one? I also you. >> [laughter] >> Any kind of flavor you have already used Uh what about Node.js HTTP or HTTPS? Plain. Oh, okay. Someone likes to get in trouble
here. Then, um what of what most of them, if not all of them, has in common? Performance. There is quite a bit of, let's say, interesting trade-offs with HTTP model from Node. Um and as one really nice uh friend of mine and also colleague that I had opportunity to work on has always said, performance is a feature. And yep, Matteo Colina, he's the original author of the
one of the things that I'm going to be speaking about, but before going further, of course, I need you need to know a little bit about myself. So, my name is Carlos. You can find me as Metcalfe95 in GitHub. I'm a software engineer at Auth0, you know. Also, my name of course is Matteo Colina. I maintain frameworks like Fastify, I don't know if you have ever heard
of it. Undici, of course. Piscina, and also Node.js collaborator, some of my work, and now I'm trying or planning to be a speaker a little bit. Let's see if it goes well. And yeah, I couldn't find any better picture, most likely because I'm not a picture guy, so most likely a picture that comes out of here is going to be my next profile pic, so let's see.
And first, let's take a look at the numbers, and I would like you to take a closer look to the numbers. Um, I hope this is a little bit clearer. It's better here than in my laptop. Um, here is a small um benchmarking done from platformatic, well, from Matteo Colina, of course. And here you can see a table of how the Node.js HTTP module performs compared to,
of course, the frameworks that are written on top of it, and and Undici, which is the the main um actor of this talk. So, first, you can see that on top of everything we have Undici, right? Around 24,000 uh requests per requests per second, if I'm not wrong. Yes, requests per second. And behaving on top of the pure HTTP, Superagent, which is a wrapper around HTTP, and
with some configurations on top that makes it a little bit more performant. And of course, you have the rest of the HTTP clients that we already spoke about, Axios, Got, um Node Fetch before. That is not maintained anymore because they suggest you to move also to fetch which also goes under undici and yeah. So as I said, performance is one of the main problems for the node
HTTP model. But of course the biggest here the biggest question here is why, right? Why is like all of them suffer for example the same performance problem. And this can be summarized in three key factors. The first factor is ergonomic trade-off. is made following the event driven pattern of JavaScript, right? Everybody knows event emitter, everybody knows event target. So in order to make it ergonomic and easy
to grasp, the node HTTP model was made around event emitters. So everything is an event. As I said, we here here we have an example taken from the node.js from the node.js website. And you can find that everything is basically an event. You want a piece of data, you use the stream and an event listener to the on on data event. You want to listen in case
of an error, you go through an error an error event listener and everywhere. So this involves a lot of the creation of several objects and on their need creating a huge garbage collector pressure for node itself or for the V8 engine itself. This comes with CPU overhead overheads of course and on top of that the garbage collector pressure because you need to constantly be creating event emitters,
removing the event listeners depending on the behavior of the HTTP client so on and so forth. And this of course with the humble goal of make it easier to use but at the expense of a little bit of overhead on top of that. Uh throughput limitations, pipelining there is no pipelining support, of course, in Node.js HTTP. And in order to have the pipelining is a little bit
complicated. So, pipelining is just, let's say, enqueueing several requests behind of the other in a single TCP round trip. it allows to send multiple requests through single TCP connection, etc. It kind of increments concurrency at a specific cost because you have to still the problem of head-of-line blocking, but it increments the throughput at some extent. But it's still um it still is a little bit um well,
in Node.js HTTP it's not so easy to get it done. So, there is this limitation already um from the model itself. and two worlds colliding because the Node.js HTTP model is just another written in C. Uh under the hood uses the HTTP parser, which is called llhttp, and interacting between JavaScript and the native layer has a cost. It's not something that happens um for free. To call
it in a way, crossing in this this um data exchange layer between crossing a good overhead on top of that cross this binding, put the data available for the V8 so you can have this data on top of on the JavaScript um world and you can start managing, doing with it whatever you want. So, this boundary has a cost. It's not for free. well, let's fix it,
right? We know that it's broken, we know already what is not working, we know the problems that we're suffering. Let's try to fix it, let's try to do something about it. It's not like we haven't tried, it's just that it's not possible. as doing it requires a lot of effort, the ecosystem is already written on top of these things. So, the ecosystem uses a lot of these
APIs, uh they are really used or tied to these contracts that in order to make it better or in order to make it perform and most likely you need to broke them. And I don't know how many of you remember the a smudge gate with uh smudge gate in Scala, yeah. The smudge gate with MooTools back in 2018, I think it was. Yeah, in 2018 when the
flatten uh array flatten method was introduced. There was a huge collision because another MooTools library was using it already and a lot of websites were using it. So, when the specification for the 2018 ECMAScript came in, it collide and it was a lot of websites broken. So, they need to change it to a different name which I don't remember right now. It's complicated with the community already
has everything built on top of it. So, there is going to become complicated because you're not trying to break the internet in really small on a really small summary. But then, what is the solution? What we can do about it if we cannot fix it from the get-go because it's already so intrinsic to the community, what else we can do? Let's just rebuild it from a scratch.
So, say ciao to Undici. Uh yes, it's a reference to 11 from a Stranger Things. It's also a 1.1 together in case you were wondering. This was written initially by uh by Matteo Collina, Ronak, and another team of uh Robin Nag and another team of um Node.js collaborators. And the key Well, in this talk, I'm not going to be trying to explain everything from the get-go. I
think Well, from the from a scratch, I think Matteo is going to be a better doing a better job. But you have a lot of talks from Matteo explaining that. I'm going to going to do to try to make a quick primer on why it goes bur compared to other to other libraries. And yeah, for the rest, you can ask Matteo. You can reach him on on
Blue Sky, Twitter, or directly in GitHub. So, instead of an event emitter, what we'd use in Undici is directly a dispatcher. So, the dispatcher is the baseline API where we build everything So, that means you have a state machine that it starts from the moment you trigger a request all the way down until you have the trailers from your Sorry. Until you have the response back from
the server. So, that makes everything easier in a sequence of events that doesn't necessarily needs to trigger an event or doesn't necessarily needs to be an event emitter, just a stream of See, a stream machine where you put some events that transitions between all of these states. So, this is an example. As you can see one of the biggest difference is that there is no more events.
These functions are going to be triggered in a specific sequence given the state of the request. We have the on request to start when we just get connected into the server, on response error in case something was wrong, on response to start as soon as we receive the headers of the request, on response data every time that we receive a chunk of data depending on, of course,
the socket capabilities and everything configured and also how the server delivers the data, and on respond end when everything is ready wrap it up and if there is any trailers, also the trailers can be collected right here. So, it's a big difference compare from the event emitter version or event emitter event emitter approach to the state machine one. And off on top of that, we also have
pipelining with keep alive. Keep alive is enabled by default. Pipelining is also customizable and you and is easier as you're setting, okay, I want this pipelining number, how many requests in a in a round trip, and how many of how long I want my keep alive timeout. Easy as that. Trying to save a little bit of round trips, trying to make things easier, and increase a little
bit the performance, of course, and along some other design decisions that make it uh way more performant than the Node.js >> [clears throat] >> And on top of that, it's powered by Wasm. We use WebAssembly to the same LLHTTP model that I talked about in the previous slide that powers Node.js HTTP. It's the same one that we use in Undici, but instead of doing this binding native
binding that you can do in Node.js, what we did instead is use Wasm to open that portal between two worlds and start communicating and exchanging data through buffers of through buffers of memory. So, everything is let's say more native, less overhead, easier to grasp, and easier to also port to the different um machines, to the different platforms that is outside. And yeah, what is a really perfect
use for Wasm. So, there are nice things that also comes with Undici. One of them is connection pooling. So, you want connection pooling, boom, you have a pool. Easy as that. How many connections do you want? What is the pool of connections? Do you want to reduce them? At one time do you want to preempt it? You have it. Do you want a smart retrying? We have
an interceptors. So, interceptors is a powerful concept that we have in Undici, which allows you to extend or as name says, intercept the before triggering the request to the server, intercept a specific um intercept that specific call and customize it with things that you want to do and things on top. For example, we have a retry interceptor that allow you to do smart retries. And we also
have Okay, you want also DNS caching? We have DNS caching. This one is an example of using a quick LRU um NPM model. We just get the interceptor from the Node.js interceptors. We create the cache the quick LRU cache. We just need to create a small adapter to make it conform the contract that the interceptor is looking for. And easy as that, you are now have DNS
caching on top. You want also support for socks 5 maybe. You also have a a support for socks 5 proxy agent. Easy as that, you can have authentication or have authentication. And it's just as easy as instantiating a simple agent. Uh also, for the people that doesn't know, Undici is the one that powers fetch in Node.js. easy as that is the way that you can all these
things that you can see for example that I already explained to you can be used on top of the fetch in For example, we have the interceptors cache which allow you to do a cache in accordance to the error of seeing 9111. You just import interceptors, import the global dispatcher. create an agent where you put the interceptors on top. You put in specifications like where is the
store, where is the size, where is the count, blah blah blah blah blah blah. Uh set the global dispatcher. And easy as that, you have automatically all your requests already cached in accordance to the error of seeing as stated before. So, the main topic of the of the call. Yeah, I put it the the title as implementing HTTP/2 for the modern web, right? I've been speaking a
lot about HTTP/1 and Undici 11 so on and so forth. Uh but what does Undici have to do with HTTP/2? Well, since 2023, we have been supporting HTTP/2 from the get-go. Which means that you are now able to use HTTP/2 features with the same HTTP/1 contracts that I have been explaining explaining to you some slide before. That means easy as this, you can have either an HTTP/2
an HTTP one call or an HTTP two connection HTTP one HTTP two connection in accordance to what the server supports. You advertise to the LPM through the LPM negotiation. Hey, I support HTTP 1.1. Hey, I support also HTTP two HTTP two. The server decides which one wants to follow. Uh first commit introduced was introduced September 8, 2023, but of course it was mostly available as an experimental
feature. Uh but then why we did it? Well, if undici it's basically 11 in Italian, it also hints the HTTP 1.1 name and so on Then there is why not? There was effort done by the community. The community wanted it. So, we gave it a look, decided to give it a try and so far we so far in this case I could go. And after almost six
months of work, we got it materialized. We have it on September September 8 at the first commit. We have a release set under And as I said, always trying to keep everything Oh, this is pretty small, I think. That's very good. Easy Oh, sorry. Easy as that. Just set the allow HTTP two or allow allow HTTP flag to true and automatically the same thing that you have
the same contract, the same events, the same what we call dispatcher, the same dispatcher handler, the same configuration that you have automatically supportable between the two HTTP HTTP versions. Without any effort, without having to worry too much about any other configuration, just put the flag and you are good to go. Same primitives but with HTTP two support. One client because in in undici we have the concept
that a single client is basically the same as as a socket or as a connection socket. So, one client, one connection. In this case for to the 2 one client in Undici is equal to one session one HTTP 2 session. Pipelining is similar but not the same to multiplexing because of course it's a bit more nuanced than pipelining. And the streams are managed under the hood so
you really don't need to worry too much about um what any other kind of intrinsics of a next for for HTTP 2. But that's not it. Of course once we added support to HTTP 2 we were thinking of we don't only need to have the support for the previous primitives which is out of the hood which is something that is a good to have. So in that
case you don't have to have much effort to do the transition. But on top of that is we also wanted to support everything that we have in in Undici we also wanted to support it with HTTP 2. That means fetch. And that means of course web sockets. So web sockets now supports a Node.js out of the get go supports HTTP 2 doing exactly the same approach as
for fetch. You just create an agent, set the flag allow HTTP to true, pass the the the agent as a dispatcher for the web socket connection and you have now web socket with full HTTP 2 support. >> and as for now as per October of 2025 so it still is in experimental so you use it you will have a small warning underneath saying it's experimental but the
idea is to eventually graduate it after getting a little bit more feedback uh from the community. And one of the amazing things about it is that this paves the road for HTTP 3. HTTP 3 is already on the get go for Node.js. Actually as that's a topic for another talk. But, as per today, this commit, just let me see if I can click on it. No, I
cannot. I will try to go right here. Oh, yes. Sorry, I'm a little bit lost. I tried to open a new one. Yes. Let me see if it opens. This commit has just landed in Node.js, which is the first commit, the first Oh, Where is Where is Where is the first support for quick in Node.js. So, this landed just 3 days ago. It is not available yet
on the new stable version of Node.js. Nonetheless, with a specific tricks and specific flag, you can start using it. And thanks to this commit by James S. Snell, of course, thank you for the for for the work on that. We are now on the works of having quick support in Node.js. And that means that as we did for HTTP/2, the idea is that eventually we can also
have support for HTTP/3 as not as soon as it comes available, but we have the opportunity to experiment more with it in a stable version. let's crush some numbers. At the very end, we are talking about we are talking about these. We really like numbers. We really like to talk about performance. We really want to see I already talked at the beginning that Undici overall behaves better
than most of the HTTP clients. But, okay, let's see how it goes with HTTP/2. And the numbers are basically here. In comparison to what we did to HTTP/1, and this is something that I need to put as a disclosure. Here, what we did is to use the same Node HTTP/2 2 instead of bundling the uh the C the C model in the same way that we did
for HTTP 1 because the HTTP 2 the Node HTTP 2 model doesn't have the same pain points as the HTTP 1 has. And we didn't see too much um let's say of a need to do the whole rewrite as we did for HTTP 1. So, what we did is to embed it on top. That's why in first place you will have that the native one behaves pretty
good. The native one uh is the one on is is the one that behaves on top, followed by the undici which has all the um you can see the performance around throughput of uh 9,000 operations per second, followed by undici fetch, which is expected given the specific constraints of the specification and everything, followed then by got, later axios, and that confirms the list. So, as per undici
8, I'm not on this one. Yeah. I'm a little bit on top. Yeah. Ooh. So, as per undici 8, we are well fully able to well, HTTP new HTTP 2 support has become officially available for undici 8. It's now the new default. That means that from the beginning anytime that you try to make a call with undici, either the request, the dispatch, the pipeline, or fetch itself
in Node, you will automatically the client will will advertise the support of HTTP 2. And as I as I said at the beginning, it's up it's up to the server to decide which protocol to use. This is included in uh this B26, which also was released few Uh that means that of course fetch will support all the out of the box as well. And what a nice
time to be alive. So, this is it. And um here you can you will find a few QR codes. One is to the Nishi repository in the case you want to get in touch with the code, in case you want to see the implementation in detail. And the second one is a QR code that goes directly to the benchmark that I did, so you can take a
look, play with it. Uh you also find in one of my one of the benchmark that I made a mistake, there is a feedback that you will try to take a look or something. Feel free to just um to leave an issue there and happy to discuss it and I do the further adjustments as we need. So, this is it. Thank you very much and if there
is any other question, feel free to get in touch and happy to answer. Thank you. Thanks for your time.
