About this talk
This talk by Vincent Ngai focuses on tree shaking in JavaScript frameworks like Vue and Next.js, emphasizing its importance for optimizing bundle size and preventing security risks. The speaker shares recent updates on Vue Language Tools, including significant improvements in syntax highlighting, type alignment, and the introduction of rich hover messages in VS Code. Ngai explains the concept of tree shaking, highlighting common pitfalls such as side effects and improper usage of exports, which can lead to unused code being retained in the final bundle. He discusses real-world examples of how tree shaking failures can expose sensitive information and offers strategies for detecting and addressing these issues, including using bundler analysis tools and applying code annotations to manage side effects effectively.
Full transcript
So, Hi everyone. I'm Vincent Ngai from Hong Kong and you can find me online at Circle. Thanks to Amsterdam uh View Amsterdam for having me. Uh this is my first time in Europe and Amsterdam is my first uh stop. I'm uh really glad to be here. Thank you. Um I'm honored to have recently become a part of View team for working on the View language tools uh
in great uh including the View uh VS Code extensions and View TSC. Uh I'm also a View use team member and co-founder of View AI for building some AI stuff like agent skills for View. I'm also a Nuxt uh and uh contribute to the Nuxt ecosystem and managing some libraries, too. So, I'm now based in Taiwan and my I'm a part of the Go lang Taiwan and
Python Taiwan communities. So, if you are interested, feel free to check out my links below. as part of the View language tool teams, before we dive into today's topic, I'd like to share a quick updates on View language tools from the past few months. From the end of last year, we launched an issue cleanup plan and after several months uh of effort from our team, we have
now closed nearly 80% of the issues and resolving multiple bugs that have remained open for many years. So, we are now getting closer to closer to our plan uh to our goal. Next, I will highlight a few of View language tools updates to share with First, we performed key type alignments for V4 and uh when using V4, the for loop in the View runtime actually aligns with
the JavaScript behavior, which will converting number keys to strings. So, in the recent update, we strengthen five number keys in objects in the language tools to align with the View runtime. I believe uh many people has uh experienced some random highlight issues over the years, right? We have finally fixed them and also improved some syntax highlight as well. So, everyone should have noticed some improvement recently. And
next, I I'd like to introduce a brand new features rich hover message. So, after you define props, emits, or slots in view components, when you use the component in template, you can just hover on it and see those definition in beautiful format. And the best part is it's all auto generated. You You don't need to change a single line of your code and it works out of
the box of all UI libraries like Knox UI or Prime Vue. So, this is currently an experimental features. To try today, just go to VS Code extension settings to enable hover rich in view extensions and that's it. We'd love to hear your feedback. And finally, shout out to Johnson for coming up with this amazing feature. We hope to bring you a unique and a better developer experience
than any other frameworks. There's more new features and bug fix over the past few months. If you are interested, you can search view welcome inside view VS Code command plate and see the full release notes and highlights of the view recent updates. Oh, one more thing. We know many of you have been looking forward to TS Go support in view language tools and we feel the same
way. Since TS Go doesn't yet have a complete API design for the third-party plugin, we are actively working with their teams to discuss the direction of the API. At the same time, we are also exploring other possible solutions and planning to build some POCs, so stay tuned. And now, let's get started off today's talk. When she shaking fails, security risk in Next.js and Vue. In this talk,
you'll get a complete understanding of what tree shaking is, what happens when tree shaking fails, and how that relates to security. I'll also introduce some common pitfalls that break tree shaking, including what side effects uh and how they uh affect tree shaking. We'll review real-world tree shaking pitfalls and examples, um how to detect tree shaking failures, and how how can we fix them. Oh, sorry. I'll share
some uh coding techniques that makes tree shaking works properly. So, today's talk will mainly use Vue and Next.js as examples, but I believe the concepts can also apply to any JavaScript or TypeScript project, especially if you are a library maintainer. I believe today's content may be helpful to you. Let's get uh get started from the basics and go deeper step by step. So, what's tree shaking? Imagine
your uh your project use a library that exports hundreds of functions uh features like Lodash, but you only use one or two of them. If you bundle the entire library into your website, that waste resources and can hurt performance, right? With tree shaking, only the exports you actually use are bundled. It's like shaking a tree so the dead leaves fall off, leaving only the useful leaves. So,
what happen if tree shaking is not done well? For sure, the first we can think about is larger bundles and worse performance, but the consequence can be uh more serious than many people people expect. For example, if you had test code or internal features that are not properly uh they could end up being bundled into your front end as well. Things like internal APIs and points for
testing, um admin panel URLs, and so on. And that could potentially lead to a further issues. at a more serious level, sensitive information can leak, too. Your back-end logic and even secrets can leak into client bundles if tree shaking is not done properly. I'll explain how this happens and how to prevent it in this talk later. So, we all know that tree shaking is smart, right? It
can remove unused code at build time. But, have we really verified that all unused code is truly removed? This is the key point of this note. Tree shaking can be lazy. It doesn't always do everything you expect. Next, I will walk through some of the most common pitfalls that break tree shaking. The first one is CommonJS. Even though we rarely use uh CommonJS directly when developing View
application nowadays, if your project use a library written in CommonJS, it may not be tree shakeable. Uh let's look at an example. Imagine there's a CommonJS library with just two exported functions. Then, we have an index file that only uses add function from the library. After that, we bundle the project. You notice that the sub function, even though we never use it, is still I guess In
this example, we can make a small change to help the bundler perform static analytic more efficiently. Right now, we are requiring the uh entire module, so the bundler it has less information about what happened that runtime after the module is required. If we change it to a named require, we give a the bundler more clues, and it can see that we only use add from the beginning
when static analytic analytic. In some bundlers, it can be tree shaken, but not everyone guarantees support for this. Also, if you move require inside a function, it becomes even harder to detect, and tree shaking may fail again. One more case, let's say you use named require and place it at the top level, but what if the library you are using look like this? This is actually a
very common pattern, splitting feature in the module files and we export them in a in them all from library entries point. When you import from it, even with a named require at the top level, it still cannot be tree shaking. You may wonder how common this pattern really is. If you do a quick quick search on GitHub, you will find over 45,000 results written in this way.
So, it's far from rare. If your project uses ES module, bundlers usually get better static analytics for tree shaking, but it's not a silver bullet. If the code is structured poorly and used code can be can you still be bundled. Let's convert the previous common JS example into ESM and use it in a view component and bundle it again. You still see an unused function is bundled.
Can you export why? Is it because we didn't use the name import? If you say switch to name import here, the IDE will report an error. Why? Because this library only has a default export, it combines add and sub in the news object and export that object instead of export each function separately. This style is not tree shake friendly. To be clear, I'm not saying I never
used export default with objects. It's fine for many config files when the object is truly one unit. A GitHub search for this style returns million of results. Although many of them are valid use case, there are still plenty of use case where unrelated things are mixed together in one export. Back to our example. If you want these two functions to be export individually, we can remove the
default export from the library and change the change it to export add and export sub. Uh this that's what we call a proper named export and this is tree-shake You can also place export directly before each function and this also a valid named export. Next, still in ESM, after learning to use named import and export, what if I want to want to import many functions for from
one library? And I use named namespace import for the whole module, can it still be tree-shaken? Yes. Even with namespace namespace import, if static analysis can see that only the add function is has been used, bundlers can still tree-shake it correctly. What if functions uh selection is dynamic, can it still be analyzed? Yes, because the code explicit uh explicitly reference add and sub from the library, so
the AST is still analyzable. So, one more change. If I chose uh the function named as a string key, can bundlers still analyze it? At first, it looked possible, but no, performing static analysis in this kind of scenario becomes more complex and hard to detect. Most of the bundlers cannot handle this. So, now we have covered some basic import export pitfalls. Let's do a quick exercise. We
use ESM named import and named export in our app and we only use one export system.version. In theory, the bundle should only include version, right? Let's check out the out check the output. You'll notice that other code from system is still bundled, even though we never imported directly. Why? This it leads to the next key topic of today. Side effects. This is another major pitfalls for tree
shaking. If I had to describe side effects in one sentence, it is effect outside scope. Let me quickly explain what's what's that mean. In this function, we change the count variable outside the function since in it affects our other scope. The line that increased count is a side effect. In other words, modifying a global variable is also a side effect. In practice, the most common way side
effects happen is by calling APIs with side effects such as console log. The console object exists outside the function and the effects happens in the browser console where a text message is logged. So, let's see what else APIs are also marked as have side effects by the bundlers. Any APIs that change browser states like console log we mentioned before are all side effects. The APIs which access
the networks, perform DOM interaction like changing CSS, elements that affects UI, or the APIs affect the event loop. These API are all marked as having side effects. So, next JSON.parse and JSON.stringify. Who thinks these APIs would have would be marked as have side effects by bundler like Rollup or Roll Down? Please raise up your hand. Oh. Since there is no Okay. It seems doesn't have side effects,
right? They really would but they really would be marked as have side effects. And the reason is they can throw an Once an error is thrown, they definitely affects the outside of program, right? And when the bundler is doing static analyze, it does not check the input will cause the the error or not. So, throw and any APIs that can throw is marked have side effects. So,
it turns out the APIs actually more than we expect. And finally, I want to mention object assign. If you use it to change an object which is outside of the scope, of course it does have side effects. But if even both the source and the target objects in the same scope, the bundlers will still mark it as side effects. Although it's a bit unreasonable, unreasonable, the main
reason is again that analytics is hard. So, bundlers take the worst case path here, too. From my observation, many libraries functions that marked as having side effects are mainly because of they use object assign, like assigning some default options with an object. So, be careful with this one. There are actually many or more APIs that marked as side effects and different bundlers may make different decision. Also,
bundlers may become smarter over time, right? So, let's keep in mind to watch out for APIs or behaviors that had have side effects. how how do side effects impact tree shaking? Let's go back to the previous In the no side effect version, system.js has two exports, settings and version. In our app, we only import the versions. We can see only version is bundled. Great, since setting is
not imported and used here, right? So, the use settings function is tree shaking correctly. Now, we add a side effect line console logs to use settings function and bundle You see that even setting is not imported, the new setting function still get bundled. Why? Because new setting has been called and it triggers console.log, which is a side effect API. The bundler will keep the code that has
side effects to ensure the results remain consistent before and after tree shaking. So, we have talked about talked for a while. So, how exactly can tree shaking affect security risk? First, in many modern web app projects nowadays, we don't have only the front-end code in our code base, right? For example, for example, if you use an SSR framework like Nuxt, your front-end project may share some code
with your back-end like configs or schemas. Once the code are shared, it means if your tree shaking goes wrong, some back-end code or config can accidentally get bundled into your front-end. Let's assume we have a shared file that contains some shared URL configs and you also put some secret data in it. So, you just think, "Oh, it's fine. My front-end only imports the public value, so that
the secret stuff isn't is unused. So, tree shaking will handle and remove it, right?" But after handling, you may see all the secret data is included in the output. Wait, I only used the public home URL here. Why did everything else got get bundled? The reason is inside the setup client, it may throw an error. So, the bundler treats calling this function as a side effect call.
And because these calls use secret key as an input parameter, so it gets bundled, too. Of course, this example may be a bit extreme. You probably won't hardcode a secret key in your as a string, right? Instead, you may put it environment variables or in Nuxt, you may use the use runtime config API. But things like private URLs or backend logics written directly in code really can
leak this way. This example shows a more serious real life real world impact when tree shaking can go get lazy. If you feel like the example I just mentioned is were not real enough, how about some real world cases? The truth is there are actually more of them you expect. Even in in libraries I personally use, I have found many of them can't be tree shaking because
of side effects. Let me pick Zod. Anyone is using Zod here? Raise up your hand. Great, many of you. So, now imagine you define your user schema with Zod. And in some files you also define 15 more and other schemas. Some of them may be just for backend only. And in your front end, you only import and use that single user schema. When you bundle, you'll find
all 50 schemas end up bundled. The reason is the function that Zod uses to define schemas like Z.object, Z.string are having side effects. If someone inspect the JavaScript code of your website, they can know how your data format as are validated and combined with other information leaks mentioned earlier, that may further increase the security risk of your website. after introduce these cases, if you find that your
own project may also be affected, how can you test or detect it? Let's start with a more serious issue. Is there any secret data get bundled into the front end? First, locate the file in your project where you keep secret data. Then find exact lines where those value are defined it. Next, add a line of code that has side effects like console.log and print a very long
and unique string, then run the bundle build. So, after bundling, search inside your build output folder for that long unique string. If you find it, open that file and check the surrounding code to see whether your secret data is included. So, this this method is just that low-tech, right? Uh but it's surprisingly effective. So, you are If you are sure your project doesn't contain uh secret things,
but you want uh unused modules are being bundled, you can use some bundler analyzer tools like Knox analyze uh Vitz bundle analyzer to visualize your bundle and see uh what got included. If you really want to go to line by line to see which part of your code were bundled, there's a a VS code extension called tree shake visualizer that uses source map to mark which part
of your code actually uh ex- end up in the bundle. So, shout-out to Anthony Fu for creating this awesome tool. if you discover that your project actually have tree shaking issues, how do you fix them? And how can you prevent tree shaking from getting lazy again in the future? The simplest approach is split uh files If you originally put different types or of data in one file,
see if you can separate them. Or if you put secret and public things together, so see if you can split them into uh separate files. If your project is large and you can't just move files around, but you still want to fix tree shaking issues caused by side effects, you can try a tree shaking uh annotation called pure that can uh and the usage is very simple.
Just add the pure annotation comment right before the function call that have side effects, and telling the bundler that the following function call has no side effects. Then the bundler will be able to tree shake it away when the client export isn't used. But, there's a problem. What What if I call the function with side effects many times? Doesn't my code become a full of pure annotation,
right? In fact, you can just place it in inside a function right before the line that actually cause the side effects. For example, if you put it before uh console.log, the bundler will mark console.log as have no side effects But, what if a single function has a ton of side effects? Then the functions become ugly again, If this way is doesn't work, and that way doesn't work,
how do we solve this problem? That brings us to the second annotation, Uh if a function contains many side effects, or you don't even know where the side effects are, you can just add a no side effects annotation above the that then any any call to that function will be marked as a pure call. So, that That's how tree shaking announce uh annotation works. So, up to
this point of the this talk, um we've learned tree shaking is about uh only bundle the code or module that we actually use. And when tree shaking fails, the worst-case could be uh uh secret leaks. We also uh saw how CommonJS and uh incorrect ESM usage can break tree We talked about side effects, and how they impact tree shaking, and common web APIs that marked as uh
side effects. We saw how bad end uh secrets could end up bundled into the front end, and we looked into the real-world case. Finally, we will uh learn different ways to detect tree shaking problems. And we also learned some coding skills to make tree shaking works properly. So, lastly, I want to share some uh story about tree shaking in the real world. Let's start with schema library.
Remember the Svelte example I mentioned earlier? The first time I noticed this issue is or was in the 2024. Uh back then, Svelte 4 hadn't uh been uh released yet, and I tried to optimize tree shaking uh for Svelte. But, because of its API design used uh function chaining, it's not really possible to optimize it with tree shaking annotations. Around that time, I started to work on
one of Anthony Fu's project, and it was uh using Validated Bot, which is a Svelte alternative. So, uh also wasn't optimized for tree shaking at that time, but after we reviewed the code base, seems its API design can be optimized. So, we shipped a PR together to Validated Bot to optimize the tree shaking. I also sent another PR to further improve the tree shaking after that. So,
after that, Svelte released version 4 in April last year. It introduced Svelte Mini at uh and that API is very similar to Validated Bot's. It also claimed to be tree shaking friendly, but I uh I had already switched to Validated Bot at that time, so I didn't dig uh dig dig deeper into uh Svelte Mini. Uh until a few months ago, I started to prepare for this
talk and want to share uh use uh Svelte as an example. That's when I found out Svelte Mini still have tree shaking problems, and it's actually exactly the same that uh Validated Bot had before. So, I opened a PR and optimized tree shaking for over hundreds of the APIs in Svelte Mini, and it got merged about 2 months ago. The last thing I want to share is
um the current tree shaking situation in the Vue ecosystem. Luckily, Vue core marks most of the APIs no side effects annotation. So, if you define some reactive states, but you never use it, it can be tree shaken away. As of the uh view ecosystem libraries, in 2024, I sent a PR to Pinia to optimize tree shaking. So, if you define a store in Pinia, but you never
use it, it it can be tree shaken away now. And last year, I also sent a PR to VueUse to optimize tree shaking for over 100 of Vue's APIs. For instance, if you use like uh create global states to create a store with VueUse, but you don't really use it, it can be tree shaken away, too. So, congratulations to all Vue developers. The library we commonly use
in Vue ecosystem are now optimized for All right. So, that's my talk for today, and hopefully it creates some positive side effects for you.
