Helion 1.0: A High-Level DSL for Performance Portable Kernels - Oguz Ulgen, Meta

This talk introduces Helium, a new domain-specific language (DSL) for kernel authoring that simplifies the process for developers. The speaker, a software engineer from Meta Super Intelligence Labs, discusses the journey of Helium from its open-source inception to its first release, Helium 1.0. With Helium, users can write PyTorch code, and the framework automatically converts it into efficient lower-level representations for various hardware backends, including Triton. Helium also features a powerful auto-tuner that optimizes performance with minimal effort, allowing kernel authors to focus on high-level code. The talk highlights Helium's functionalities, including support for multiple backends and enhanced control flow capabilities compared to existing technologies. The speaker emphasizes the aim to make Helium user-friendly while facilitating high performance across diverse hardware platforms.

Hello. I'm Ouz. Today I will be talking about Helium, the new DSL we're releasing for kernel authoring. I am a software engineer at Meta Super Intelligence Labs, and this is a work we've been working on for the last year or so. So, let's talk about like how we came here. So, Helium started an open source sometime around the beginning of 2025. From even from day zero, we

were always GitHub first. We wanted to share all of all the work we're doing with you directly on the GitHub. And in October during the PyTorch conference PyTorch conference in San Francisco, we announced Helium as the beta version. And with this and at the same time we also presented Helium at the Triton Deep Conference and also at the PyTorch conference. And at the time we only had

our first backend, which was the Triton backend. Essentially, Helium lowered into Triton only. And then progressing into the beginning of this year, we worked with our close collaborators and then we did our first Helium hackathon. And then in San Francisco we were able to have a ton 350 plus applicants and 50 teams writing state-of-the-art kernels for various attention models. And then at the NVIDIA GTC, we were

able to demo the Helium kernels and then like introduce the world like new ways of authoring kernels. And now at the Paris PyTorch conference, we're finally joining the PyTorch Foundation and as of today we're releasing the Helium 1.0. With this, we'll be introducing more backends to the ecosystem and make Helium be the common place to author kernels in a very easy and portable fashion. So, having said

that, let's talk a bit about like what actually is Helium. Normally for most people, kernel authoring is kind of tedious because you have to learn about all these like hardware semantics. You have to understand how I don't know, like suppose you're writing like you're working on an ampere kernel or a blackwell blackwell kernel. You need to understand like how warp specialization works. You need to understand how

like individual I don't know tiling works. With Helium we our goal is to make this as simple as possible. You know how to write PyTorch code and essentially this means you know how to write Helium code. All you have to do is write your PyTorch code using tiles and then we're we we're able to lower it to various different lower level languages and then you we're able

to get your high performance kernel just by writing simple PyTorch code. Um as I mentioned earlier we currently lower to Triton by default. Uh in the future we're currently working on different backends and an example like some examples of these are we can currently work on our Palace backend, on a QTESL backend and also a metal backend for the Apple devices. another nice thing about Helium is

that comes with a very powerful auto tuner. You don't have to worry about exactly like how to write different semantics for your kernel. We just take care of all these for you through our automated searching. So, Tri Dao has this nice tweet from last year which I really like. Essentially what this what this like picture graph here describes is that how much effort you need you need

to put into your kernel authoring in order to how much performance you get. So, what essentially this says is if you are use torch compile you have to put in very little amount of effort because torch compile is like very straightforward and then you get a quite a bit of the performance. But if you want to get like eke out more performance then you have to start

using lower level languages like Triton or QTESL. At this point you probably spend like not that like your productivity is still pretty high. You did not spend all that all that much time and you already got like 90 plus percent of the performance. But if you want to get like the the very last piece of the performance now you have to go down to like CUDA or

PTX and then spend a ton of effort to do this. So, in this chart I tried to I thought about like how how could we place Helium? And the place I feel like the Helium fits best is you get as good performance as Triton or Q DSL with the same amount of productivity as writing torch compile code. The reason being is that as you will see in

a in a bit helium is very similar to PyTorch. If as I mentioned like if you know how to write PyTorch you already know how to write helium. And then as we lower into these like Triton or Q DSL type of languages you're able to get the same amount of performance from them. So we're kind of best of both worlds in some sense. So let's talk about

like why we actually came up with this new DSL. As you all know like the the space of DSL like kernel DSLs is actually quite crowded. There is like many different DSLs like there is and like people are coming up with more DSLs like there's Thunder Kittens, there is Gluon, there is TLX, there is Triton, Q DSL, Flywheel. It's just like it's getting a very crowded space.

But what all of the most of these DSLs do are they're like so low level. And then in some cases they're only work on a single type of like machine like for example Q DSL works on Nvidia GPUs. Or like in a similar fashion we don't we want to make sure that there's as much collective like heterogeneity as Another example here is that like when we were

inspecting our own code base we noticed that there are so many similar kernels with very few differences. For example, you will see the ton of like add kernels or attention kernels with very similar like epilogue or like prologue fusions. In this case we want to make it as easy as possible for you to write these kernels. Essentially you write the same kernel and then helium lets you

do these different type of like prologue epilogue fusions or port to different hardware. And in terms of auto tuning the nice thing about helium is that you write your kernel once and then we're able to generate many different types of configs for you. And all these configs end up creating many different kernels and then our powerful auto tuner just looks at these kernels and then decides which

one is the like the best kernel that it can pick and then it tells you the best kernel. So essentially you again write once and use everywhere. And I guess yeah, with this motto, Halide currently works on all like Ampere plus Nvidia GPUs, similarly on AMD GPUs, and you can even use it on CPU via the Triton CPU backend. And essentially like if there are other accelerators

that can consume Triton, for example like Metal Zone MTAA or other like Intel's accelerators as well, you're able to just Halide use Halide out of the box because we support Triton backend. And on top of this, as I mentioned earlier, we are working on other backends to support like QTSL for Nvidia GPUs, Palace for Google TPUs, and Metal for Apple devices. So now let's like look at

what Halide looks like. So this is a matrix multiply kernel in Halide. If you notice one major difference between regular PyTorch, like there are two major differences between regular PyTorch code. One is that you have the Halide.kernel decorator, and then two, your for loops, you have this HL.tile construct. If you look at the internal part of the kernel, the color the code that's highlighted with green is

what I would like to call the host code. Host code is the part that doesn't have anything special in it. All it All it is is that like you can do your create your output tensor, you can do assertions, you can get your sizes, and you can like return your value. And then the interesting part with Halide is the device code, where this is the code we

lower into the device. Your outermost HL.tile turns into the grid of the like outermost grid, and then your inner inner loop is another for loop that's inside the kernel. And as you see, like writing Halide is very straightforward. It is like you're using torch.addmm, you're using various other like regular torch constructs. There's nothing that's special about like just essentially you already know how to write Halide. So

Halide uses the HL.tile to do this subdividing iteration into tiles, and then um we're able to do various certain ordering, flattening, swizzling in the outer tuner. And as I mentioned, like if you know PyTorch operations, you're already able to lower them. So, for example, like pointwise operations like add, sigmoid, they get lowered in lowered via the torch inductors existing lowering capabilities. Same with like reduction ops like

sum, uh, softmax. And if you want to do like a matrix multiplication, uh, we're able to lower it directly into the native backend, uh, like operation. For example, in Triton that would be a H like tl. And similar for V ops. One other nice thing about Helion is that we're able to support various different types of control flow. If you're familiar with PyTorch, uh, sorry, like torch

compile, uh, one thing you will notice is that, uh, it doesn't really support like various control flow operations, but for Helion, uh, we want to be as close to like a native, uh, kernel authoring experience. So, we support, um, these operations via creating multiple, uh, graphs. So, now let's talk about like the some of the configuration space, uh, for Helion, like what we're able to do easily.

So, um, a nice way to start this, uh, is the how you actually index into a tensor. In Triton, uh, you can choose between different types of indexing. For example, you can do pointer base, uh, block pointers, and tensor grid descriptors. As you see, these are like quite, uh, complicated syntaxes in order to achieve these. And with Helion, all you do is you just write as a

regular array index and then we're able to output all these three different types of, um, indexing choices. Similarly with block sizes, like one of the most tedious things about uh, writing a kernel is having to use like do all these, uh, calculations to like that how does my block size like fit? Do I need to, uh, like what what do I need to update? But every time

you have to change it, you also have to go through every single place and then update these. With Triton with Helion, all you do again is you write your for loop and then we take care of we we take care of all of this And then similarly, um, sometimes, uh, you want to actually like flatten your like you have a single loop and then you want to

flatten it. If you're able to also just do this we are just one configuration option. Another thing I really like about Helium is the reduction rolling. Sometimes you want to do a reduction, but you don't you are not able to like if you try to load that entire tensor into the memory, it's too big and it like causes a problems in Helium in Triton for example this

is with a TL load operation, but loading this entire row is too expensive. It doesn't like maybe it doesn't fit into Triton's shared memory. So what you in Helium what we are instead able to do is you if you specify a reduction loop, then we're able to just roll it for you and then now you're able to get better performance without actually changing your code at all.

Another nice feature is the different bit types. Most of the time when you write a kernel you either use like a flat bit or like you use like a multiple dimension like XYZ in this case. And sometimes you also want to use persistent kernels. But in order to test try between these different kernels, you actually have to significantly modify your code. You're not able to just like

make a small change. It's like it's a major architectural change in most cases. And with Helium all you do is just write one like you just set your bit type to whichever one you want and then you get the best perform like you get all these different types of options and then we're able to auto tune to pick whichever one is the ideal one. And similarly we're

able to do like L2 orderings, we're able to switch X and Y to Y and X um for like reorderings. And then the last part another interesting part here is the Triton tunables. So a lot of the Triton for example like the TL.range, they provide some of these actuable operations. You're able to do unroll, warp specialization, do num stage and buffers as etc. But in most cases

people don't actually realize these and they don't actually tune through these spaces. But most people end up doing is they just write a TL range with like the begin of the range and of the range and then they kind of skip over the rest of the arguments. And similarly for num stages and warp stages you can use the Triton auto tune decorator, but it is very frequently

committed. And in this case like healing is already able to do all this for you. You don't have to think about all these like fine details. Healing just takes care of these. And I think I believe the last configuration I want to talk about is automatic masking. So something that I've noticed actually from my own kernel authoring experience is that in most of time like you write

your kernel for like powers of two shapes and then for example if you're doing like online inference you end up having to deal with like these odd sizes. And in those cases like you might run into like a CUDA AMA or you might run into other problems. With healing this is actually super simple. All you do is you set your static shapes to be false and then

we're able to generate all these like masks all these like offsetting operations for you. So you don't have to worry about is my kernel specific to a particular power or we take care of this. And also you're able to introduce like user defined auto tuner knobs as well. You're not only limited by like what we provide but you're able to try different things. For example if you

wanted to do say sub tiling you can say here's like if you have a you have your enum and then you say like if this is zero don't do sub tiling if it's one you do sub tiling and then you are able to just feed it into the healing auto tuner to tell what would be the ideal choice. So having talked about all these auto tuning options

let's talk actually talk about how auto tuner works. So currently our we are default auto tuning algorithm takes about 10 minutes to search through 1200 different Triton the built-in auto tuning algorithms that we have are like pattern search differential evolution finite search and our default one is the LFPO pattern search. If you look at our documentation website you're able to like you can learn about all these

options but in most cases we recommend you to use the default one as it is our like most powerful auto tuner. And in the future, we believe that you can easily augment the human auto tuner doing various reinforcement learning or even you can like connect to various like LLM like you can have a cloud for example the side like given this I don't know like given this

benchmarking options, what would be a good way to decide on the next generation of the options. So, we are able to easily connect this auto tuner into the various different backends in the future as well. So, let's talk about how pattern search actually works. So, we select initial population of 100 random configurations and then we hill climb the top five of these and then once we get

to these we find there there the local maximas and then we select the fastest one. This by this logic we are able to the figure out actually like the fastest configurations between these. What makes this even better with LFPO likelihood free Bayesian optimization is that we're able to use an ML model to do better classification and through this we were able to significantly reduce the amount of

time we spend on tuning. Previously like one of the biggest complaints here we received was like how long it took to auto tune and in some cases you would you would see like 20 30 minutes but when we switched over to the likelihood free Bayesian optimization, we were able to reduce auto tuning tuning time time anywhere from like 5 minutes to 10 minutes depending on how the

operation tends to be. So, some interesting comparison here is if you look at like these comparisons with LFPO version we are able to have So, in the top chart LFPO is the blue one. We are able to get as good performance as the differential evolution which was our previous However, if you look at the bottom chart, the LFPO one was able to complete in significantly faster times.

In some cases like in for example layer norm you are able to see it is like almost four times faster than the previous differential evolution algorithm. And similarly you can see that in most cases our LFPO pattern search of random forest particle swarm optimization is able to get to a pretty good configuration with only 30% of the candidates and in order to actually like like the absolute

best configuration variable variable to it raise for a few more percentages in order to get there. But in most cases auto tuner is so powerful that only after a few steps you're already able to get like a pretty good configuration. Um so as I mentioned one of the powerful parts of healing is auto tuner and it takes a quite a bit of time. So you don't actually

want to run auto tuner in production. Like when we talk about deploying a kernel, we don't actually want the kernel to be deployed in production while with the active running auto tuner. Auto tuning is something we think about as something you do ahead of time for the known like the shapes you have or like for the unknown shapes we are dynamic shapes and then we want you

to either do something along these lines where you hard code the particular decision for your kernel or we want you to do like a more dynamic approach where like you can say oh my dates for my smaller data set use this configuration for my larger data set use this configuration. We also you can also do this decision based on any kind of hardware. Essentially our auto tuning

is just a decorator in Python. It is very flexible you're able to do any Python magic you want in order to control the particular version at spec. Now let's look at performance. So this is a I'm going to talk about three different cases this year. The first case that we is the attention performance as attention as we know attention is like one of the most important kernels

in machine learning so the interesting things to look here is the this green chart with the I guess dashes. So the healing plus custom Triton is a collaboration we did with the Triton team at face at meta as well as the Nvidia's compile IQ. Compile IQ was tuning system released by Nvidia during the GTC past like past month and we have been collaborating with them for a

while. And with Compile IQ plus our own Facebook Facebook Triton which contains automatic warp specialization for both Hopper and Blackwell, we are able to get as good performance with Helion as we would get with significantly lower language like Gluon which exposes a ton a lot more of the hardware like the hardware semantics. And the next thing here is that the same performance you're getting, you're getting with

like only few lines of Helion code compared to like hundreds lines of Gluon code. And of course in this case like CUDA and still is like has a is faster than both Gluon and Helion because it is able to do more. But we are very with just with Helion we are still able to get very close to it. The second case study is looking at uh CuPy's

cuTSSL kernel library. On this chart, you're able to see in the lower in smaller shapes Torch Compile actually performs pretty good. And in higher in the like larger shapes CuPy is able to perform pretty good. But if you look across the board, Helion performs great in every single uh shape here. You don't have to like do a pick and mix and match. You can just use Helion

and you get the best performance in all different types of shapes. And the last comparison I want to look at is comparing against Tile Lang and the Flash Linear Attention. In this chart you can see that Helion again performs across the board just beats every configuration provided by like Tile Lang's own experimentations and we are able to have a higher performance in both GDN forward and Mamba

2 chunk scan. So in all of these case studies, now let's also quickly take a look at some of the kernels that we have selected that is like that we decided are quite representative. So, here we are uh having our our number our speedups are compared against the PyTorch eager code, and we're comparing against Heatsink speedup, Torch Compile speedup, and Triton speedup. In these cases, all the

Triton kernels were taken from the Liger's Triton kernel library. There we did not write the Triton kernels, we just selected them from like the high-performance Liger library. And then you can see in most cases Heatsink just outperforms across Also, I guess another thing to mention here is that the all these speedups are done for many different shapes for all these kernels, and then we take the geo

mean for all the individual kernels. And you can see that Heatsink outperforms Torch Compile and bespoke Triton kernels in almost all cases. There is still like we still have a few little bit more room to catch up in few of the kernels. And you can see this So, the previous slide was for Nvidia B200, and similarly for AMD MI350X, here you can see the exact same comparison,

where we are able to Torch Compile and Triton in most cases Um so, as earlier as Mark mentioned, we're joining the PyTorch Foundation. This is a huge step for us. We're moving like previously all all the maintainers of Heatsink were Meta employees, and now we're moving into a multi-org maintainer system. We have our first maintainer from Thinking Machines, and we'll be onboarding more maintainers. You want to

be as like open source friendly as We will have a technical steering committee under PyTorch to set the priorities. And going forward, we'll become more multi-hardware, and we'll become like more vendor neutral. We don't want to be locked into a specific vendor, we want to be like the one DSL that's able to handle many different types of hardwares, so that you just write once, and then you

use it everywhere. That's essentially our motto. And again, all our repos repos repos entirely open source is pytorch/helin. Your all contributions are welcome. So, the last thing I want to talk about is what's is happening in 2020 actually sorry not that thing. So, what I really want to talk about is the what's happening in 2026. Our technical roadmap includes distributed support. So, so far everything I've showed

you has been about a single single GPU. But, we actually want to depart those into the distributed world and use NVSHM and similar sub in order to be supporting more distributed kernels. Another thing we want to do is automatically generate backward kernels. So, Helin is actually quite strong. You're able to write only your forward kernel and we're able to do the differentiating using various like pytorch compiler

mechanisms in order to get generate the backward kernel for you automatically. In addition to this, one of the most requested features we have got so far is better compatibility with torch compile. Like just writing your handwritten Helin kernel and having it having like prologue and epilogue fusions with other torch compile operations so that you are able to easily get these performance optimizations. And as I mentioned, we're

already working on different backends and we're also collaborating with the VLM team in order to bring Helin as a official backend to VLM. We already have a lot of contributions into VLM repository and we Helin VLM as easy to write Helin code as well. And in a few months, we'll be giving a tutorial at PLDI talking more about Helin. And please stay tuned for more events. I

guess very quickly I also want to touch upon the compiler internals. So, quite actually like a similar to torch compile where we do the we try to capture the graph of the python code. The one big difference is in Helin we actually use the python AST as opposed to in torch compile there's actually like there's runtime tracing happening. Helin the code generation is entirely static. You don't

actually have to execute the code in order to generate the various different back ends. We do AST parsing and then we extend this AST with various types and metadata. And then we use we have our device passes. What this does is that we lower it into an effect effects graph that can be fed into the torch compiles code gen mechanism so that we are able to take

advantage of the existing torch compa- like torch inductor code gen. In order to like one of the simplest ways of introducing a new back end to Helion is actually to implement a torch inductor back end for torch inductor code code gen for that back end which cre- essentially creates a more or less free way to introduce into the Helion. And then we do the AST rewrite and

then we are able to generate the new kernel Yeah, Triton is available today. We just released our 1.0. I I believe I published this on last Friday. Any open source contribution is welcome. Please connect connect with us on the GP or Discord. There's a Helion channel. We actively monitor that channel and then we do our announcements there. We interact with the community there and then you can

look at our code on pytorch/helion and our documentation at helion-lang.com. And we also released blogs about on about Helion on the pytorch organization. I believe there's more news coming out today about us joining the foundation as well. Thank you very much.