Keynote: Co-Evolution: How the Open Source Intelligence Stack Compounds - Mark Collier

This talk marks the inaugural PyTorch conference in Europe and emphasizes the significance of the PyTorch Foundation in fostering open-source AI. The speaker highlights the foundation's support for projects like PyTorch, vLLM, Ray, and DeepSpeed, and discusses their efforts to expand capabilities and introduce new projects. There is a strong focus on collaboration within the community, which includes over 600 participants from 250 organizations and 45 countries. The speaker underscores the unique opportunity presented by the simultaneous evolution of hardware and software architectures in the AI sector. Additionally, the introduction of Healion, a new domain-specific language for building fast kernels, showcases the ongoing innovation within the PyTorch ecosystem.

Bonjour, welcome everyone to the first ever PyTorch conference in Europe. Now, every PyTorch conference to date has been in Silicon Valley and I think it's way past time that we have one here in Europe. So, I'm super excited to be involved and able to to kick it off today. Um we do have a lot of sponsors that we want to thank and you'll be able to meet

them all out in the area outside. I think you are all meandering through earlier, but I want to especially call out AMD, the Cloud Native Computing Foundation, Meta, and Red Hat. They'll also be talking later today and tomorrow and giving keynotes that you'll be able to hear from. And we have a number of sponsors, as I said, you'll be able to meet throughout the day. We definitely

want to thank our programming committee. The programming chairs are Alban, Lisandra, and Luca. They're all out here somewhere. So, let's give them a round of applause. They put so much hard work into putting this together. And what an amazing venue. I'm so glad we're we're here in Paris. Now, we have the community expo upstairs again and that's where you'll find, of course, the sponsors, the birds of

the feather, there are opportunities to meet the developers and poster presentations. Another reminder, some of the activities we have going today, the women and non-binary in PyTorch lunch. And then at the end of the day, we'll be having the flare party. So, you don't want to miss that. That'll be right here in the building as well. And as a foundation, the PyTorch Foundation, we are supported by

a ton of great companies from around the industry and around the world and we've just added nine new members this year alone, just since the start of the year. So, the foundation's been growing fast and as a foundation, you know, we really have one primary purpose, which is to serve this community. And that really means our projects. And some of you may be aware, but maybe not

all, that the PyTorch Foundation actually supports four projects today. PyTorch, of course, the name on the door. But also vLLM, Ray, and DeepSpeed. And when we think about the PyTorch Foundation and our mission this year and what we need to do to really meet the moment as the industry just continues to have higher and higher ambitions for for what we can do in open-source AI and the

role we can play in it, I really think there are three specific areas where we need to be thinking about um doing our part. One is within the existing projects that we host, adding new capabilities. And I think you'll be hearing about that throughout this week. And this morning, in particular, um we also need to add new projects. So, we uh last year started to host additional

projects and have four today, but I think by the end of this year we'll have quite a few more. So, I think that's going to be part of how we do our part as a foundation to move the industry forward. And third, a very important one is to work across communities and across foundations. I mentioned earlier the Cloud Native Computing Foundation is is one of our headline

sponsors and will be will be speaking tomorrow, so it's important that we work across communities with the other open source that's out there. Um now, this community is an incredible uh diverse group of folks and just at this event alone, we have well over 600 people, but more importantly than just the total number, we have people from 250 different organizations and the my favorite stat, there people

here from over 45 countries. And there's really nowhere else besides open source where you get to work with people from around the world to solve hard problems. And that's really special and I think that we all have to like be grateful we can be a part of that. Um, also I I skipped over the uh uh the survey, but we did ask what topics you all wanted

to discuss, and over half of you said agents, more than 70% said training, and the number one answer was actually inference, which is a good sign that I think we're really focusing on production and a lot of the hard work and hard problems to be solved in the inference layer. Now, as far as community goes, um I've been involved in open source community for a long time,

almost 20 years, but I am new to PyTorch, and in my short time of trying to get to know some of you and get to know this community, I've really tried to figure out what is it that makes this community special. Because there's really something going on here that you don't necessarily see every day um when you're working with different communities and different uh industries. And I

think I have kind of a hypothesis, but I'll I'll run it by you. You can You can tell me if you think I'm on the right track, but I really think that a big reason this community's different is because I think you're trying to do things that have never been done before, ever. And just to put it in context, I think part of what's happening is that

we have this insane level of capital that's going in to build 100 gigawatts of data centers, all this incredible hardware that's being built by a lot of the people in this room, a lot of the companies that are here. But at the end of the day, all that hardware, I think we all know, is pretty useless without the software, right? But what's really unique about this moment

is I think this is the first time in a really long time when we're trying to design and update the hardware architecture and software architectures all at the same time, and not in some vertically integrated way, like, you know, an Apple or going back in the mainframe days, but really globally across thousands of companies. And if you think about the hardware side, we have GPUs, TPUs, custom

ASICs. I mean, it's incredible complexity and innovation trying to serve the market from training inference agents. In the software side, every new model, you need to update your inference framework, right? New model architecture. So, we're really trying to do this all at once. It's a massive undertaking. And even on the software side, the way software's getting written is changing, right? Ever since, you know, Opus 4.5 and

Claude code and the sort of Claude Christmas that took everybody by surprise, people are starting to really lean into to coding agents. And even in the hardware side, we're developing uh new hardware with some of these tools as And so, what I think is important to take away from this is that this is a coordination problem. When you're trying to coordinate worldwide hardware and software and you're

changing all the architectures at once, you really need it to massively cooperate and coordinate. And the only way to do that is open source. I don't think there is another way, actually. And so, when you think about where that open source coordination's happening and what communities are doing the hard work, and like I said, I think trying to pull off something that's never been done before, I

would say that PyTorch is absolutely at the epicenter of this. So, this project with 12,000 plus contributors over, you know, the history of the project, 2,000 organizations, you know, I went through that list and just tried to identify and I'm sure I I couldn't have even fit them all on the slide, but all of the companies that are in some way involved in building all these AI

accelerators contributing upstream to PyTorch. I mean, that's incredible. This is why this work is happening here in this community and why we're trying to pull before in terms of hardware and software co-evolving together. And certainly in the inference world, VLLM, extremely popular, incredible pace. The VLM community just blows me away how quickly they release a new version when a new model comes out, which seems really like

like it's happening daily. And And once again, you see the who's who of companies that are building accelerators, their path to market is through VLM and PyTorch. And we welcome all those companies in the community, and they contribute, and that's what makes it all all go around. And lastly, I'll mention Ray. So, uh Ray does a lot of things, and there's talks on Ray later this week,

and you can learn a lot more about it. But in particular, RL is extremely uh popular for reinforcement learning. And a couple of concrete examples, um Uber has recently disclosed that they are training and running specialized models, thousands of them. They're using Ray. And uh the other example is Cursor. So, they've recently been kind of doing RL fine-tuning on these Chem E base models, and there's been

a lot of talk about how they're doing that for coding. And they recently said that they are now doing new version every 5 hours. So, that's a huge shift from the idea of, you know, models that you might be training for 6 months. If you're releasing a new checkpoint every 5 hours, that's pretty That's pretty wild. And so, you know, I said earlier that we need to

do three things, one of which is add capabilities to our existing projects. And I'm really excited to announce that today we're announcing ExecuTorch becoming part of PyTorch Core. And this is a exciting addition. There's going to be a talk later this afternoon to learn more about ExecuTorch, but it's really uh helpful in edge inference, and there's a lot more to it, but we're going to go ahead

and welcome up some other speakers to talk about the second way that we're going to be improving open-source AI this year, which is by adding new projects to the PyTorch Foundation. So, I would like to welcome up the folks from the Healion team to talk about the newest PyTorch Foundation project. Tell us all about it. Yeah, thanks so much. Uh Healion is a new DSL for building

uh fast kernels. Um it's a uh a higher-level DSL than a lot of others out there like Triton, um but it's a little it lower level than PyTorch. You could sort of think of it like PyTorch with tiles. Makes it very easy um for everyone to build custom kernels that do exactly what you want, have have high degree of control over numerics. And we're really thrilled to

be joining the PyTorch Foundation as we think neutral government governance is critical for open-source projects, especially for programming languages where you want to build communities that span across many different companies. Yeah, very very grateful to join the foundation. We'll be with this joining we'll be increasing our uh like capabilities in different backends. We'll be going from supporting GPUs to TPUs and other accelerators. And then we'll be

having more neutral governance so that like every uh different company can participate in the governance of Healion. And then we'll be introducing more uh maintainers. Thank you very Thank you. Yeah, and you should catch Ozan's talk later today. Or Yes, at 11:00 a.m. I will be giving a talk about Healion and all the new capabilities we have built since the PyTorch conference uh in San Francisco last

year. All right, THANK YOU VERY MUCH. ALL RIGHT, SO JUST TO WRAP THINGS UP, I think we're going to continue to work within this community and across with other communities to make sure this open flywheel continues to do the incredible progress we've seen and we know is possible when we co-develop and co-evolve with hardware and software and the whole industry all at once. And that's what we're

here to do. So, I think we're going to keep seeing this co-evolution, but if we commit to doing it together and doing it in the open and doing it as a community, we're going to see incredible progress.