Lightning Talk: Live Migration of PyTorch GPU Nodes From Azure To European Clouds - Mike Krom

This talk discusses the process of migrating PyTorch projects to European cloud providers, specifically for machine learning engineers concerned about compliance with GDPR and other regulations. The speaker, Mike from the Netherlands, outlines how he helps clients move their projects efficiently, often completing migrations from American clouds like Azure to European options such as Leaf Cloud or Civo Cloud in as little as 30 minutes. He explains the technical aspects of migration, including stopping the current virtual machine, exporting disks, and transforming OS disks for compatibility with different cloud environments. The speaker also mentions the development of a user-friendly tool that automates much of this process, providing an intuitive interface for selecting source and destination platforms, thus simplifying the transition to new cloud resources.

Welcome all to my lightning talk session. I hope to amaze you during this session about how to migrate your PyTorch projects to European cloud providers. Um So, now let's imagine you're like a machine learning engineer. You're working on your project. Then suddenly your security advisor is calling you up and says How secure is your your project running in an American cloud? Or maybe even worse if like

the GDPR compliant officer calls you and ask you like, "How How fast can you migrate your entire PyTorch project away from American technology and move that to European cloud providers?" I'm actually really curious like if they would ask that to you, how long do you think it would take to migrate your project to a European cloud provider? Is it like minutes, days, hours, months? Oh, that's quite

bad. Uh but before I will reveal my answer, let me first introduce myself. My name is Mike. I'm from the Netherlands. I run my own IT service company mostly for innovation teams and big data teams. So, what I usually do is they call me up and say, "Okay, I have problem with performance or my traffic get lost." Then I help them to to fix it. Uh but

like the last year, I got more and more the question from like, "How easy is it to migrate to European cloud And to give you the answer, for some of my customers, I migrated their PyTorch projects in like 5 minutes to 30 minutes max away from Azure to in some cases Leaf Cloud, in some cases Civo Cloud. And we are now trying to migrate also parts to

OVH's cloud. question then how do we do it so fast instead of like what you said months? How can we do it in in a day? Uh and it a little bit depends on of course how big your project is. What we usually see is that machine learning engineers are building their PyTorch code in a Python framework. Uh and the hardware is all the way on the

top and there are like a lot of different virtualization layers uh in between. Um so you can run your PyTorch in a container. You can run it on a virtual machine. Even some clouds virtualization technologies are in between. Uh sometimes a hypervisor and then there's the hardware. Uh so in case yeah one of my customers said, "Okay, I'm running this already for a year. I want to

move uh my whole project to a cloud which has newer GPUs. Please help us." And actually what we did was we did like a cut between the virtual machine layer and the cloud virtual station layer and moved the whole stack uh at once to a different hardware. Uh so in that way yeah we can migrate the whole PyTorch project from in this case Azure to Leaf Cloud.

Um yeah we built like a whole tool around it so to make it even more easier for the user. Uh so in this case this is like the the the the interface. They just select their source platform. They select their destination platform. They enter the virtual machine name. They click migrate and then the whole stack will be migrated to this new destination platform. I can also show

it how it works. So we have like different kind of options for your source platform. Different kind of options for your destination platform. yeah, you enter your uh VM name uh and you just click migrate and then yeah, we automate the whole process, so it will uh go. And what it actually does, it will uh stop your current virtual machine, so we get like a a fixed

state. Uh then we will export the the disks of this VM uh and then we will Yeah, sometimes we need to transform the OS disk uh for different cloud providers, but that's also done automatically. Uh and what we then do is we the exported disk, we import again in the other cloud environments. So actually you get like two copies of your virtual machine, one in the new

and one in the old uh clouds. Uh after that's done, we start a new virtual machine with the whole project uh and then the whole project runs in the different clouds. then it's up to the developers then it's up to the developers uh yeah, to test out this new environment. Uh if it's better, if something is missing or something doesn't work, then they can still go back

to the old one. If it works, they can go back to the new and that's actually in short what we did. Um so it's like a open source project. Uh so if you want to try it yourself, you can just go to our website. Um run the code, see if it works, uh and yeah, if you're happy, then um yeah, you run them on faster I do

this actually together with the IT Gilde. They are like open source community in the Netherlands. Uh and if you want, I can tell also story about the customer that we did migrate, but I Do I still have time for that? Uh thanks. Um so one uh company in Netherlands, it's a government agency. They had like innovation team of five uh big data and machine learning AI uh

experts. Um they had like a their own laptop, but they were not allowed to work on their laptop, or at least not allowed to install anything on their laptop. So, we got like a virtual machine where they were like installing all their different kind of machine learning tools, big data tools, visualization tools. but then yeah, they were already running for a year. Their VMs became slower. They

needed more GPUs. Uh they didn't want to install like other CUDA advanced parallelism options. So, what they suggested was they were trying to find like a different cloud provider. Uh in this case, we went to the Leafcloud platform. yeah, we run the tool in like 30 minutes, they were able to access the new VMs. Uh and as a result, they were quite happy because uh the price

uh was like halved because Azure was more expensive than Leafcloud. Uh also, the GPU amount was more in the Uh and on top of that, also their compliance officer was happy uh because now they were moving away from American cloud to European clouds. yeah, have slightly less issues with the GDPR rules and uh the American Cloud Act uh laws. Um so, yeah, that's my summary. Uh not

sure if anybody has any questions. And thanks for your time.