Lightning Talk: From Pretrained To Personal: Privacy-First... Daniel Holanda Noronha & Iswarya Alex

This talk focuses on fine-tuning machine learning models on AMD's AIPCs, highlighting the benefits of keeping data within institutions due to privacy concerns. The speaker discusses the barriers to sharing data, particularly in regulated industries such as healthcare and finance. With advancements in hardware and techniques like LoRA and QLoRA, fine-tuning has become more accessible and cost-effective, allowing institutions to enhance model performance without needing cloud resources. The session covers efficient methods for model training, adaptation methods involving user preferences and labels, and the role of frameworks like PyTorch with ROCm. The speaker also presents experimental results from fine-tuning models on AMD's Ryzen AI Max, demonstrating the capability of local AIPCs to manage demanding tasks previously limited to cloud environments.

Good afternoon. Uh my name is Daniel. I work at AMD. I'm a machine learning engineer there, and I'm very excited to be here today to talk about fine-tuning on AIPCs and why your next fine-tune may never touch the cloud. So, I'd like to start this presentation in a different way with a rhetorical question to the audience, which is, what would you fine-tune if sharing data outside of

your institution was possible? What would you fine-tune institution was not such a terrible pain? Uh there's so many things that could be done, but most of us would not even entertain this question because, well, privacy is often non-negotiable. The data that you mostly care about, the data that you mostly want to fine-tune on, is the data that you can't really share. And that's a big problem, right?

Many industries are highly regulated, as they should. We have health care data, government and public sector workloads, public uh financial services, proprietary enterprise systems, all of those very highly regulated. And there is this inability to share data externally. It may come from many different uh fronts. It may come externally for regulatory reasons, as we discussed, but it may also come because you have customers. Your customers do

not want you sharing their data around. They trust that you have the data. They don't want you sharing the data. It may also come from inside of your institution. Maybe your institution is very uh they really think that they have a special sauce that they want they don't want you to share outside of the institution. Maybe they don't trust cloud providers as much as they actually uh

should, and they are way too worried about security risks and liability. All of that to say that we really created a system here that is based on scale. It's optimized for scale. It's not optimized for privacy. And as a result here, most companies use out-of-the-box models that are not tuned for their data and use cases. there's a big shift that is happening now that may actually allow

us to go around this, which is that fine-tuning is finally becoming much cheaper, not only in terms of cloud cost, but actually in terms of compute costs. What this means is that you can now use PyTorch on AIPCs to actually do meaningful fine-tuning, not only inference. This is a capability shift. It's not an incremental improvement. So, in other words, you have your data, it's inside of your

institution, you do your fine-tuning, data never goes to the cloud. You can do everything there inside, everything is safe and secure. And to enable this, there are really three things that happened there that allow us to basically do this capability shift. One is efficient methods. There's no reason for you to basically do full fine-tuning if we have amazing methods like Laura and QLoRA out there, and I'll

touch on those in a second. Second thing is the hardware evolved quite a bit. We have now hardware that has amazing MPUs and GPUs on AIPCs, and we can use those for fine-tuning, which is actually amazing. And finally, efficient frameworks. If you have PyTorch with ROCm or many of the frameworks that are built on top of this, you can actually do meaningful fine-tuning. So, my quick proposal

here for the this talk is the following. I'm going to be talking about some of those I'm going to be talking about those efficient frameworks, and then I'm going to show you some results to hopefully get some of you here excited about fine-tuning on your local AIPCs. Sounds good? Okay. first thing that you have to do when you're thinking about fine-tuning your own models is you have

to choose the right method. There's no one-size-fits-all approach. The right method actually depends on what you're trying to do. Most intuitive method of the models full fine-tuning where you're basically going and you're updating all of the parameters of your model. This is gives you the best quality, but it's not necessarily something that is going to give you the best usage out of your local IPC because this

requires a lot of uh compute, requires a lot of memory, right? That being said, there are many other methods that are very interesting out there, such as LoRAs, which is a parameter-efficient fine-tuning technique. In LoRAs, we basically are training small adapter matrices while freezing the rest of the model. So, effectively, we're only uh training something that corresponds to like 1 to 5% of the model, but we're

still keeping 95 to 98% of the quality. So, for local AI PC, fine-tuning is a method that ends up being very popular. We didn't have QLoRA, which is basically a small variant of LoRA where we're freezing the actual model into four-bit quantization. We still have those LoRA adapters. Uh so, the main takeaway here is that this is even more efficient than LoRA themselves. That said, uh there

are some quality tradeoffs there to be noted. When you're doing fine-tuning on a local AI PC, you're usually doing LoRA or QLoRA, so you're uh often not doing full fine-tuning. Now, once you understand what's the tuning method that you want to use, then you have to understand what's the adaptation method that you want to use. Tuning methods is mostly about which weights updated, while adaptation methods is

like how behavior is aligned. Is behavior aligned based on some label prompt-response pairs, or is it aligned based on a model that we created that basically basically mimics user preference, right? So, a method that was very used in the past and still used today up to some extent is SFD or supervised fine-tuning, where we are basically training the model on label prompt-response pairs. In terms of doing

this on an AI PC, the VRAM required is very low, but at the same time uh there are better methods out there that's going to they're basically going to give you even better quality. One of it The example of those methods is uh is reinforcement learning from human feedback, where we have a reward model that is trained on human preference, and we use this to basically guide

the policy updates. Now, the issue with this type of uh fine-tuning is that we do have a policy model, we have a reward model, sometimes we have a reference model. And as you can think, this requires a lot of VRAM, requires a lot of forward passes. So, although it may give you amazing quality, this is not always feasible if you're doing this on any AI PC. You

have to be very careful on how you actually do this. Um and the final method that I want to talk about here is DPO or direct preference optimization. Here, we're directly optimizing the model on preference pairs rather than actually using reinforcement learning. So, as a result here, we end up in something that still has very high quality, but in terms of VRAM usage, it's somewhere in between

SFT and RLHF. Now, we learned about the tuning methods. We talked about adaptation methods. Now, what we have to choose is like, okay, which framework can actually use to run those methods? We're at the PyTorch conference, so many of you will just go and use PyTorch directly. This perfectly works if you run If you have PyTorch today and you run this with Rockum, you can do full

fine-tuning, LoRA, QLoRA, all of those are available for you there, not only on the server side, but also on the local side if you're using your cloud if you're using your AIPCs. That said, many users they do prefer a slightly more guided experience. So, folks like Unsloth and Llama Factory, they're actually doing an amazing job at creating something that is a little bit more user-friendly. So, if

you prefer using a UI, for example, Llama Factory is a great option. Unsloth is a great option, and they're both doing a lot of optimizations there to allow you to get started very quickly with fine-tuning on device. turns out I'm going quite fast, but this is good. Before I end this presentation, I'd like to show you some numbers as motivations here so you guys can get excited

about fine-tuning on your own AIPCs. So, what I did here is that I got a Strix Halo. Strix Halo is the code name that we use inside of AMD for our Ryzen AI Max plus 395. Uh it's a machine that has 128 gigs of RAM, but it's not something that sits on a server like this laptop here, and I don't know if I want to hold it

too high because it's HDMI's connected. It has one of those, right? So, it's not something that sits on a server, stuff that you can have here in a laptop. You can have this in a tiny box at home. So, it's it it's for local AIPCs. And it goes up to 120 watts, which is quite impressive as well. So, what I did here is that I installed by

torch 2.10 in Rockum 7.12, and I decided to do some experiments. Those experiments were done with both Laura's and full fine-tuning. So, Laura's is basically what is seen here as blue. Full fine-tuning is what is seen here as red {slash} orange. Um and what I did here is that I fine-tuned a variety of models. I got the coin 3.5 models because they came out recently. I think

that they're awesome. And I decided to do fine-tuning on all of the models from 0.8 all the way to 27B. As you can imagine that that makes a lot of sense. The curve that you see here just means like sure, as you increase in the size of the model, the amount of VRAM required and the time to actually fine-tune the model increase over time. That makes a

lot of sense, but I think that one of the main takeaways that I'd like you to see here is that even on a local device, which is like this laptop here, we're able to train a model that is coin 3.5 27B, which if you didn't try is a very capable model. Right? Uh that said, this is only 100 optimizer steps. This is a fairly small data sets

that we chose here, but if you were to keep running this for a few hours or even overnight, you can actually do some meaningful fine-tuning on your AI PC, which I think is amazing because like just a few years ago, we're only able to do this in big cloud servers. So, I think this is really a big shift that we're seeing here. Now, before I end this

presentation, the only thing that I want to do here is to make an announcement. This is the first time that AMD's publicly talking about this. At the end of this month, AMD's going to publish a thing that is called AMD Playbooks. This is going to be a set of guided experiences that are going to basically help many folks to become AI developers. This is going to have

a lot of very interesting content related to fine-tuning as you can see here, like fine-tuning with Rockum, Lama Factory, Unsloth, and so on, as well as many other things. So, if you want to know more about this as soon as it's out, please sign up for our developer program. And that's me. Thank you very much.